Integrate with Langchain

[Sharpz7]

Just wanted to add this here.

langchain-ai/langchain#8563

Ideally we would write it into langchain in a way that the user can choose the URL of the endpoint they want to use (Since its not recommended to use chat.petals.dev)

[borzunov]

For the record, there is an existing integration by Langchain devs that runs the native Petals client: https://python.langchain.com/docs/integrations/llms/petals This connects to the swarm directly (without using this API endpoint), but requires downloading input/output embeddings of the model (a few GB) before running.

@Sharpz7, do you think Petals API support will be still helpful, even though Langchain provides integration for the native client?

[Sharpz7]

Hmm. I think for applications where space is limited potentially? I think having more options rather than less is still useful to have.

Also just to lower the barrier to entry, that is something that using the petals API directly would give you.