Skip to main content

DeepInfra

DeepInfra allows us to run the latest machine learning models with ease. DeepInfra takes care of all the heavy lifting related to running, scaling and monitoring the models. Users can focus on your application and integrate the models with simple REST API calls.

DeepInfra provides examples of integration with LangChain.

This page covers how to use the DeepInfra ecosystem within LangChain. It is broken into two parts: installation and setup, and then references to specific DeepInfra wrappers.

Installation and Setup​

  • Get your DeepInfra api key from this link here.
  • Get an DeepInfra api key and set it as an environment variable (DEEPINFRA_API_TOKEN)

Available Models​

DeepInfra provides a range of Open Source LLMs ready for deployment.

You can see supported models for text-generation and embeddings.

You can view a list of request and response parameters.

Chat models follow openai api

LLM​

See a usage example.

from langchain_community.llms import DeepInfra
API Reference:DeepInfra

Embeddings​

See a usage example.

from langchain_community.embeddings import DeepInfraEmbeddings
API Reference:DeepInfraEmbeddings

Chat Models​

See a usage example.

from langchain_community.chat_models import ChatDeepInfra
API Reference:ChatDeepInfra

Was this page helpful?