Skip to main contentğŸ¦œï¸�ğŸ”— LangChainDocsUse casesIntegrationsAPILangSmithJS/TS DocsCTRLKIntegrationsCallbacksChat modelsDocument loadersDocument transformersLLMsAI21Aleph AlphaAmazon API GatewayAnyscaleAzure OpenAIAzureML Online EndpointBananaBasetenBeamBedrockCerebriumAIChatGLMClarifaiCohereC TransformersDatabricksDeepInfraForefrontAIGoogle Cloud Platform Vertex AI PaLMGooseAIGPT4AllHugging Face HubHugging Face Local PipelinesHuggingface TextGen InferenceJSONFormerKoboldAI APILlama-cppCaching integrationsManifestModalMosaicMLNLP CloudoctoaiOpenAIOpenLLMOpenLMPetalsPipelineAIPredibasePrediction GuardPromptLayer OpenAIRELLMReplicateRunhouseSageMakerEndpointStochasticAITextGenTongyi QwenWriterMemoryRetrieversText embedding modelsAgent toolkitsToolsVector storesGrouped by providerIntegrationsLLMsPrediction GuardOn this pagePrediction Guardpip install predictionguard langchainimport osimport predictionguard as pgfrom langchain.llms import PredictionGuardfrom langchain import PromptTemplate, LLMChainBasic LLM usageâ€‹# Optional, add your OpenAI API Key. This is optional, as Prediction Guard allows# you to access all the latest open access models (see https://docs.predictionguard.com)os.environ["OPENAI_API_KEY"] = "<your OpenAI api key>"# Your Prediction Guard API key. Get one at predictionguard.comos.environ["PREDICTIONGUARD_TOKEN"] = "<your Prediction Guard access token>"pgllm = PredictionGuard(model="OpenAI-text-davinci-003")pgllm("Tell me a joke")Control the output structure/ type of LLMsâ€‹template = """Respond to


the output structure/ type of LLMsâ€‹template = """Respond to the following query based on the context.Context: EVERY comment, DM + email suggestion has led us to this EXCITING announcement! ğŸ�‰ We have officially added TWO new candle subscription box options! ğŸ“¦Exclusive Candle Box - $80 Monthly Candle Box - $45 (NEW!)Scent of The Month Box - $28 (NEW!)Head to stories to get ALLL the deets on each box! ğŸ‘† BONUS: Save 50% on your first box with code 50OFF! ğŸ�‰Query: {query}Result: """prompt = PromptTemplate(template=template, input_variables=["query"])# Without "guarding" or controlling the output of the LLM.pgllm(prompt.format(query="What kind of post is this?"))# With "guarding" or controlling the output of the LLM. See the# Prediction Guard docs (https://docs.predictionguard.com) to learn how to# control the output with integer, float, boolean, JSON, and other types and# structures.pgllm = PredictionGuard(    model="OpenAI-text-davinci-003",    output={        "type": "categorical",        "categories": ["product announcement", "apology", "relational"],    },)pgllm(prompt.format(query="What kind of post is this?"))Chainingâ€‹pgllm = PredictionGuard(model="OpenAI-text-davinci-003")template = """Question: {question}Answer: Let's think step by step."""prompt = PromptTemplate(template=template, input_variables=["question"])llm_chain = LLMChain(prompt=prompt, llm=pgllm, verbose=True)question =


= LLMChain(prompt=prompt, llm=pgllm, verbose=True)question = "What NFL team won the Super Bowl in the year Justin Beiber was born?"llm_chain.predict(question=question)template = """Write a {adjective} poem about {subject}."""prompt = PromptTemplate(template=template, input_variables=["adjective", "subject"])llm_chain = LLMChain(prompt=prompt, llm=pgllm, verbose=True)llm_chain.predict(adjective="sad", subject="ducks")PreviousPredibaseNextPromptLayer OpenAIBasic LLM usageControl the output structure/ type of LLMsChainingCommunityDiscordTwitterGitHubPythonJS/TSMoreHomepageBlogCopyright Â© 2023 LangChain, Inc.


Petals | ğŸ¦œï¸�ğŸ”— Langchain


Skip to main contentğŸ¦œï¸�ğŸ”— LangChainDocsUse casesIntegrationsAPILangSmithJS/TS DocsCTRLKIntegrationsCallbacksChat modelsDocument loadersDocument transformersLLMsAI21Aleph AlphaAmazon API GatewayAnyscaleAzure OpenAIAzureML Online EndpointBananaBasetenBeamBedrockCerebriumAIChatGLMClarifaiCohereC TransformersDatabricksDeepInfraForefrontAIGoogle Cloud Platform Vertex AI PaLMGooseAIGPT4AllHugging Face HubHugging Face Local PipelinesHuggingface TextGen InferenceJSONFormerKoboldAI APILlama-cppCaching integrationsManifestModalMosaicMLNLP CloudoctoaiOpenAIOpenLLMOpenLMPetalsPipelineAIPredibasePrediction GuardPromptLayer OpenAIRELLMReplicateRunhouseSageMakerEndpointStochasticAITextGenTongyi QwenWriterMemoryRetrieversText embedding modelsAgent toolkitsToolsVector storesGrouped by providerIntegrationsLLMsPetalsOn this pagePetalsPetals runs 100B+ language models at home, BitTorrent-style.This notebook goes over how to use Langchain with Petals.Install petalsâ€‹The petals package is required to use the Petals API. Install petals using pip3 install petals.pip3 install petalsImportsâ€‹import osfrom langchain.llms import Petalsfrom langchain import PromptTemplate, LLMChainSet the Environment API Keyâ€‹Make sure to get your API key from Huggingface.from getpass import getpassHUGGINGFACE_API_KEY = getpass()     Â·Â·Â·Â·Â·Â·Â·Â·os.environ["HUGGINGFACE_API_KEY"] = HUGGINGFACE_API_KEYCreate the Petals instanceâ€‹You can specify different parameters


HUGGINGFACE_API_KEYCreate the Petals instanceâ€‹You can specify different parameters such as the model name, max new tokens, temperature, etc.# this can take several minutes to download big files!llm = Petals(model_name="bigscience/bloom-petals")    Downloading:   1%|â–�                        | 40.8M/7.19G [00:24<15:44, 7.57MB/s]Create a Prompt Templateâ€‹We will create a prompt template for Question and Answer.template = """Question: {question}Answer: Let's think step by step."""prompt = PromptTemplate(template=template, input_variables=["question"])Initiate the LLMChainâ€‹llm_chain = LLMChain(prompt=prompt, llm=llm)Run the LLMChainâ€‹Provide a question and run the LLMChain.question = "What NFL team won the Super Bowl in the year Justin Beiber was born?"llm_chain.run(question)PreviousOpenLMNextPipelineAIInstall petalsImportsSet the Environment API KeyCreate the Petals instanceCreate a Prompt TemplateInitiate the LLMChainRun the LLMChainCommunityDiscordTwitterGitHubPythonJS/TSMoreHomepageBlogCopyright Â© 2023 LangChain, Inc.


ChatGLM | ğŸ¦œï¸�ğŸ”— Langchain


Skip to main contentğŸ¦œï¸�ğŸ”— LangChainDocsUse casesIntegrationsAPILangSmithJS/TS DocsCTRLKIntegrationsCallbacksChat modelsDocument loadersDocument transformersLLMsAI21Aleph AlphaAmazon API GatewayAnyscaleAzure OpenAIAzureML Online EndpointBananaBasetenBeamBedrockCerebriumAIChatGLMClarifaiCohereC TransformersDatabricksDeepInfraForefrontAIGoogle Cloud Platform Vertex AI PaLMGooseAIGPT4AllHugging Face HubHugging Face Local PipelinesHuggingface TextGen InferenceJSONFormerKoboldAI APILlama-cppCaching integrationsManifestModalMosaicMLNLP CloudoctoaiOpenAIOpenLLMOpenLMPetalsPipelineAIPredibasePrediction GuardPromptLayer OpenAIRELLMReplicateRunhouseSageMakerEndpointStochasticAITextGenTongyi QwenWriterMemoryRetrieversText embedding modelsAgent toolkitsToolsVector storesGrouped by providerIntegrationsLLMsChatGLMChatGLMChatGLM-6B is an open bilingual language model based on General Language Model (GLM) framework, with 6.2 billion parameters. With the quantization technique, users can deploy locally on consumer-grade graphics cards (only 6GB of GPU memory is required at the INT4 quantization level). ChatGLM2-6B is the second-generation version of the open-source bilingual (Chinese-English) chat model ChatGLM-6B. It retains the smooth conversation flow and low deployment threshold of the first-generation model, while introducing the new features like better performance, longer context and more efficient inference.This example goes over how to use LangChain to interact with ChatGLM2-6B Inference for text completion.


ChatGLM-6B and ChatGLM2-6B has the same api specs, so this example should work with both.from langchain.llms import ChatGLMfrom langchain import PromptTemplate, LLMChain# import ostemplate = """{question}"""prompt = PromptTemplate(template=template, input_variables=["question"])# default endpoint_url for a local deployed ChatGLM api serverendpoint_url = "http://127.0.0.1:8000"# direct access endpoint in a proxied environment# os.environ['NO_PROXY'] = '127.0.0.1'llm = ChatGLM(    endpoint_url=endpoint_url,    max_token=80000,    history=[["æˆ‘å°†ä»�ç¾�å›½åˆ°ä¸å›½æ�¥æ—…æ¸¸ï¼Œå‡ºè¡Œå‰�å¸Œæœ›äº†è§£ä¸å›½çš„åŸ�å¸‚", "æ¬¢è¿�é—®æˆ‘ä»»ä½•é—®é¢˜ã€‚"]],    top_p=0.9,    model_kwargs={"sample_model_args": False},)# turn on with_history only when you want the LLM object to keep track of the conversation history# and send the accumulated context to the backend model api, which make it stateful. By default it is stateless.# llm.with_history = Truellm_chain = LLMChain(prompt=prompt, llm=llm)question =


Truellm_chain = LLMChain(prompt=prompt, llm=llm)question = "åŒ—äº¬å’Œä¸Šæµ·ä¸¤åº§åŸ�å¸‚æœ‰ä»€ä¹ˆä¸�å�Œï¼Ÿ"llm_chain.run(question)    ChatGLM payload: {'prompt': 'åŒ—äº¬å’Œä¸Šæµ·ä¸¤åº§åŸ�å¸‚æœ‰ä»€ä¹ˆä¸�å�Œï¼Ÿ', 'temperature': 0.1, 'history': [['æˆ‘å°†ä»�ç¾�å›½åˆ°ä¸å›½æ�¥æ—…æ¸¸ï¼Œå‡ºè¡Œå‰�å¸Œæœ›äº†è§£ä¸å›½çš„åŸ�å¸‚', 'æ¬¢è¿�é—®æˆ‘ä»»ä½•é—®é¢˜ã€‚']], 'max_length': 80000, 'top_p': 0.9, 'sample_model_args': False}


'åŒ—äº¬å’Œä¸Šæµ·æ˜¯ä¸å›½çš„ä¸¤ä¸ªé¦–éƒ½ï¼Œå®ƒä»¬åœ¨è®¸å¤šæ–¹é�¢éƒ½æœ‰æ‰€ä¸�å�Œã€‚\n\nåŒ—äº¬æ˜¯ä¸å›½çš„æ”¿æ²»å’Œæ–‡åŒ–ä¸å¿ƒï¼Œæ‹¥æœ‰æ‚ ä¹…çš„å�†å�²å’Œç�¿çƒ‚çš„æ–‡åŒ–ã€‚å®ƒæ˜¯ä¸å›½æœ€é‡�è¦�çš„å�¤éƒ½ä¹‹ä¸€ï¼Œä¹Ÿæ˜¯ä¸å›½å�†å�²ä¸Šæœ€å��ä¸€ä¸ªå°�å»ºç�‹æœ�çš„éƒ½åŸ�ã€‚åŒ—äº¬æœ‰è®¸å¤šè‘—å��çš„å�¤è¿¹å’Œæ™¯ç‚¹ï¼Œä¾‹å¦‚ç´«ç¦�åŸ�ã€�å¤©å®‰é—¨å¹¿åœºå’Œé


å¤©å®‰é—¨å¹¿åœºå’Œé•¿åŸ�ç‰ã€‚\n\nä¸Šæµ·æ˜¯ä¸å›½æœ€ç�°ä»£åŒ–çš„åŸ�å¸‚ä¹‹ä¸€ï¼Œä¹Ÿæ˜¯ä¸å›½å•†ä¸šå’Œé‡‘è��ä¸å¿ƒã€‚ä¸Šæµ·æ‹¥æœ‰è®¸å¤šå›½é™…çŸ¥å��çš„ä¼�ä¸šå’Œé‡‘è��æœºæ�„ï¼Œå�Œæ—¶ä¹Ÿæœ‰è®¸å¤šè‘—å��çš„æ™¯ç‚¹å’Œç¾�é£Ÿã€‚ä¸Šæµ·çš„å¤–æ»©æ˜¯ä¸€ä¸ªå�†å�²æ‚ ä¹…çš„å•†ä¸šåŒºï¼Œæ‹¥æœ‰è®¸å¤šæ¬§å¼�å»ºç‘å’Œé¤�é¦†ã€‚\n\né™¤æ¤ä¹‹å¤–ï¼ŒåŒ—äº¬å’Œä¸Šæµ·åœ¨äº¤é€šå’


ä¸Šæµ·åœ¨äº¤é€šå’Œäººå�£æ–¹é�¢ä¹Ÿæœ‰å¾ˆå¤§å·®å¼‚ã€‚åŒ—äº¬æ˜¯ä¸å›½çš„é¦–éƒ½ï¼Œäººå�£ä¼—å¤šï¼Œäº¤é€šæ‹¥å µé—®é¢˜è¾ƒä¸ºä¸¥é‡�ã€‚è€Œä¸Šæµ·æ˜¯ä¸å›½çš„å•†ä¸šå’Œé‡‘è��ä¸å¿ƒï¼Œäººå�£å¯†åº¦è¾ƒä½�ï¼Œäº¤é€šç›¸å¯¹è¾ƒä¸ºä¾¿åˆ©ã€‚\n\næ€»çš„æ�¥è¯´ï¼ŒåŒ—äº¬å’Œä¸Šæµ·æ˜¯ä¸¤ä¸ªæ‹¥æœ‰ç‹¬ç‰¹é…åŠ›å’Œç‰¹ç‚¹çš„åŸ�å¸‚ï¼Œå�¯ä»¥æ ¹æ�®è‡ªå·±çš„å…´è¶£å’Œæ—¶é—´æ�¥é€‰æ‹©å‰


Œæ—¶é—´æ�¥é€‰æ‹©å‰�å¾€å…¶ä¸ä¸€åº§åŸ�å¸‚æ—…æ¸¸ã€‚'PreviousCerebriumAINextClarifaiCommunityDiscordTwitterGitHubPythonJS/TSMoreHomepageBlogCopyright


Â© 2023 LangChain, Inc.


Predibase | ğŸ¦œï¸�ğŸ”— Langchain


Skip to main contentğŸ¦œï¸�ğŸ”— LangChainDocsUse casesIntegrationsAPILangSmithJS/TS DocsCTRLKIntegrationsCallbacksChat modelsDocument loadersDocument transformersLLMsAI21Aleph AlphaAmazon API GatewayAnyscaleAzure OpenAIAzureML Online EndpointBananaBasetenBeamBedrockCerebriumAIChatGLMClarifaiCohereC TransformersDatabricksDeepInfraForefrontAIGoogle Cloud Platform Vertex AI PaLMGooseAIGPT4AllHugging Face HubHugging Face Local PipelinesHuggingface TextGen InferenceJSONFormerKoboldAI APILlama-cppCaching integrationsManifestModalMosaicMLNLP CloudoctoaiOpenAIOpenLLMOpenLMPetalsPipelineAIPredibasePrediction GuardPromptLayer OpenAIRELLMReplicateRunhouseSageMakerEndpointStochasticAITextGenTongyi QwenWriterMemoryRetrieversText embedding modelsAgent toolkitsToolsVector storesGrouped by providerIntegrationsLLMsPredibaseOn this pagePredibasePredibase allows you to train, finetune, and deploy any ML modelâ€”from linear regression to large language model. This example demonstrates using Langchain with models deployed on PredibaseSetupTo run this notebook, you'll need a Predibase account and an API key.You'll also need to install the Predibase Python package:pip install predibaseimport osos.environ["PREDIBASE_API_TOKEN"] = "{PREDIBASE_API_TOKEN}"Initial Callâ€‹from langchain.llms import Predibasemodel = Predibase(    model="vicuna-13b", predibase_api_key=os.environ.get("PREDIBASE_API_TOKEN"))response = model("Can you recommend me a nice dry wine?")print(response)Chain Call Setupâ€‹llm = Predibase(


wine?")print(response)Chain Call Setupâ€‹llm = Predibase(    model="vicuna-13b", predibase_api_key=os.environ.get("PREDIBASE_API_TOKEN"))SequentialChainâ€‹from langchain.chains import LLMChainfrom langchain.prompts import PromptTemplate# This is an LLMChain to write a synopsis given a title of a play.template = """You are a playwright. Given the title of play, it is your job to write a synopsis for that title.Title: {title}Playwright: This is a synopsis for the above play:"""prompt_template = PromptTemplate(input_variables=["title"], template=template)synopsis_chain = LLMChain(llm=llm, prompt=prompt_template)# This is an LLMChain to write a review of a play given a synopsis.template = """You are a play critic from the New York Times. Given the synopsis of play, it is your job to write a review for that play.Play Synopsis:{synopsis}Review from a New York Times play critic of the above play:"""prompt_template = PromptTemplate(input_variables=["synopsis"], template=template)review_chain = LLMChain(llm=llm, prompt=prompt_template)# This is the overall chain where we run these two chains in sequence.from langchain.chains import SimpleSequentialChainoverall_chain = SimpleSequentialChain(    chains=[synopsis_chain, review_chain], verbose=True)review = overall_chain.run("Tragedy at sunset on the beach")Fine-tuned LLM (Use your own fine-tuned LLM from Predibase)â€‹from langchain.llms import Predibasemodel = Predibase(    model="my-finetuned-LLM", predibase_api_key=os.environ.get("PREDIBASE_API_TOKEN"))# replace my-finetuned-LLM with the name of your model in


replace my-finetuned-LLM with the name of your model in Predibase# response = model("Can you help categorize the following emails into positive, negative, and neutral?")PreviousPipelineAINextPrediction GuardInitial CallChain Call SetupSequentialChainFine-tuned LLM (Use your own fine-tuned LLM from Predibase)CommunityDiscordTwitterGitHubPythonJS/TSMoreHomepageBlogCopyright Â© 2023 LangChain, Inc.


AI21 | ğŸ¦œï¸�ğŸ”— Langchain


Skip to main contentğŸ¦œï¸�ğŸ”— LangChainDocsUse casesIntegrationsAPILangSmithJS/TS DocsCTRLKIntegrationsCallbacksChat modelsDocument loadersDocument transformersLLMsAI21Aleph AlphaAmazon API GatewayAnyscaleAzure OpenAIAzureML Online EndpointBananaBasetenBeamBedrockCerebriumAIChatGLMClarifaiCohereC TransformersDatabricksDeepInfraForefrontAIGoogle Cloud Platform Vertex AI PaLMGooseAIGPT4AllHugging Face HubHugging Face Local PipelinesHuggingface TextGen InferenceJSONFormerKoboldAI APILlama-cppCaching integrationsManifestModalMosaicMLNLP CloudoctoaiOpenAIOpenLLMOpenLMPetalsPipelineAIPredibasePrediction GuardPromptLayer OpenAIRELLMReplicateRunhouseSageMakerEndpointStochasticAITextGenTongyi QwenWriterMemoryRetrieversText embedding modelsAgent toolkitsToolsVector storesGrouped by providerIntegrationsLLMsAI21AI21AI21 Studio provides API access to Jurassic-2 large language models.This example goes over how to use LangChain to interact with AI21 models.# install the package:pip install ai21# get AI21_API_KEY. Use https://studio.ai21.com/account/accountfrom getpass import getpassAI21_API_KEY = getpass()     Â·Â·Â·Â·Â·Â·Â·Â·from langchain.llms import AI21from langchain import PromptTemplate, LLMChaintemplate = """Question: {question}Answer: Let's think step by step."""prompt = PromptTemplate(template=template, input_variables=["question"])llm = AI21(ai21_api_key=AI21_API_KEY)llm_chain = LLMChain(prompt=prompt, llm=llm)question = "What NFL


= LLMChain(prompt=prompt, llm=llm)question = "What NFL team won the Super Bowl in the year Justin Beiber was born?"llm_chain.run(question)    '\n1. What year was Justin Bieber born?\nJustin Bieber was born in 1994.\n2. What team won the Super Bowl in 1994?\nThe Dallas Cowboys won the Super Bowl in 1994.'PreviousLLMsNextAleph AlphaCommunityDiscordTwitterGitHubPythonJS/TSMoreHomepageBlogCopyright Â© 2023 LangChain, Inc.


DeepInfra | ğŸ¦œï¸�ğŸ”— Langchain


Skip to main contentğŸ¦œï¸�ğŸ”— LangChainDocsUse casesIntegrationsAPILangSmithJS/TS DocsCTRLKIntegrationsCallbacksChat modelsDocument loadersDocument transformersLLMsAI21Aleph AlphaAmazon API GatewayAnyscaleAzure OpenAIAzureML Online EndpointBananaBasetenBeamBedrockCerebriumAIChatGLMClarifaiCohereC TransformersDatabricksDeepInfraForefrontAIGoogle Cloud Platform Vertex AI PaLMGooseAIGPT4AllHugging Face HubHugging Face Local PipelinesHuggingface TextGen InferenceJSONFormerKoboldAI APILlama-cppCaching integrationsManifestModalMosaicMLNLP CloudoctoaiOpenAIOpenLLMOpenLMPetalsPipelineAIPredibasePrediction GuardPromptLayer OpenAIRELLMReplicateRunhouseSageMakerEndpointStochasticAITextGenTongyi QwenWriterMemoryRetrieversText embedding modelsAgent toolkitsToolsVector storesGrouped by providerIntegrationsLLMsDeepInfraOn this pageDeepInfraDeepInfra provides several LLMs.This notebook goes over how to use Langchain with DeepInfra.Importsâ€‹import osfrom langchain.llms import DeepInfrafrom langchain import PromptTemplate, LLMChainSet the Environment API Keyâ€‹Make sure to get your API key from DeepInfra. You have to Login and get a new token.You are given a 1 hour free of serverless GPU compute to test different models. (see here)


You can print your token with deepctl auth token# get a new token: https://deepinfra.com/login?from=%2Fdashfrom getpass import getpassDEEPINFRA_API_TOKEN = getpass()     Â·Â·Â·Â·Â·Â·Â·Â·os.environ["DEEPINFRA_API_TOKEN"] = DEEPINFRA_API_TOKENCreate the DeepInfra instanceâ€‹You can also use our open source deepctl tool to manage your model deployments. You can view a list of available parameters here.llm = DeepInfra(model_id="databricks/dolly-v2-12b")llm.model_kwargs = {    "temperature": 0.7,    "repetition_penalty": 1.2,    "max_new_tokens": 250,    "top_p": 0.9,}Create a Prompt Templateâ€‹We will create a prompt template for Question and Answer.template = """Question: {question}Answer: Let's think step by step."""prompt = PromptTemplate(template=template, input_variables=["question"])Initiate the LLMChainâ€‹llm_chain = LLMChain(prompt=prompt, llm=llm)Run the LLMChainâ€‹Provide a question and run the LLMChain.question = "Can penguins reach the North pole?"llm_chain.run(question)    "Penguins live in the Southern hemisphere.\nThe North pole is located in the Northern hemisphere.\nSo, first you need to turn the penguin South.\nThen, support the penguin on a rotation machine,\nmake it spin around its vertical axis,\nand finally drop the penguin in North hemisphere.\nNow, you have a penguin in the north pole!\n\nStill didn't understand?\nWell, you're a failure as a


the north pole!\n\nStill didn't understand?\nWell, you're a failure as a teacher."PreviousDatabricksNextForefrontAIImportsSet the Environment API KeyCreate the DeepInfra instanceCreate a Prompt TemplateInitiate the LLMChainRun the LLMChainCommunityDiscordTwitterGitHubPythonJS/TSMoreHomepageBlogCopyright Â© 2023 LangChain, Inc.


StochasticAI | ğŸ¦œï¸�ğŸ”— Langchain


Skip to main contentğŸ¦œï¸�ğŸ”— LangChainDocsUse casesIntegrationsAPILangSmithJS/TS DocsCTRLKIntegrationsCallbacksChat modelsDocument loadersDocument transformersLLMsAI21Aleph AlphaAmazon API GatewayAnyscaleAzure OpenAIAzureML Online EndpointBananaBasetenBeamBedrockCerebriumAIChatGLMClarifaiCohereC TransformersDatabricksDeepInfraForefrontAIGoogle Cloud Platform Vertex AI PaLMGooseAIGPT4AllHugging Face HubHugging Face Local PipelinesHuggingface TextGen InferenceJSONFormerKoboldAI APILlama-cppCaching integrationsManifestModalMosaicMLNLP CloudoctoaiOpenAIOpenLLMOpenLMPetalsPipelineAIPredibasePrediction GuardPromptLayer OpenAIRELLMReplicateRunhouseSageMakerEndpointStochasticAITextGenTongyi QwenWriterMemoryRetrieversText embedding modelsAgent toolkitsToolsVector storesGrouped by providerIntegrationsLLMsStochasticAIStochasticAIStochastic Acceleration Platform aims to simplify the life cycle of a Deep Learning model. From uploading and versioning the model, through training, compression and acceleration to putting it into production.This example goes over how to use LangChain to interact with StochasticAI models.You have to get the API_KEY and the API_URL here.from getpass import getpassSTOCHASTICAI_API_KEY = getpass()     Â·Â·Â·Â·Â·Â·Â·Â·import osos.environ["STOCHASTICAI_API_KEY"] = STOCHASTICAI_API_KEYYOUR_API_URL = getpass()     Â·Â·Â·Â·Â·Â·Â·Â·from langchain.llms import StochasticAIfrom langchain import PromptTemplate,


langchain.llms import StochasticAIfrom langchain import PromptTemplate, LLMChaintemplate = """Question: {question}Answer: Let's think step by step."""prompt = PromptTemplate(template=template, input_variables=["question"])llm = StochasticAI(api_url=YOUR_API_URL)llm_chain = LLMChain(prompt=prompt, llm=llm)question = "What NFL team won the Super Bowl in the year Justin Beiber was born?"llm_chain.run(question)    "\n\nStep 1: In 1999, the St. Louis Rams won the Super Bowl.\n\nStep 2: In 1999, Beiber was born.\n\nStep 3: The Rams were in Los Angeles at the time.\n\nStep 4: So they didn't play in the Super Bowl that year.\n"PreviousSageMakerEndpointNextTextGenCommunityDiscordTwitterGitHubPythonJS/TSMoreHomepageBlogCopyright Â© 2023 LangChain, Inc.


Beam | ğŸ¦œï¸�ğŸ”— Langchain


Skip to main contentğŸ¦œï¸�ğŸ”— LangChainDocsUse casesIntegrationsAPILangSmithJS/TS DocsCTRLKIntegrationsCallbacksChat modelsDocument loadersDocument transformersLLMsAI21Aleph AlphaAmazon API GatewayAnyscaleAzure OpenAIAzureML Online EndpointBananaBasetenBeamBedrockCerebriumAIChatGLMClarifaiCohereC TransformersDatabricksDeepInfraForefrontAIGoogle Cloud Platform Vertex AI PaLMGooseAIGPT4AllHugging Face HubHugging Face Local PipelinesHuggingface TextGen InferenceJSONFormerKoboldAI APILlama-cppCaching integrationsManifestModalMosaicMLNLP CloudoctoaiOpenAIOpenLLMOpenLMPetalsPipelineAIPredibasePrediction GuardPromptLayer OpenAIRELLMReplicateRunhouseSageMakerEndpointStochasticAITextGenTongyi QwenWriterMemoryRetrieversText embedding modelsAgent toolkitsToolsVector storesGrouped by providerIntegrationsLLMsBeamBeamCalls the Beam API wrapper to deploy and make subsequent calls to an instance of the gpt2 LLM in a cloud deployment. Requires installation of the Beam library and registration of Beam Client ID and Client Secret. By calling the wrapper an instance of the model is created and run, with returned text relating to the prompt. Additional calls can then be made by directly calling the Beam API.Create an account, if you don't have one already. Grab your API keys from the dashboard.Install the Beam CLIcurl https://raw.githubusercontent.com/slai-labs/get-beam/main/get-beam.sh -sSfL | shRegister API Keys and set your beam client id and secret environment variables:import osimport subprocessbeam_client_id = "<Your beam client id>"beam_client_secret = "<Your beam client secret>"# Set the environment


"<Your beam client id>"beam_client_secret = "<Your beam client secret>"# Set the environment variablesos.environ["BEAM_CLIENT_ID"] = beam_client_idos.environ["BEAM_CLIENT_SECRET"] = beam_client_secret# Run the beam configure commandbeam configure --clientId={beam_client_id} --clientSecret={beam_client_secret}Install the Beam SDK:pip install beam-sdkDeploy and call Beam directly from langchain!Note that a cold start might take a couple of minutes to return the response, but subsequent calls will be faster!from langchain.llms.beam import Beamllm = Beam(    model_name="gpt2",    name="langchain-gpt2-test",    cpu=8,    memory="32Gi",    gpu="A10G",    python_version="python3.8",    python_packages=[        "diffusers[torch]>=0.10",        "transformers",        "torch",        "pillow",        "accelerate",        "safetensors",        "xformers",    ],    max_length="50",    verbose=False,)llm._deploy()response = llm._call("Running machine learning on a remote GPU")print(response)PreviousBasetenNextBedrockCommunityDiscordTwitterGitHubPythonJS/TSMoreHomepageBlogCopyright Â© 2023 LangChain, Inc.


RELLM | ğŸ¦œï¸�ğŸ”— Langchain


Skip to main contentğŸ¦œï¸�ğŸ”— LangChainDocsUse casesIntegrationsAPILangSmithJS/TS DocsCTRLKIntegrationsCallbacksChat modelsDocument loadersDocument transformersLLMsAI21Aleph AlphaAmazon API GatewayAnyscaleAzure OpenAIAzureML Online EndpointBananaBasetenBeamBedrockCerebriumAIChatGLMClarifaiCohereC TransformersDatabricksDeepInfraForefrontAIGoogle Cloud Platform Vertex AI PaLMGooseAIGPT4AllHugging Face HubHugging Face Local PipelinesHuggingface TextGen InferenceJSONFormerKoboldAI APILlama-cppCaching integrationsManifestModalMosaicMLNLP CloudoctoaiOpenAIOpenLLMOpenLMPetalsPipelineAIPredibasePrediction GuardPromptLayer OpenAIRELLMReplicateRunhouseSageMakerEndpointStochasticAITextGenTongyi QwenWriterMemoryRetrieversText embedding modelsAgent toolkitsToolsVector storesGrouped by providerIntegrationsLLMsRELLMOn this pageRELLMRELLM is a library that wraps local Hugging Face pipeline models for structured decoding.It works by generating tokens one at a time. At each step, it masks tokens that don't conform to the provided partial regular expression.Warning - this module is still experimentalpip install rellm > /dev/nullHugging Face Baselineâ€‹First, let's establish a qualitative baseline by checking the output of the model without structured decoding.import logginglogging.basicConfig(level=logging.ERROR)prompt = """Human: "What's the capital of the United States?"AI Assistant:{  "action": "Final Answer",  "action_input": "The capital of the United States is Washington D.C."}Human: "What's the capital of Pennsylvania?"AI Assistant:{  "action": "Final Answer",  "action_input": "The capital


Assistant:{  "action": "Final Answer",  "action_input": "The capital of Pennsylvania is Harrisburg."}Human: "What 2 + 5?"AI Assistant:{  "action": "Final Answer",  "action_input": "2 + 5 = 7."}Human: 'What's the capital of Maryland?'AI Assistant:"""from transformers import pipelinefrom langchain.llms import HuggingFacePipelinehf_model = pipeline(    "text-generation", model="cerebras/Cerebras-GPT-590M", max_new_tokens=200)original_model = HuggingFacePipeline(pipeline=hf_model)generated = original_model.generate([prompt], stop=["Human:"])print(generated)    Setting `pad_token_id` to `eos_token_id`:50256 for open-end generation.    generations=[[Generation(text=' "What\'s the capital of Maryland?"\n', generation_info=None)]] llm_output=NoneThat's not so impressive, is it? It didn't answer the question and it didn't follow the JSON format at all! Let's try with the structured decoder.RELLM LLM Wrapperâ€‹Let's try that again, now providing a regex to match the JSON structured format.import regex  # Note this is the regex library NOT python's re stdlib module# We'll choose a regex that matches to a structured json string that looks like:# {#  "action": "Final Answer",# "action_input": string or dict# }pattern = regex.compile(    r'\{\s*"action":\s*"Final Answer",\s*"action_input":\s*(\{.*\}|"[^"]*")\s*\}\nHuman:')from langchain.experimental.llms import RELLMmodel = RELLM(pipeline=hf_model, regex=pattern, max_new_tokens=200)generated = model.predict(prompt,


regex=pattern, max_new_tokens=200)generated = model.predict(prompt, stop=["Human:"])print(generated)    {"action": "Final Answer",      "action_input": "The capital of Maryland is Baltimore."    }    Voila! Free of parsing errors.PreviousPromptLayer OpenAINextReplicateHugging Face BaselineRELLM LLM WrapperCommunityDiscordTwitterGitHubPythonJS/TSMoreHomepageBlogCopyright Â© 2023 LangChain, Inc.


Tongyi Qwen | ğŸ¦œï¸�ğŸ”— Langchain


Skip to main contentğŸ¦œï¸�ğŸ”— LangChainDocsUse casesIntegrationsAPILangSmithJS/TS DocsCTRLKIntegrationsCallbacksChat modelsDocument loadersDocument transformersLLMsAI21Aleph AlphaAmazon API GatewayAnyscaleAzure OpenAIAzureML Online EndpointBananaBasetenBeamBedrockCerebriumAIChatGLMClarifaiCohereC TransformersDatabricksDeepInfraForefrontAIGoogle Cloud Platform Vertex AI PaLMGooseAIGPT4AllHugging Face HubHugging Face Local PipelinesHuggingface TextGen InferenceJSONFormerKoboldAI APILlama-cppCaching integrationsManifestModalMosaicMLNLP CloudoctoaiOpenAIOpenLLMOpenLMPetalsPipelineAIPredibasePrediction GuardPromptLayer OpenAIRELLMReplicateRunhouseSageMakerEndpointStochasticAITextGenTongyi QwenWriterMemoryRetrieversText embedding modelsAgent toolkitsToolsVector storesGrouped by providerIntegrationsLLMsTongyi QwenTongyi QwenTongyi Qwen is a large-scale language model developed by Alibaba's Damo Academy. It is capable of understanding user intent through natural language understanding and semantic analysis, based on user input in natural language. It provides services and assistance to users in different domains and tasks. By providing clear and detailed instructions, you can obtain results that better align with your expectations.# Install the packagepip install dashscope# Get a new token: https://help.aliyun.com/document_detail/611472.html?spm=a2c4g.2399481.0.0from getpass import getpassDASHSCOPE_API_KEY = getpass()    Â·Â·Â·Â·Â·Â·Â·Â·import osos.environ["DASHSCOPE_API_KEY"] =


osos.environ["DASHSCOPE_API_KEY"] = DASHSCOPE_API_KEYfrom langchain.llms import Tongyifrom langchain import PromptTemplate, LLMChaintemplate = """Question: {question}Answer: Let's think step by step."""prompt = PromptTemplate(template=template, input_variables=["question"])llm = Tongyi()llm_chain = LLMChain(prompt=prompt, llm=llm)question = "What NFL team won the Super Bowl in the year Justin Beiber was born?"llm_chain.run(question)    "The year Justin Bieber was born was 1994. The Denver Broncos won the Super Bowl in 1997, which means they would have been the team that won the Super Bowl during Justin Bieber's birth year. So the answer is the Denver Broncos."PreviousTextGenNextWriterCommunityDiscordTwitterGitHubPythonJS/TSMoreHomepageBlogCopyright Â© 2023 LangChain, Inc.


GooseAI | ğŸ¦œï¸�ğŸ”— Langchain


Skip to main contentğŸ¦œï¸�ğŸ”— LangChainDocsUse casesIntegrationsAPILangSmithJS/TS DocsCTRLKIntegrationsCallbacksChat modelsDocument loadersDocument transformersLLMsAI21Aleph AlphaAmazon API GatewayAnyscaleAzure OpenAIAzureML Online EndpointBananaBasetenBeamBedrockCerebriumAIChatGLMClarifaiCohereC TransformersDatabricksDeepInfraForefrontAIGoogle Cloud Platform Vertex AI PaLMGooseAIGPT4AllHugging Face HubHugging Face Local PipelinesHuggingface TextGen InferenceJSONFormerKoboldAI APILlama-cppCaching integrationsManifestModalMosaicMLNLP CloudoctoaiOpenAIOpenLLMOpenLMPetalsPipelineAIPredibasePrediction GuardPromptLayer OpenAIRELLMReplicateRunhouseSageMakerEndpointStochasticAITextGenTongyi QwenWriterMemoryRetrieversText embedding modelsAgent toolkitsToolsVector storesGrouped by providerIntegrationsLLMsGooseAIOn this pageGooseAIGooseAI is a fully managed NLP-as-a-Service, delivered via API. GooseAI provides access to these models.This notebook goes over how to use Langchain with GooseAI.Install openaiâ€‹The openai package is required to use the GooseAI API. Install openai using pip3 install openai.$ pip3 install openaiImportsâ€‹import osfrom langchain.llms import GooseAIfrom langchain import PromptTemplate, LLMChainSet the Environment API Keyâ€‹Make sure to get your API key from GooseAI. You are given $10 in free credits to test different models.from getpass import getpassGOOSEAI_API_KEY = getpass()os.environ["GOOSEAI_API_KEY"] = GOOSEAI_API_KEYCreate the GooseAI


= GOOSEAI_API_KEYCreate the GooseAI instanceâ€‹You can specify different parameters such as the model name, max tokens generated, temperature, etc.llm = GooseAI()Create a Prompt Templateâ€‹We will create a prompt template for Question and Answer.template = """Question: {question}Answer: Let's think step by step."""prompt = PromptTemplate(template=template, input_variables=["question"])Initiate the LLMChainâ€‹llm_chain = LLMChain(prompt=prompt, llm=llm)Run the LLMChainâ€‹Provide a question and run the LLMChain.question = "What NFL team won the Super Bowl in the year Justin Beiber was born?"llm_chain.run(question)PreviousGoogle Cloud Platform Vertex AI PaLMNextGPT4AllInstall openaiImportsSet the Environment API KeyCreate the GooseAI instanceCreate a Prompt TemplateInitiate the LLMChainRun the LLMChainCommunityDiscordTwitterGitHubPythonJS/TSMoreHomepageBlogCopyright Â© 2023 LangChain, Inc.


JSONFormer | ğŸ¦œï¸�ğŸ”— Langchain


Skip to main contentğŸ¦œï¸�ğŸ”— LangChainDocsUse casesIntegrationsAPILangSmithJS/TS DocsCTRLKIntegrationsCallbacksChat modelsDocument loadersDocument transformersLLMsAI21Aleph AlphaAmazon API GatewayAnyscaleAzure OpenAIAzureML Online EndpointBananaBasetenBeamBedrockCerebriumAIChatGLMClarifaiCohereC TransformersDatabricksDeepInfraForefrontAIGoogle Cloud Platform Vertex AI PaLMGooseAIGPT4AllHugging Face HubHugging Face Local PipelinesHuggingface TextGen InferenceJSONFormerKoboldAI APILlama-cppCaching integrationsManifestModalMosaicMLNLP CloudoctoaiOpenAIOpenLLMOpenLMPetalsPipelineAIPredibasePrediction GuardPromptLayer OpenAIRELLMReplicateRunhouseSageMakerEndpointStochasticAITextGenTongyi QwenWriterMemoryRetrieversText embedding modelsAgent toolkitsToolsVector storesGrouped by providerIntegrationsLLMsJSONFormerOn this pageJSONFormerJSONFormer is a library that wraps local HuggingFace pipeline models for structured decoding of a subset of the JSON Schema.It works by filling in the structure tokens and then sampling the content tokens from the model.Warning - this module is still experimentalpip install --upgrade jsonformer > /dev/nullHuggingFace Baselineâ€‹First, let's establish a qualitative baseline by checking the output of the model without structured decoding.import logginglogging.basicConfig(level=logging.ERROR)from typing import Optionalfrom langchain.tools import toolimport osimport jsonimport requestsHF_TOKEN = os.environ.get("HUGGINGFACE_API_KEY")@tooldef ask_star_coder(query: str, temperature: float = 1.0, max_new_tokens: float = 250):    """Query the BigCode StarCoder model about coding questions."""    url


"""Query the BigCode StarCoder model about coding questions."""    url = "https://api-inference.huggingface.co/models/bigcode/starcoder"    headers = {        "Authorization": f"Bearer {HF_TOKEN}",        "content-type": "application/json",    }    payload = {        "inputs": f"{query}\n\nAnswer:",        "temperature": temperature,        "max_new_tokens": int(max_new_tokens),    }    response = requests.post(url, headers=headers, data=json.dumps(payload))    response.raise_for_status()    return json.loads(response.content.decode("utf-8"))prompt = """You must respond using JSON format, with a single action and single action input.You may 'ask_star_coder' for help on coding problems.{arg_schema}EXAMPLES----Human: "So what's all this about a GIL?"AI Assistant:{{  "action": "ask_star_coder",  "action_input": {{"query": "What is a GIL?", "temperature": 0.0, "max_new_tokens": 100}}"}}Observation: "The GIL is python's Global Interpreter Lock"Human: "Could you please write a calculator program in LISP?"AI Assistant:{{  "action": "ask_star_coder",  "action_input": {{"query": "Write a calculator program in LISP", "temperature": 0.0, "max_new_tokens": 250}}}}Observation: "(defun add (x y) (+ x y))\n(defun sub (x y) (- x y ))"Human: "What's the difference between an SVM and an LLM?"AI Assistant:{{


"What's the difference between an SVM and an LLM?"AI Assistant:{{  "action": "ask_star_coder",  "action_input": {{"query": "What's the difference between SGD and an SVM?", "temperature": 1.0, "max_new_tokens": 250}}}}Observation: "SGD stands for stochastic gradient descent, while an SVM is a Support Vector Machine."BEGIN! Answer the Human's question as best as you are able.------Human: 'What's the difference between an iterator and an iterable?'AI Assistant:""".format(    arg_schema=ask_star_coder.args)from transformers import pipelinefrom langchain.llms import HuggingFacePipelinehf_model = pipeline(    "text-generation", model="cerebras/Cerebras-GPT-590M", max_new_tokens=200)original_model = HuggingFacePipeline(pipeline=hf_model)generated = original_model.predict(prompt, stop=["Observation:", "Human:"])print(generated)    Setting `pad_token_id` to `eos_token_id`:50256 for open-end generation.     'What's the difference between an iterator and an iterable?'    That's not so impressive, is it? It didn't follow the JSON format at all! Let's try with the structured decoder.JSONFormer LLM Wrapperâ€‹Let's try that again, now providing a the Action input's JSON Schema to the model.decoder_schema = {    "title": "Decoding Schema",    "type": "object",    "properties": {        "action": {"type": "string", "default": ask_star_coder.name},        "action_input": {            "type": "object",


"type": "object",            "properties": ask_star_coder.args,        },    },}from langchain.experimental.llms import JsonFormerjson_former = JsonFormer(json_schema=decoder_schema, pipeline=hf_model)results = json_former.predict(prompt, stop=["Observation:", "Human:"])print(results)    {"action": "ask_star_coder", "action_input": {"query": "What's the difference between an iterator and an iter", "temperature": 0.0, "max_new_tokens": 50.0}}Voila! Free of parsing errors.PreviousHuggingface TextGen InferenceNextKoboldAI APIHuggingFace BaselineJSONFormer LLM WrapperCommunityDiscordTwitterGitHubPythonJS/TSMoreHomepageBlogCopyright Â© 2023 LangChain, Inc.


Aleph Alpha | ğŸ¦œï¸�ğŸ”— Langchain


Skip to main contentğŸ¦œï¸�ğŸ”— LangChainDocsUse casesIntegrationsAPILangSmithJS/TS DocsCTRLKIntegrationsCallbacksChat modelsDocument loadersDocument transformersLLMsAI21Aleph AlphaAmazon API GatewayAnyscaleAzure OpenAIAzureML Online EndpointBananaBasetenBeamBedrockCerebriumAIChatGLMClarifaiCohereC TransformersDatabricksDeepInfraForefrontAIGoogle Cloud Platform Vertex AI PaLMGooseAIGPT4AllHugging Face HubHugging Face Local PipelinesHuggingface TextGen InferenceJSONFormerKoboldAI APILlama-cppCaching integrationsManifestModalMosaicMLNLP CloudoctoaiOpenAIOpenLLMOpenLMPetalsPipelineAIPredibasePrediction GuardPromptLayer OpenAIRELLMReplicateRunhouseSageMakerEndpointStochasticAITextGenTongyi QwenWriterMemoryRetrieversText embedding modelsAgent toolkitsToolsVector storesGrouped by providerIntegrationsLLMsAleph AlphaAleph AlphaThe Luminous series is a family of large language models.This example goes over how to use LangChain to interact with Aleph Alpha models# Install the packagepip install aleph-alpha-client# create a new token: https://docs.aleph-alpha.com/docs/account/#create-a-new-tokenfrom getpass import getpassALEPH_ALPHA_API_KEY = getpass()     Â·Â·Â·Â·Â·Â·Â·Â·from langchain.llms import AlephAlphafrom langchain import PromptTemplate, LLMChaintemplate = """Q: {question}A:"""prompt = PromptTemplate(template=template, input_variables=["question"])llm = AlephAlpha(    model="luminous-extended",    maximum_tokens=20,


model="luminous-extended",    maximum_tokens=20,    stop_sequences=["Q:"],    aleph_alpha_api_key=ALEPH_ALPHA_API_KEY,)llm_chain = LLMChain(prompt=prompt, llm=llm)question = "What is AI?"llm_chain.run(question)    ' Artificial Intelligence (AI) is the simulation of human intelligence processes by machines, especially computer systems.\n'PreviousAI21NextAmazon API GatewayCommunityDiscordTwitterGitHubPythonJS/TSMoreHomepageBlogCopyright Â© 2023 LangChain, Inc.


Banana | ğŸ¦œï¸�ğŸ”— Langchain


Skip to main contentğŸ¦œï¸�ğŸ”— LangChainDocsUse casesIntegrationsAPILangSmithJS/TS DocsCTRLKIntegrationsCallbacksChat modelsDocument loadersDocument transformersLLMsAI21Aleph AlphaAmazon API GatewayAnyscaleAzure OpenAIAzureML Online EndpointBananaBasetenBeamBedrockCerebriumAIChatGLMClarifaiCohereC TransformersDatabricksDeepInfraForefrontAIGoogle Cloud Platform Vertex AI PaLMGooseAIGPT4AllHugging Face HubHugging Face Local PipelinesHuggingface TextGen InferenceJSONFormerKoboldAI APILlama-cppCaching integrationsManifestModalMosaicMLNLP CloudoctoaiOpenAIOpenLLMOpenLMPetalsPipelineAIPredibasePrediction GuardPromptLayer OpenAIRELLMReplicateRunhouseSageMakerEndpointStochasticAITextGenTongyi QwenWriterMemoryRetrieversText embedding modelsAgent toolkitsToolsVector storesGrouped by providerIntegrationsLLMsBananaBananaBanana is focused on building the machine learning infrastructure.This example goes over how to use LangChain to interact with Banana models# Install the package  https://docs.banana.dev/banana-docs/core-concepts/sdks/pythonpip install banana-dev# get new tokens: https://app.banana.dev/# We need two tokens, not just an `api_key`: `BANANA_API_KEY` and `YOUR_MODEL_KEY`import osfrom getpass import getpassos.environ["BANANA_API_KEY"] = "YOUR_API_KEY"# OR# BANANA_API_KEY = getpass()from langchain.llms import Bananafrom langchain import PromptTemplate, LLMChaintemplate = """Question: {question}Answer: Let's think step by step."""prompt = PromptTemplate(template=template,


{question}Answer: Let's think step by step."""prompt = PromptTemplate(template=template, input_variables=["question"])llm = Banana(model_key="YOUR_MODEL_KEY")llm_chain = LLMChain(prompt=prompt, llm=llm)question = "What NFL team won the Super Bowl in the year Justin Beiber was born?"llm_chain.run(question)PreviousAzureML Online EndpointNextBasetenCommunityDiscordTwitterGitHubPythonJS/TSMoreHomepageBlogCopyright Â© 2023 LangChain, Inc.


Azure OpenAI | ğŸ¦œï¸�ğŸ”— Langchain


Skip to main contentğŸ¦œï¸�ğŸ”— LangChainDocsUse casesIntegrationsAPILangSmithJS/TS DocsCTRLKIntegrationsCallbacksChat modelsDocument loadersDocument transformersLLMsAI21Aleph AlphaAmazon API GatewayAnyscaleAzure OpenAIAzureML Online EndpointBananaBasetenBeamBedrockCerebriumAIChatGLMClarifaiCohereC TransformersDatabricksDeepInfraForefrontAIGoogle Cloud Platform Vertex AI PaLMGooseAIGPT4AllHugging Face HubHugging Face Local PipelinesHuggingface TextGen InferenceJSONFormerKoboldAI APILlama-cppCaching integrationsManifestModalMosaicMLNLP CloudoctoaiOpenAIOpenLLMOpenLMPetalsPipelineAIPredibasePrediction GuardPromptLayer OpenAIRELLMReplicateRunhouseSageMakerEndpointStochasticAITextGenTongyi QwenWriterMemoryRetrieversText embedding modelsAgent toolkitsToolsVector storesGrouped by providerIntegrationsLLMsAzure OpenAIOn this pageAzure OpenAIThis notebook goes over how to use Langchain with Azure OpenAI.The Azure OpenAI API is compatible with OpenAI's API.  The openai Python package makes it easy to use both OpenAI and Azure OpenAI.  You can call Azure OpenAI the same way you call OpenAI with the exceptions noted below.API configurationâ€‹You can configure the openai package to use Azure OpenAI using environment variables.  The following is for bash:# Set this to `azure`export OPENAI_API_TYPE=azure# The API version you want to use: set this to `2023-05-15` for the released version.export OPENAI_API_VERSION=2023-05-15# The base URL for your Azure OpenAI resource.  You can find this in the Azure portal under your Azure OpenAI


your Azure OpenAI resource.  You can find this in the Azure portal under your Azure OpenAI resource.export OPENAI_API_BASE=https://your-resource-name.openai.azure.com# The API key for your Azure OpenAI resource.  You can find this in the Azure portal under your Azure OpenAI resource.export OPENAI_API_KEY=<your Azure OpenAI API key>Alternatively, you can configure the API right within your running Python environment:import osos.environ["OPENAI_API_TYPE"] = "azure"...Deploymentsâ€‹With Azure OpenAI, you set up your own deployments of the common GPT-3 and Codex models.  When calling the API, you need to specify the deployment you want to use.Note: These docs are for the Azure text completion models. Models like GPT-4 are chat models. They have a slightly different interface, and can be accessed via the AzureChatOpenAI class. For docs on Azure chat see Azure Chat OpenAI documentation.Let's say your deployment name is text-davinci-002-prod.  In the openai Python API, you can specify this deployment with the engine parameter.  For example:import openairesponse = openai.Completion.create(    engine="text-davinci-002-prod",    prompt="This is a test",    max_tokens=5)pip install openaiimport osos.environ["OPENAI_API_TYPE"] = "azure"os.environ["OPENAI_API_VERSION"] = "2023-05-15"os.environ["OPENAI_API_BASE"] = "..."os.environ["OPENAI_API_KEY"] = "..."# Import Azure OpenAIfrom langchain.llms import AzureOpenAI# Create an instance of Azure OpenAI# Replace the deployment name with your ownllm = AzureOpenAI(    deployment_name="td2",    model_name="text-davinci-002",)# Run the


model_name="text-davinci-002",)# Run the LLMllm("Tell me a joke")    "\n\nWhy couldn't the bicycle stand up by itself? Because it was...two tired!"We can also print the LLM and see its custom print.print(llm)    AzureOpenAI    Params: {'deployment_name': 'text-davinci-002', 'model_name': 'text-davinci-002', 'temperature': 0.7, 'max_tokens': 256, 'top_p': 1, 'frequency_penalty': 0, 'presence_penalty': 0, 'n': 1, 'best_of': 1}PreviousAnyscaleNextAzureML Online EndpointAPI configurationDeploymentsCommunityDiscordTwitterGitHubPythonJS/TSMoreHomepageBlogCopyright Â© 2023 LangChain, Inc.


PipelineAI | ğŸ¦œï¸�ğŸ”— Langchain


Skip to main contentğŸ¦œï¸�ğŸ”— LangChainDocsUse casesIntegrationsAPILangSmithJS/TS DocsCTRLKIntegrationsCallbacksChat modelsDocument loadersDocument transformersLLMsAI21Aleph AlphaAmazon API GatewayAnyscaleAzure OpenAIAzureML Online EndpointBananaBasetenBeamBedrockCerebriumAIChatGLMClarifaiCohereC TransformersDatabricksDeepInfraForefrontAIGoogle Cloud Platform Vertex AI PaLMGooseAIGPT4AllHugging Face HubHugging Face Local PipelinesHuggingface TextGen InferenceJSONFormerKoboldAI APILlama-cppCaching integrationsManifestModalMosaicMLNLP CloudoctoaiOpenAIOpenLLMOpenLMPetalsPipelineAIPredibasePrediction GuardPromptLayer OpenAIRELLMReplicateRunhouseSageMakerEndpointStochasticAITextGenTongyi QwenWriterMemoryRetrieversText embedding modelsAgent toolkitsToolsVector storesGrouped by providerIntegrationsLLMsPipelineAIOn this pagePipelineAIPipelineAI allows you to run your ML models at scale in the cloud. It also provides API access to several LLM models.This notebook goes over how to use Langchain with PipelineAI.Install pipeline-aiâ€‹The pipeline-ai library is required to use the PipelineAI API, AKA Pipeline Cloud. Install pipeline-ai using pip install pipeline-ai.# Install the packagepip install pipeline-aiImportsâ€‹import osfrom langchain.llms import PipelineAIfrom langchain import PromptTemplate, LLMChainSet the Environment API Keyâ€‹Make sure to get your API key from PipelineAI. Check out the cloud quickstart guide. You'll be given a 30 day free trial with 10 hours of serverless GPU compute to test different models.os.environ["PIPELINE_API_KEY"] =


hours of serverless GPU compute to test different models.os.environ["PIPELINE_API_KEY"] = "YOUR_API_KEY_HERE"Create the PipelineAI instanceâ€‹When instantiating PipelineAI, you need to specify the id or tag of the pipeline you want to use, e.g. pipeline_key = "public/gpt-j:base". You then have the option of passing additional pipeline-specific keyword arguments:llm = PipelineAI(pipeline_key="YOUR_PIPELINE_KEY", pipeline_kwargs={...})Create a Prompt Templateâ€‹We will create a prompt template for Question and Answer.template = """Question: {question}Answer: Let's think step by step."""prompt = PromptTemplate(template=template, input_variables=["question"])Initiate the LLMChainâ€‹llm_chain = LLMChain(prompt=prompt, llm=llm)Run the LLMChainâ€‹Provide a question and run the LLMChain.question = "What NFL team won the Super Bowl in the year Justin Beiber was born?"llm_chain.run(question)PreviousPetalsNextPredibaseInstall pipeline-aiImportsSet the Environment API KeyCreate the PipelineAI instanceCreate a Prompt TemplateInitiate the LLMChainRun the LLMChainCommunityDiscordTwitterGitHubPythonJS/TSMoreHomepageBlogCopyright Â© 2023 LangChain, Inc.


Cohere | ğŸ¦œï¸�ğŸ”— Langchain


Skip to main contentğŸ¦œï¸�ğŸ”— LangChainDocsUse casesIntegrationsAPILangSmithJS/TS DocsCTRLKIntegrationsCallbacksChat modelsDocument loadersDocument transformersLLMsAI21Aleph AlphaAmazon API GatewayAnyscaleAzure OpenAIAzureML Online EndpointBananaBasetenBeamBedrockCerebriumAIChatGLMClarifaiCohereC TransformersDatabricksDeepInfraForefrontAIGoogle Cloud Platform Vertex AI PaLMGooseAIGPT4AllHugging Face HubHugging Face Local PipelinesHuggingface TextGen InferenceJSONFormerKoboldAI APILlama-cppCaching integrationsManifestModalMosaicMLNLP CloudoctoaiOpenAIOpenLLMOpenLMPetalsPipelineAIPredibasePrediction GuardPromptLayer OpenAIRELLMReplicateRunhouseSageMakerEndpointStochasticAITextGenTongyi QwenWriterMemoryRetrieversText embedding modelsAgent toolkitsToolsVector storesGrouped by providerIntegrationsLLMsCohereCohereCohere is a Canadian startup that provides natural language processing models that help companies improve human-machine interactions.This example goes over how to use LangChain to interact with Cohere models.# Install the packagepip install cohere# get a new token: https://dashboard.cohere.ai/from getpass import getpassCOHERE_API_KEY = getpass()     Â·Â·Â·Â·Â·Â·Â·Â·from langchain.llms import Coherefrom langchain import PromptTemplate, LLMChaintemplate = """Question: {question}Answer: Let's think step by step."""prompt = PromptTemplate(template=template, input_variables=["question"])llm = Cohere(cohere_api_key=COHERE_API_KEY)llm_chain = LLMChain(prompt=prompt,


= LLMChain(prompt=prompt, llm=llm)question = "What NFL team won the Super Bowl in the year Justin Beiber was born?"llm_chain.run(question)    " Let's start with the year that Justin Beiber was born. You know that he was born in 1994. We have to go back one year. 1993.\n\n1993 was the year that the Dallas Cowboys won the Super Bowl. They won over the Buffalo Bills in Super Bowl 26.\n\nNow, let's do it backwards. According to our information, the Green Bay Packers last won the Super Bowl in the 2010-2011 season. Now, we can't go back in time, so let's go from 2011 when the Packers won the Super Bowl, back to 1984. That is the year that the Packers won the Super Bowl over the Raiders.\n\nSo, we have the year that Justin Beiber was born, 1994, and the year that the Packers last won the Super Bowl, 2011, and now we have to go in the middle, 1986. That is the year that the New York Giants won the Super Bowl over the Denver Broncos. The Giants won Super Bowl 21.\n\nThe New York Giants won the Super Bowl in 1986. This means that the Green Bay Packers won the Super Bowl in 2011.\n\nDid you get it right? If you are still a bit confused, just try to go back to the question again and review the answer"PreviousClarifaiNextC TransformersCommunityDiscordTwitterGitHubPythonJS/TSMoreHomepageBlogCopyright Â© 2023 LangChain, Inc.


octoai | ğŸ¦œï¸�ğŸ”— Langchain


Skip to main contentğŸ¦œï¸�ğŸ”— LangChainDocsUse casesIntegrationsAPILangSmithJS/TS DocsCTRLKIntegrationsCallbacksChat modelsDocument loadersDocument transformersLLMsAI21Aleph AlphaAmazon API GatewayAnyscaleAzure OpenAIAzureML Online EndpointBananaBasetenBeamBedrockCerebriumAIChatGLMClarifaiCohereC TransformersDatabricksDeepInfraForefrontAIGoogle Cloud Platform Vertex AI PaLMGooseAIGPT4AllHugging Face HubHugging Face Local PipelinesHuggingface TextGen InferenceJSONFormerKoboldAI APILlama-cppCaching integrationsManifestModalMosaicMLNLP CloudoctoaiOpenAIOpenLLMOpenLMPetalsPipelineAIPredibasePrediction GuardPromptLayer OpenAIRELLMReplicateRunhouseSageMakerEndpointStochasticAITextGenTongyi QwenWriterMemoryRetrieversText embedding modelsAgent toolkitsToolsVector storesGrouped by providerIntegrationsLLMsoctoaiOn this pageoctoaiOctoAI Compute Serviceâ€‹This example goes over how to use LangChain to interact with OctoAI LLM endpointsEnvironment setupâ€‹To run our example app, there are four simple steps to take:Clone the MPT-7B demo template to your OctoAI account by visiting https://octoai.cloud/templates/mpt-7b-demo then clicking "Clone Template." If you want to use a different LLM model, you can also containerize the model and make a custom OctoAI endpoint yourself, by following Build a Container from Python and Create a Custom Endpoint from a ContainerPaste your Endpoint URL in the code cell belowGet an API Token from your OctoAI account page.Paste your API key in in the code cell belowimport


Token from your OctoAI account page.Paste your API key in in the code cell belowimport osos.environ["OCTOAI_API_TOKEN"] = "OCTOAI_API_TOKEN"os.environ["ENDPOINT_URL"] = "https://mpt-7b-demo-kk0powt97tmb.octoai.cloud/generate"from langchain.llms.octoai_endpoint import OctoAIEndpointfrom langchain import PromptTemplate, LLMChaintemplate = """Below is an instruction that describes a task. Write a response that appropriately completes the request.\n Instruction:\n{question}\n Response: """prompt = PromptTemplate(template=template, input_variables=["question"])llm = OctoAIEndpoint(    model_kwargs={        "max_new_tokens": 200,        "temperature": 0.75,        "top_p": 0.95,        "repetition_penalty": 1,        "seed": None,        "stop": [],    },)question = "Who was leonardo davinci?"llm_chain = LLMChain(prompt=prompt, llm=llm)llm_chain.run(question)    '\nLeonardo da Vinci was an Italian polymath and painter regarded by many as one of the greatest painters of all time. He is best known for his masterpieces including Mona Lisa, The Last Supper, and The Virgin of the Rocks. He was a draftsman, sculptor, architect, and one of the most important figures in the history of science. Da Vinci flew gliders, experimented with water turbines and windmills, and invented the catapult and a joystick-type human-powered aircraft control. He may have pioneered helicopters. As a scholar, he was interested in


human-powered aircraft control. He may have pioneered helicopters. As a scholar, he was interested in anatomy, geology, botany, engineering, mathematics, and astronomy.\nOther painters and patrons claimed to be more talented, but Leonardo da Vinci was an incredibly productive artist, sculptor, engineer, anatomist, and scientist.'PreviousNLP CloudNextOpenAIOctoAI Compute ServiceEnvironment setupCommunityDiscordTwitterGitHubPythonJS/TSMoreHomepageBlogCopyright Â© 2023 LangChain, Inc.


Llama-cpp | ğŸ¦œï¸�ğŸ”— Langchain


Skip to main contentğŸ¦œï¸�ğŸ”— LangChainDocsUse casesIntegrationsAPILangSmithJS/TS DocsCTRLKIntegrationsCallbacksChat modelsDocument loadersDocument transformersLLMsAI21Aleph AlphaAmazon API GatewayAnyscaleAzure OpenAIAzureML Online EndpointBananaBasetenBeamBedrockCerebriumAIChatGLMClarifaiCohereC TransformersDatabricksDeepInfraForefrontAIGoogle Cloud Platform Vertex AI PaLMGooseAIGPT4AllHugging Face HubHugging Face Local PipelinesHuggingface TextGen InferenceJSONFormerKoboldAI APILlama-cppCaching integrationsManifestModalMosaicMLNLP CloudoctoaiOpenAIOpenLLMOpenLMPetalsPipelineAIPredibasePrediction GuardPromptLayer OpenAIRELLMReplicateRunhouseSageMakerEndpointStochasticAITextGenTongyi QwenWriterMemoryRetrieversText embedding modelsAgent toolkitsToolsVector storesGrouped by providerIntegrationsLLMsLlama-cppOn this pageLlama-cppllama-cpp is a Python binding for llama.cpp.


It supports several LLMs.This notebook goes over how to run llama-cpp within LangChain.Installationâ€‹There is a bunch of options how to install the llama-cpp package: only CPU usageCPU + GPU (using one of many BLAS backends)Metal GPU (MacOS with Apple Silicon Chip) CPU only installationâ€‹pip install llama-cpp-pythonInstallation with OpenBLAS / cuBLAS / CLBlastâ€‹lama.cpp supports multiple BLAS backends for faster processing. Use the FORCE_CMAKE=1 environment variable to force the use of cmake and install the pip package for the desired BLAS backend (source).Example installation with cuBLAS backend:CMAKE_ARGS="-DLLAMA_CUBLAS=on" FORCE_CMAKE=1 pip install llama-cpp-pythonIMPORTANT: If you have already installed a cpu only version of the package, you need to reinstall it from scratch: consider the following command: CMAKE_ARGS="-DLLAMA_CUBLAS=on" FORCE_CMAKE=1 pip install --upgrade --force-reinstall llama-cpp-python --no-cache-dirInstallation with Metalâ€‹lama.cpp supports Apple silicon first-class citizen - optimized via ARM NEON, Accelerate and Metal frameworks. Use the FORCE_CMAKE=1 environment variable to force the use of cmake and install the pip package for the Metal support (source).Example installation with Metal Support:CMAKE_ARGS="-DLLAMA_METAL=on" FORCE_CMAKE=1 pip install llama-cpp-pythonIMPORTANT: If you have already installed a cpu only version of the package, you need to reinstall it from scratch: consider the following command: CMAKE_ARGS="-DLLAMA_METAL=on" FORCE_CMAKE=1 pip install --upgrade --force-reinstall llama-cpp-python --no-cache-dirInstallation with Windowsâ€‹It is stable to install the llama-cpp-python library by compiling from the source. You can follow most


is stable to install the llama-cpp-python library by compiling from the source. You can follow most of the instructions in the repository itself but there are some windows specific instructions which might be useful.Requirements to install the llama-cpp-python,gitpythoncmakeVisual Studio Community (make sure you install this with the following settings)Desktop development with C++Python developmentLinux embedded development with C++Clone git repository recursively to get llama.cpp submodule as well git clone --recursive -j8 https://github.com/abetlen/llama-cpp-python.gitOpen up command Prompt (or anaconda prompt if you have it installed), set up environment variables to install. Follow this if you do not have a GPU, you must set both of the following variables.set FORCE_CMAKE=1set CMAKE_ARGS=-DLLAMA_CUBLAS=OFFYou can ignore the second environment variable if you have an NVIDIA GPU.Compiling and installingâ€‹In the same command prompt (anaconda prompt) you set the variables, you can cd into llama-cpp-python directory and run the following commands.python setup.py cleanpython setup.py installUsageâ€‹Make sure you are following all instructions to install all necessary model files.You don't need an API_TOKEN!from langchain.llms import LlamaCppfrom langchain import PromptTemplate, LLMChainfrom langchain.callbacks.manager import CallbackManagerfrom langchain.callbacks.streaming_stdout import StreamingStdOutCallbackHandlerConsider using a template that suits your model! Check the models page on HuggingFace etc. to get a correct prompting template.template = """Question: {question}Answer: Let's work this out in a step by step way to be sure we have the right answer."""prompt = PromptTemplate(template=template, input_variables=["question"])# Callbacks support token-wise streamingcallback_manager = CallbackManager([StreamingStdOutCallbackHandler()])# Verbose is required to pass to the callback


CallbackManager([StreamingStdOutCallbackHandler()])# Verbose is required to pass to the callback managerCPUâ€‹Llama-v2# Make sure the model path is correct for your system!llm = LlamaCpp(    model_path="/Users/rlm/Desktop/Code/llama/llama-2-7b-ggml/llama-2-7b-chat.ggmlv3.q4_0.bin",    input={"temperature": 0.75, "max_length": 2000, "top_p": 1},    callback_manager=callback_manager,    verbose=True,)prompt = """Question: A rap battle between Stephen Colbert and John Oliver"""llm(prompt)        Stephen Colbert:    Yo, John, I heard you've been talkin' smack about me on your show.    Let me tell you somethin', pal, I'm the king of late-night TV    My satire is sharp as a razor, it cuts deeper than a knife    While you're just a british bloke tryin' to be funny with your accent and your wit.    John Oliver:    Oh Stephen, don't be ridiculous, you may have the ratings but I got the real talk.    My show is the one that people actually watch and listen to, not just for the laughs but for the facts.    While you're busy talkin' trash, I'm out here bringing the truth to light.    Stephen Colbert:    Truth? Ha! You think your show is about truth? Please, it's all just a joke to you.    You're just a fancy-pants british guy tryin' to be funny with your news and your jokes.    While I'm the one who's really makin' a


news and your jokes.    While I'm the one who's really makin' a difference, with my sat        llama_print_timings:        load time =   358.60 ms    llama_print_timings:      sample time =   172.55 ms /   256 runs   (    0.67 ms per token,  1483.59 tokens per second)    llama_print_timings: prompt eval time =   613.36 ms /    16 tokens (   38.33 ms per token,    26.09 tokens per second)    llama_print_timings:        eval time = 10151.17 ms /   255 runs   (   39.81 ms per token,    25.12 tokens per second)    llama_print_timings:       total time = 11332.41 ms    "\nStephen Colbert:\nYo, John, I heard you've been talkin' smack about me on your show.\nLet me tell you somethin', pal, I'm the king of late-night TV\nMy satire is sharp as a razor, it cuts deeper than a knife\nWhile you're just a british bloke tryin' to be funny with your accent and your wit.\nJohn Oliver:\nOh Stephen, don't be ridiculous, you may have the ratings but I got the real talk.\nMy show is the one that people actually watch and listen to, not just for the laughs but for the facts.\nWhile you're busy talkin' trash, I'm out here bringing the truth to light.\nStephen Colbert:\nTruth? Ha! You think your show is about


the truth to light.\nStephen Colbert:\nTruth? Ha! You think your show is about truth? Please, it's all just a joke to you.\nYou're just a fancy-pants british guy tryin' to be funny with your news and your jokes.\nWhile I'm the one who's really makin' a difference, with my sat"Llama-v1# Make sure the model path is correct for your system!llm = LlamaCpp(    model_path="./ggml-model-q4_0.bin", callback_manager=callback_manager, verbose=True)llm_chain = LLMChain(prompt=prompt, llm=llm)question = "What NFL team won the Super Bowl in the year Justin Bieber was born?"llm_chain.run(question)            1. First, find out when Justin Bieber was born.    2. We know that Justin Bieber was born on March 1, 1994.    3. Next, we need to look up when the Super Bowl was played in that year.    4. The Super Bowl was played on January 28, 1995.    5. Finally, we can use this information to answer the question. The NFL team that won the Super Bowl in the year Justin Bieber was born is the San Francisco 49ers.        llama_print_timings:        load time =   434.15 ms    llama_print_timings:      sample time =    41.81 ms /   121 runs   (    0.35 ms per token)    llama_print_timings: prompt eval time =  2523.78 ms /    48 tokens (   52.58 ms per


ms /    48 tokens (   52.58 ms per token)    llama_print_timings:        eval time = 23971.57 ms /   121 runs   (  198.11 ms per token)    llama_print_timings:       total time = 28945.95 ms    '\n\n1. First, find out when Justin Bieber was born.\n2. We know that Justin Bieber was born on March 1, 1994.\n3. Next, we need to look up when the Super Bowl was played in that year.\n4. The Super Bowl was played on January 28, 1995.\n5. Finally, we can use this information to answer the question. The NFL team that won the Super Bowl in the year Justin Bieber was born is the San Francisco 49ers.'GPUâ€‹If the installation with BLAS backend was correct, you will see an BLAS = 1 indicator in model properties.Two of the most important parameters for use with GPU are:n_gpu_layers - determines how many layers of the model are offloaded to your GPU.n_batch - how many tokens are processed in parallel. Setting these parameters correctly will dramatically improve the evaluation speed (see wrapper code for more details).n_gpu_layers = 40  # Change this value based on your model and your GPU VRAM pool.n_batch = 512  # Should be between 1 and n_ctx, consider the amount of VRAM in your GPU.# Make sure the model path is correct for your system!llm = LlamaCpp(    model_path="./ggml-model-q4_0.bin",    n_gpu_layers=n_gpu_layers,    n_batch=n_batch,    callback_manager=callback_manager,    verbose=True,)llm_chain =


callback_manager=callback_manager,    verbose=True,)llm_chain = LLMChain(prompt=prompt, llm=llm)question = "What NFL team won the Super Bowl in the year Justin Bieber was born?"llm_chain.run(question)     We are looking for an NFL team that won the Super Bowl when Justin Bieber (born March 1, 1994) was born.         First, let's look up which year is closest to when Justin Bieber was born:        * The year before he was born: 1993    * The year of his birth: 1994    * The year after he was born: 1995        We want to know what NFL team won the Super Bowl in the year that is closest to when Justin Bieber was born. Therefore, we should look up the NFL team that won the Super Bowl in either 1993 or 1994.        Now let's find out which NFL team did win the Super Bowl in either of those years:        * In 1993, the San Francisco 49ers won the Super Bowl against the Dallas Cowboys by a score of 20-16.    * In 1994, the San Francisco 49ers won the Super Bowl again, this time against the San Diego Chargers by a score of 49-26.        llama_print_timings:        load time =   238.10 ms    llama_print_timings:      sample time =    84.23 ms /   256 runs   (    0.33 ms per token)    llama_print_timings: prompt eval time =


ms per token)    llama_print_timings: prompt eval time =   238.04 ms /    49 tokens (    4.86 ms per token)    llama_print_timings:        eval time = 10391.96 ms /   255 runs   (   40.75 ms per token)    llama_print_timings:       total time = 15664.80 ms    " We are looking for an NFL team that won the Super Bowl when Justin Bieber (born March 1, 1994) was born. \n\nFirst, let's look up which year is closest to when Justin Bieber was born:\n\n* The year before he was born: 1993\n* The year of his birth: 1994\n* The year after he was born: 1995\n\nWe want to know what NFL team won the Super Bowl in the year that is closest to when Justin Bieber was born. Therefore, we should look up the NFL team that won the Super Bowl in either 1993 or 1994.\n\nNow let's find out which NFL team did win the Super Bowl in either of those years:\n\n* In 1993, the San Francisco 49ers won the Super Bowl against the Dallas Cowboys by a score of 20-16.\n* In 1994, the San Francisco 49ers won the Super Bowl again, this time against the San Diego Chargers by a score of 49-26.\n"Metalâ€‹If the installation with Metal was correct, you will see an NEON = 1 indicator in model properties.Two of the most important parameters for use with GPU are:n_gpu_layers - determines how many layers of the model are offloaded to your Metal GPU, in the most case, set it to


layers of the model are offloaded to your Metal GPU, in the most case, set it to 1 is enough for Metaln_batch - how many tokens are processed in parallel, default is 8, set to bigger number.f16_kv - for some reason, Metal only support True, otherwise you will get error such as Asserting on type 0


GGML_ASSERT: .../ggml-metal.m:706: false && "not implemented"Setting these parameters correctly will dramatically improve the evaluation speed (see wrapper code for more details).n_gpu_layers = 1  # Metal set to 1 is enough.n_batch = 512  # Should be between 1 and n_ctx, consider the amount of RAM of your Apple Silicon Chip.# Make sure the model path is correct for your system!llm = LlamaCpp(    model_path="./ggml-model-q4_0.bin",    n_gpu_layers=n_gpu_layers,    n_batch=n_batch,    f16_kv=True,  # MUST set to True, otherwise you will run into problem after a couple of calls    callback_manager=callback_manager,    verbose=True,)The rest are almost same as GPU, the console log will show the following log to indicate the Metal was enable properly.ggml_metal_init: allocatingggml_metal_init: using MPS...You also could check the Activity Monitor by watching the % GPU of the process, the % CPU will drop dramatically after turn on n_gpu_layers=1. Also for the first time call LLM, the performance might be slow due to the model compilation in Metal GPU.PreviousKoboldAI APINextCaching integrationsInstallationCPU only installationInstallation with OpenBLAS / cuBLAS / CLBlastInstallation with MetalInstallation with WindowsUsageCPUGPUMetalCommunityDiscordTwitterGitHubPythonJS/TSMoreHomepageBlogCopyright Â© 2023 LangChain, Inc.


Bedrock | ğŸ¦œï¸�ğŸ”— Langchain


Skip to main contentğŸ¦œï¸�ğŸ”— LangChainDocsUse casesIntegrationsAPILangSmithJS/TS DocsCTRLKIntegrationsCallbacksChat modelsDocument loadersDocument transformersLLMsAI21Aleph AlphaAmazon API GatewayAnyscaleAzure OpenAIAzureML Online EndpointBananaBasetenBeamBedrockCerebriumAIChatGLMClarifaiCohereC TransformersDatabricksDeepInfraForefrontAIGoogle Cloud Platform Vertex AI PaLMGooseAIGPT4AllHugging Face HubHugging Face Local PipelinesHuggingface TextGen InferenceJSONFormerKoboldAI APILlama-cppCaching integrationsManifestModalMosaicMLNLP CloudoctoaiOpenAIOpenLLMOpenLMPetalsPipelineAIPredibasePrediction GuardPromptLayer OpenAIRELLMReplicateRunhouseSageMakerEndpointStochasticAITextGenTongyi QwenWriterMemoryRetrieversText embedding modelsAgent toolkitsToolsVector storesGrouped by providerIntegrationsLLMsBedrockOn this pageBedrockAmazon Bedrock is a fully managed service that makes FMs from leading AI startups and Amazon available via an API, so you can choose from a wide range of FMs to find the model that is best suited for your use case%pip install boto3from langchain.llms.bedrock import Bedrockllm = Bedrock(    credentials_profile_name="bedrock-admin",    model_id="amazon.titan-tg1-large",    endpoint_url="custom_endpoint_url",)Using in a conversation chainâ€‹from langchain.chains import ConversationChainfrom langchain.memory import ConversationBufferMemoryconversation = ConversationChain(    llm=llm, verbose=True, memory=ConversationBufferMemory())conversation.predict(input="Hi there!")PreviousBeamNextCerebriumAIUsing in a conversation


there!")PreviousBeamNextCerebriumAIUsing in a conversation chainCommunityDiscordTwitterGitHubPythonJS/TSMoreHomepageBlogCopyright Â© 2023 LangChain, Inc.


Hugging Face Local Pipelines | ğŸ¦œï¸�ğŸ”— Langchain


Skip to main contentğŸ¦œï¸�ğŸ”— LangChainDocsUse casesIntegrationsAPILangSmithJS/TS DocsCTRLKIntegrationsCallbacksChat modelsDocument loadersDocument transformersLLMsAI21Aleph AlphaAmazon API GatewayAnyscaleAzure OpenAIAzureML Online EndpointBananaBasetenBeamBedrockCerebriumAIChatGLMClarifaiCohereC TransformersDatabricksDeepInfraForefrontAIGoogle Cloud Platform Vertex AI PaLMGooseAIGPT4AllHugging Face HubHugging Face Local PipelinesHuggingface TextGen InferenceJSONFormerKoboldAI APILlama-cppCaching integrationsManifestModalMosaicMLNLP CloudoctoaiOpenAIOpenLLMOpenLMPetalsPipelineAIPredibasePrediction GuardPromptLayer OpenAIRELLMReplicateRunhouseSageMakerEndpointStochasticAITextGenTongyi QwenWriterMemoryRetrieversText embedding modelsAgent toolkitsToolsVector storesGrouped by providerIntegrationsLLMsHugging Face Local PipelinesOn this pageHugging Face Local PipelinesHugging Face models can be run locally through the HuggingFacePipeline class.The Hugging Face Model Hub hosts over 120k models, 20k datasets, and 50k demo apps (Spaces), all open source and publicly available, in an online platform where people can easily collaborate and build ML together.These can be called from LangChain either through this local pipeline wrapper or by calling their hosted inference endpoints through the HuggingFaceHub class. For more information on the hosted pipelines, see the HuggingFaceHub notebook.To use, you should have the transformers python package installed.pip install transformers > /dev/nullLoad the modelâ€‹from langchain import HuggingFacePipelinellm = HuggingFacePipeline.from_model_id(


HuggingFacePipelinellm = HuggingFacePipeline.from_model_id(    model_id="bigscience/bloom-1b7",    task="text-generation",    model_kwargs={"temperature": 0, "max_length": 64},)    WARNING:root:Failed to default session, using empty session: HTTPConnectionPool(host='localhost', port=8000): Max retries exceeded with url: /sessions (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x1117f9790>: Failed to establish a new connection: [Errno 61] Connection refused'))Integrate the model in an LLMChainâ€‹from langchain import PromptTemplate, LLMChaintemplate = """Question: {question}Answer: Let's think step by step."""prompt = PromptTemplate(template=template, input_variables=["question"])llm_chain = LLMChain(prompt=prompt, llm=llm)question = "What is electroencephalography?"print(llm_chain.run(question))    /Users/wfh/code/lc/lckg/.venv/lib/python3.11/site-packages/transformers/generation/utils.py:1288: UserWarning: Using `max_length`'s default (64) to control the generation length. This behaviour is deprecated and will be removed from the config in v5 of Transformers -- we recommend using `max_new_tokens` to control the maximum length of the generation.      warnings.warn(    WARNING:root:Failed to persist run: HTTPConnectionPool(host='localhost', port=8000): Max retries exceeded with url: /chain-runs (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x144d06910>: Failed to establish a new connection: [Errno 61] Connection refused'))


Failed to establish a new connection: [Errno 61] Connection refused'))     First, we need to understand what is an electroencephalogram. An electroencephalogram is a recording of brain activity. It is a recording of brain activity that is made by placing electrodes on the scalp. The electrodes are placedPreviousHugging Face HubNextHuggingface TextGen InferenceLoad the modelIntegrate the model in an LLMChainCommunityDiscordTwitterGitHubPythonJS/TSMoreHomepageBlogCopyright Â© 2023 LangChain, Inc.


ForefrontAI | ğŸ¦œï¸�ğŸ”— Langchain


Skip to main contentğŸ¦œï¸�ğŸ”— LangChainDocsUse casesIntegrationsAPILangSmithJS/TS DocsCTRLKIntegrationsCallbacksChat modelsDocument loadersDocument transformersLLMsAI21Aleph AlphaAmazon API GatewayAnyscaleAzure OpenAIAzureML Online EndpointBananaBasetenBeamBedrockCerebriumAIChatGLMClarifaiCohereC TransformersDatabricksDeepInfraForefrontAIGoogle Cloud Platform Vertex AI PaLMGooseAIGPT4AllHugging Face HubHugging Face Local PipelinesHuggingface TextGen InferenceJSONFormerKoboldAI APILlama-cppCaching integrationsManifestModalMosaicMLNLP CloudoctoaiOpenAIOpenLLMOpenLMPetalsPipelineAIPredibasePrediction GuardPromptLayer OpenAIRELLMReplicateRunhouseSageMakerEndpointStochasticAITextGenTongyi QwenWriterMemoryRetrieversText embedding modelsAgent toolkitsToolsVector storesGrouped by providerIntegrationsLLMsForefrontAIOn this pageForefrontAIThe Forefront platform gives you the ability to fine-tune and use open source large language models.This notebook goes over how to use Langchain with ForefrontAI.Importsâ€‹import osfrom langchain.llms import ForefrontAIfrom langchain import PromptTemplate, LLMChainSet the Environment API Keyâ€‹Make sure to get your API key from ForefrontAI. You are given a 5 day free trial to test different models.# get a new token: https://docs.forefront.ai/forefront/api-reference/authenticationfrom getpass import getpassFOREFRONTAI_API_KEY = getpass()os.environ["FOREFRONTAI_API_KEY"] = FOREFRONTAI_API_KEYCreate the ForefrontAI instanceâ€‹You can specify different parameters such as the model endpoint url,


ForefrontAI instanceâ€‹You can specify different parameters such as the model endpoint url, length, temperature, etc. You must provide an endpoint url.llm = ForefrontAI(endpoint_url="YOUR ENDPOINT URL HERE")Create a Prompt Templateâ€‹We will create a prompt template for Question and Answer.template = """Question: {question}Answer: Let's think step by step."""prompt = PromptTemplate(template=template, input_variables=["question"])Initiate the LLMChainâ€‹llm_chain = LLMChain(prompt=prompt, llm=llm)Run the LLMChainâ€‹Provide a question and run the LLMChain.question = "What NFL team won the Super Bowl in the year Justin Beiber was born?"llm_chain.run(question)PreviousDeepInfraNextGoogle Cloud Platform Vertex AI PaLMImportsSet the Environment API KeyCreate the ForefrontAI instanceCreate a Prompt TemplateInitiate the LLMChainRun the LLMChainCommunityDiscordTwitterGitHubPythonJS/TSMoreHomepageBlogCopyright Â© 2023 LangChain, Inc.


OpenAI | ğŸ¦œï¸�ğŸ”— Langchain


Skip to main contentğŸ¦œï¸�ğŸ”— LangChainDocsUse casesIntegrationsAPILangSmithJS/TS DocsCTRLKIntegrationsCallbacksChat modelsDocument loadersDocument transformersLLMsAI21Aleph AlphaAmazon API GatewayAnyscaleAzure OpenAIAzureML Online EndpointBananaBasetenBeamBedrockCerebriumAIChatGLMClarifaiCohereC TransformersDatabricksDeepInfraForefrontAIGoogle Cloud Platform Vertex AI PaLMGooseAIGPT4AllHugging Face HubHugging Face Local PipelinesHuggingface TextGen InferenceJSONFormerKoboldAI APILlama-cppCaching integrationsManifestModalMosaicMLNLP CloudoctoaiOpenAIOpenLLMOpenLMPetalsPipelineAIPredibasePrediction GuardPromptLayer OpenAIRELLMReplicateRunhouseSageMakerEndpointStochasticAITextGenTongyi QwenWriterMemoryRetrieversText embedding modelsAgent toolkitsToolsVector storesGrouped by providerIntegrationsLLMsOpenAIOpenAIOpenAI offers a spectrum of models with different levels of power suitable for different tasks.This example goes over how to use LangChain to interact with OpenAI models# get a token: https://platform.openai.com/account/api-keysfrom getpass import getpassOPENAI_API_KEY = getpass()import osos.environ["OPENAI_API_KEY"] = OPENAI_API_KEYShould you need to specify your organization ID, you can use the following cell. However, it is not required if you are only part of a single organization or intend to use your default organization. You can check your default organization here.To specify your organization, you can use this:OPENAI_ORGANIZATION = getpass()os.environ["OPENAI_ORGANIZATION"] = OPENAI_ORGANIZATIONfrom langchain.llms import OpenAIfrom langchain import PromptTemplate,


OPENAI_ORGANIZATIONfrom langchain.llms import OpenAIfrom langchain import PromptTemplate, LLMChaintemplate = """Question: {question}Answer: Let's think step by step."""prompt = PromptTemplate(template=template, input_variables=["question"])llm = OpenAI()If you manually want to specify your OpenAI API key and/or organization ID, you can use the following:llm = OpenAI(openai_api_key="YOUR_API_KEY", openai_organization="YOUR_ORGANIZATION_ID")Remove the openai_organization parameter should it not apply to you.llm_chain = LLMChain(prompt=prompt, llm=llm)question = "What NFL team won the Super Bowl in the year Justin Beiber was born?"llm_chain.run(question)    ' Justin Bieber was born in 1994, so the NFL team that won the Super Bowl in 1994 was the Dallas Cowboys.'If you are behind an explicit proxy, you can use the OPENAI_PROXY environment variable to pass throughos.environ["OPENAI_PROXY"] = "http://proxy.yourcompany.com:8080"PreviousoctoaiNextOpenLLMCommunityDiscordTwitterGitHubPythonJS/TSMoreHomepageBlogCopyright Â© 2023 LangChain, Inc.


Amazon API Gateway | ğŸ¦œï¸�ğŸ”— Langchain


Skip to main contentğŸ¦œï¸�ğŸ”— LangChainDocsUse casesIntegrationsAPILangSmithJS/TS DocsCTRLKIntegrationsCallbacksChat modelsDocument loadersDocument transformersLLMsAI21Aleph AlphaAmazon API GatewayAnyscaleAzure OpenAIAzureML Online EndpointBananaBasetenBeamBedrockCerebriumAIChatGLMClarifaiCohereC TransformersDatabricksDeepInfraForefrontAIGoogle Cloud Platform Vertex AI PaLMGooseAIGPT4AllHugging Face HubHugging Face Local PipelinesHuggingface TextGen InferenceJSONFormerKoboldAI APILlama-cppCaching integrationsManifestModalMosaicMLNLP CloudoctoaiOpenAIOpenLLMOpenLMPetalsPipelineAIPredibasePrediction GuardPromptLayer OpenAIRELLMReplicateRunhouseSageMakerEndpointStochasticAITextGenTongyi QwenWriterMemoryRetrieversText embedding modelsAgent toolkitsToolsVector storesGrouped by providerIntegrationsLLMsAmazon API GatewayOn this pageAmazon API GatewayAmazon API Gateway is a fully managed service that makes it easy for developers to create, publish, maintain, monitor, and secure APIs at any scale. APIs act as the "front door" for applications to access data, business logic, or functionality from your backend services. Using API Gateway, you can create RESTful APIs and WebSocket APIs that enable real-time two-way communication applications. API Gateway supports containerized and serverless workloads, as well as web applications.API Gateway handles all the tasks involved in accepting and processing up to hundreds of thousands of concurrent API calls, including traffic management, CORS support, authorization and access control, throttling, monitoring, and API version management. API Gateway has no minimum fees or startup costs. You pay for the API calls you receive and the amount of data transferred out and, with the API Gateway tiered


the API calls you receive and the amount of data transferred out and, with the API Gateway tiered pricing model, you can reduce your cost as your API usage scales.LLMâ€‹from langchain.llms import AmazonAPIGatewayapi_url = "https://<api_gateway_id>.execute-api.<region>.amazonaws.com/LATEST/HF"llm = AmazonAPIGateway(api_url=api_url)# These are sample parameters for Falcon 40B Instruct Deployed from Amazon SageMaker JumpStartparameters = {    "max_new_tokens": 100,    "num_return_sequences": 1,    "top_k": 50,    "top_p": 0.95,    "do_sample": False,    "return_full_text": True,    "temperature": 0.2,}prompt = "what day comes after Friday?"llm.model_kwargs = parametersllm(prompt)    'what day comes after Friday?\nSaturday'Agentâ€‹from langchain.agents import load_toolsfrom langchain.agents import initialize_agentfrom langchain.agents import AgentTypeparameters = {    "max_new_tokens": 50,    "num_return_sequences": 1,    "top_k": 250,    "top_p": 0.25,    "do_sample": False,    "temperature": 0.1,}llm.model_kwargs = parameters# Next, let's load some tools to use. Note that the `llm-math` tool uses an LLM, so we need to pass that in.tools = load_tools(["python_repl", "llm-math"], llm=llm)# Finally, let's initialize an agent with the tools, the language model, and the type of agent we want to use.agent = initialize_agent(


the language model, and the type of agent we want to use.agent = initialize_agent(    tools,    llm,    agent=AgentType.ZERO_SHOT_REACT_DESCRIPTION,    verbose=True,)# Now let's test it out!agent.run(    """Write a Python script that prints "Hello, world!"""")            > Entering new  chain...        I need to use the print function to output the string "Hello, world!"    Action: Python_REPL    Action Input: `print("Hello, world!")`    Observation: Hello, world!        Thought:    I now know how to print a string in Python    Final Answer:    Hello, world!        > Finished chain.    'Hello, world!'result = agent.run(    """What is 2.3 ^ 4.5?""")result.split("\n")[0]            > Entering new  chain...     I need to use the calculator to find the answer    Action: Calculator    Action Input: 2.3 ^ 4.5    Observation: Answer: 42.43998894277659    Thought: I now know the final answer    Final Answer: 42.43998894277659        Question:     What is the square root of 144?        Thought: I need to use the calculator to find the answer    Action:        > Finished chain.    '42.43998894277659'PreviousAleph


> Finished chain.    '42.43998894277659'PreviousAleph AlphaNextAnyscaleLLMAgentCommunityDiscordTwitterGitHubPythonJS/TSMoreHomepageBlogCopyright Â© 2023 LangChain, Inc.


Anyscale | ğŸ¦œï¸�ğŸ”— Langchain


Skip to main contentğŸ¦œï¸�ğŸ”— LangChainDocsUse casesIntegrationsAPILangSmithJS/TS DocsCTRLKIntegrationsCallbacksChat modelsDocument loadersDocument transformersLLMsAI21Aleph AlphaAmazon API GatewayAnyscaleAzure OpenAIAzureML Online EndpointBananaBasetenBeamBedrockCerebriumAIChatGLMClarifaiCohereC TransformersDatabricksDeepInfraForefrontAIGoogle Cloud Platform Vertex AI PaLMGooseAIGPT4AllHugging Face HubHugging Face Local PipelinesHuggingface TextGen InferenceJSONFormerKoboldAI APILlama-cppCaching integrationsManifestModalMosaicMLNLP CloudoctoaiOpenAIOpenLLMOpenLMPetalsPipelineAIPredibasePrediction GuardPromptLayer OpenAIRELLMReplicateRunhouseSageMakerEndpointStochasticAITextGenTongyi QwenWriterMemoryRetrieversText embedding modelsAgent toolkitsToolsVector storesGrouped by providerIntegrationsLLMsAnyscaleAnyscaleAnyscale is a fully-managed Ray platform, on which you can build, deploy, and manage scalable AI and Python applicationsThis example goes over how to use LangChain to interact with Anyscale service. It will send the requests to Anyscale Service endpoint, which is concatenate ANYSCALE_SERVICE_URL and ANYSCALE_SERVICE_ROUTE, with a token defined in ANYSCALE_SERVICE_TOKENimport osos.environ["ANYSCALE_SERVICE_URL"] = ANYSCALE_SERVICE_URLos.environ["ANYSCALE_SERVICE_ROUTE"] = ANYSCALE_SERVICE_ROUTEos.environ["ANYSCALE_SERVICE_TOKEN"] = ANYSCALE_SERVICE_TOKENfrom langchain.llms import Anyscalefrom langchain import PromptTemplate, LLMChaintemplate = """Question: {question}Answer: Let's think step by step."""prompt = PromptTemplate(template=template, input_variables=["question"])llm = Anyscale()llm_chain


PromptTemplate(template=template, input_variables=["question"])llm = Anyscale()llm_chain = LLMChain(prompt=prompt, llm=llm)question = "When was George Washington president?"llm_chain.run(question)With Ray, we can distribute the queries without asyncrhonized implementation. This not only applies to Anyscale LLM model, but to any other Langchain LLM models which do not have _acall or _agenerate implementedprompt_list = [    "When was George Washington president?",    "Explain to me the difference between nuclear fission and fusion.",    "Give me a list of 5 science fiction books I should read next.",    "Explain the difference between Spark and Ray.",    "Suggest some fun holiday ideas.",    "Tell a joke.",    "What is 2+2?",    "Explain what is machine learning like I am five years old.",    "Explain what is artifical intelligence.",]import ray@ray.remotedef send_query(llm, prompt):    resp = llm(prompt)    return respfutures = [send_query.remote(llm, prompt) for prompt in prompt_list]results = ray.get(futures)PreviousAmazon API GatewayNextAzure OpenAICommunityDiscordTwitterGitHubPythonJS/TSMoreHomepageBlogCopyright Â© 2023 LangChain, Inc.


MosaicML | ğŸ¦œï¸�ğŸ”— Langchain


Skip to main contentğŸ¦œï¸�ğŸ”— LangChainDocsUse casesIntegrationsAPILangSmithJS/TS DocsCTRLKIntegrationsCallbacksChat modelsDocument loadersDocument transformersLLMsAI21Aleph AlphaAmazon API GatewayAnyscaleAzure OpenAIAzureML Online EndpointBananaBasetenBeamBedrockCerebriumAIChatGLMClarifaiCohereC TransformersDatabricksDeepInfraForefrontAIGoogle Cloud Platform Vertex AI PaLMGooseAIGPT4AllHugging Face HubHugging Face Local PipelinesHuggingface TextGen InferenceJSONFormerKoboldAI APILlama-cppCaching integrationsManifestModalMosaicMLNLP CloudoctoaiOpenAIOpenLLMOpenLMPetalsPipelineAIPredibasePrediction GuardPromptLayer OpenAIRELLMReplicateRunhouseSageMakerEndpointStochasticAITextGenTongyi QwenWriterMemoryRetrieversText embedding modelsAgent toolkitsToolsVector storesGrouped by providerIntegrationsLLMsMosaicMLMosaicMLMosaicML offers a managed inference service. You can either use a variety of open source models, or deploy your own.This example goes over how to use LangChain to interact with MosaicML Inference for text completion.# sign up for an account: https://forms.mosaicml.com/demo?utm_source=langchainfrom getpass import getpassMOSAICML_API_TOKEN = getpass()import osos.environ["MOSAICML_API_TOKEN"] = MOSAICML_API_TOKENfrom langchain.llms import MosaicMLfrom langchain import PromptTemplate, LLMChaintemplate = """Question: {question}"""prompt = PromptTemplate(template=template, input_variables=["question"])llm = MosaicML(inject_instruction_format=True, model_kwargs={"do_sample": False})llm_chain =


model_kwargs={"do_sample": False})llm_chain = LLMChain(prompt=prompt, llm=llm)question = "What is one good reason why you should train a large language model on domain specific data?"llm_chain.run(question)PreviousModalNextNLP CloudCommunityDiscordTwitterGitHubPythonJS/TSMoreHomepageBlogCopyright Â© 2023 LangChain, Inc.


Memory | ğŸ¦œï¸�ğŸ”— Langchain


Skip to main contentğŸ¦œï¸�ğŸ”— LangChainDocsUse casesIntegrationsAPILangSmithJS/TS DocsCTRLKIntegrationsCallbacksChat modelsDocument loadersDocument transformersLLMsMemoryCassandra Chat Message HistoryDynamodb Chat Message HistoryEntity Memory with SQLite storageMomento Chat Message HistoryMongodb Chat Message HistoryMotÃ¶rhead MemoryMotÃ¶rhead Memory (Managed)Postgres Chat Message HistoryRedis Chat Message HistoryZep MemoryRetrieversText embedding modelsAgent toolkitsToolsVector storesGrouped by providerIntegrationsMemoryMemoryğŸ“„ï¸� Cassandra Chat Message HistoryApache CassandraÂ® is a NoSQL, row-oriented, highly scalable and highly available database, well suited for storing large amounts of data.ğŸ“„ï¸� Dynamodb Chat Message HistoryThis notebook goes over how to use Dynamodb to store chat message history.ğŸ“„ï¸� Entity Memory with SQLite storageIn this walkthrough we'll create a simple conversation chain which uses ConversationEntityMemory backed by a SqliteEntityStore.ğŸ“„ï¸� Momento Chat Message HistoryThis notebook goes over how to use Momento Cache to store chat message history using the MomentoChatMessageHistory class. See the Momento docs for more detail on how to get set up with Momento.ğŸ“„ï¸� Mongodb Chat Message HistoryThis notebook goes over how to use Mongodb to store chat message history.ğŸ“„ï¸� MotÃ¶rhead MemoryMotÃ¶rhead is a memory server implemented in Rust. It automatically handles incremental summarization in the background and allows for stateless applications.ğŸ“„ï¸� MotÃ¶rhead Memory (Managed)MotÃ¶rhead is a memory server implemented in Rust. It automatically handles incremental summarization in the background and allows


is a memory server implemented in Rust. It automatically handles incremental summarization in the background and allows for stateless applications.ğŸ“„ï¸� Postgres Chat Message HistoryThis notebook goes over how to use Postgres to store chat message history.ğŸ“„ï¸� Redis Chat Message HistoryThis notebook goes over how to use Redis to store chat message history.ğŸ“„ï¸� Zep MemoryREACT Agent Chat Message History with Zep - A long-term memory store for LLM applications.PreviousWriterNextCassandra Chat Message HistoryCommunityDiscordTwitterGitHubPythonJS/TSMoreHomepageBlogCopyright Â© 2023 LangChain, Inc.


MotÃ¶rhead Memory (Managed) | ğŸ¦œï¸�ğŸ”— Langchain


Skip to main contentğŸ¦œï¸�ğŸ”— LangChainDocsUse casesIntegrationsAPILangSmithJS/TS DocsCTRLKIntegrationsCallbacksChat modelsDocument loadersDocument transformersLLMsMemoryCassandra Chat Message HistoryDynamodb Chat Message HistoryEntity Memory with SQLite storageMomento Chat Message HistoryMongodb Chat Message HistoryMotÃ¶rhead MemoryMotÃ¶rhead Memory (Managed)Postgres Chat Message HistoryRedis Chat Message HistoryZep MemoryRetrieversText embedding modelsAgent toolkitsToolsVector storesGrouped by providerIntegrationsMemoryMotÃ¶rhead Memory (Managed)On this pageMotÃ¶rhead Memory (Managed)MotÃ¶rhead is a memory server implemented in Rust. It automatically handles incremental summarization in the background and allows for stateless applications.Setupâ€‹See instructions at MotÃ¶rhead for running the managed version of Motorhead. You can retrieve your api_key and client_id by creating an account on Metal.from langchain.memory.motorhead_memory import MotorheadMemoryfrom langchain import OpenAI, LLMChain, PromptTemplatetemplate = """You are a chatbot having a conversation with a human.{chat_history}Human: {human_input}AI:"""prompt = PromptTemplate(    input_variables=["chat_history", "human_input"],     template=template)memory = MotorheadMemory(    api_key="YOUR_API_KEY",    client_id="YOUR_CLIENT_ID"    session_id="testing-1",    memory_key="chat_history")await memory.init();  # loads previous state from MotÃ¶rhead ğŸ¤˜llm_chain = LLMChain(    llm=OpenAI(),     prompt=prompt,     verbose=True,     memory=memory,)llm_chain.run("hi


verbose=True,     memory=memory,)llm_chain.run("hi im bob")            > Entering new LLMChain chain...    Prompt after formatting:    You are a chatbot having a conversation with a human.            Human: hi im bob    AI:        > Finished chain.    ' Hi Bob, nice to meet you! How are you doing today?'llm_chain.run("whats my name?")            > Entering new LLMChain chain...    Prompt after formatting:    You are a chatbot having a conversation with a human.        Human: hi im bob    AI:  Hi Bob, nice to meet you! How are you doing today?    Human: whats my name?    AI:        > Finished chain.    ' You said your name is Bob. Is that correct?'llm_chain.run("whats for dinner?")            > Entering new LLMChain chain...    Prompt after formatting:    You are a chatbot having a conversation with a human.        Human: hi im bob    AI:  Hi Bob, nice to meet you! How are you doing today?    Human: whats my name?    AI:  You said your name is Bob. Is that correct?    Human: whats for dinner?    AI:        > Finished chain.    "  I'm sorry, I'm not sure what you're asking. Could you please rephrase your


I'm sorry, I'm not sure what you're asking. Could you please rephrase your question?"PreviousMotÃ¶rhead MemoryNextPostgres Chat Message HistorySetupCommunityDiscordTwitterGitHubPythonJS/TSMoreHomepageBlogCopyright Â© 2023 LangChain, Inc.


Momento Chat Message History | ğŸ¦œï¸�ğŸ”— Langchain


Skip to main contentğŸ¦œï¸�ğŸ”— LangChainDocsUse casesIntegrationsAPILangSmithJS/TS DocsCTRLKIntegrationsCallbacksChat modelsDocument loadersDocument transformersLLMsMemoryCassandra Chat Message HistoryDynamodb Chat Message HistoryEntity Memory with SQLite storageMomento Chat Message HistoryMongodb Chat Message HistoryMotÃ¶rhead MemoryMotÃ¶rhead Memory (Managed)Postgres Chat Message HistoryRedis Chat Message HistoryZep MemoryRetrieversText embedding modelsAgent toolkitsToolsVector storesGrouped by providerIntegrationsMemoryMomento Chat Message HistoryMomento Chat Message HistoryThis notebook goes over how to use Momento Cache to store chat message history using the MomentoChatMessageHistory class. See the Momento docs for more detail on how to get set up with Momento.Note that, by default we will create a cache if one with the given name doesn't already exist.You'll need to get a Momento auth token to use this class. This can either be passed in to a momento.CacheClient if you'd like to instantiate that directly, as a named parameter auth_token to MomentoChatMessageHistory.from_client_params, or can just be set as an environment variable MOMENTO_AUTH_TOKEN.from datetime import timedeltafrom langchain.memory import MomentoChatMessageHistorysession_id = "foo"cache_name = "langchain"ttl = timedelta(days=1)history = MomentoChatMessageHistory.from_client_params(    session_id,    cache_name,    ttl,)history.add_user_message("hi!")history.add_ai_message("whats up?")history.messages    [HumanMessage(content='hi!', additional_kwargs={}, example=False),     AIMessage(content='whats up?', additional_kwargs={}, example=False)]PreviousEntity Memory with SQLite storageNextMongodb Chat Message HistoryCommunityDiscordTwitterGitHubPythonJS/TSMoreHomepageBlogCopyright Â© 2023 LangChain, Inc.


Redis Chat Message History | ğŸ¦œï¸�ğŸ”— Langchain

Skip to main contentğŸ¦œï¸�ğŸ”— LangChainDocsUse casesIntegrationsAPILangSmithJS/TS DocsCTRLKIntegrationsCallbacksChat modelsDocument loadersDocument transformersLLMsMemoryCassandra Chat Message HistoryDynamodb Chat Message HistoryEntity Memory with SQLite storageMomento Chat Message HistoryMongodb Chat Message HistoryMotÃ¶rhead MemoryMotÃ¶rhead Memory (Managed)Postgres Chat Message HistoryRedis Chat Message HistoryZep MemoryRetrieversText embedding modelsAgent toolkitsToolsVector storesGrouped by providerIntegrationsMemoryRedis Chat Message HistoryRedis Chat Message HistoryThis notebook goes over how to use Redis to store chat message history.from langchain.memory import RedisChatMessageHistoryhistory = RedisChatMessageHistory("foo")history.add_user_message("hi!")history.add_ai_message("whats up?")history.messages    [AIMessage(content='whats up?', additional_kwargs={}),     HumanMessage(content='hi!', additional_kwargs={})]PreviousPostgres Chat Message HistoryNextZep MemoryCommunityDiscordTwitterGitHubPythonJS/TSMoreHomepageBlogCopyright Â© 2023 LangChain, Inc.


Dynamodb Chat Message History | ğŸ¦œï¸�ğŸ”— Langchain


Skip to main contentğŸ¦œï¸�ğŸ”— LangChainDocsUse casesIntegrationsAPILangSmithJS/TS DocsCTRLKIntegrationsCallbacksChat modelsDocument loadersDocument transformersLLMsMemoryCassandra Chat Message HistoryDynamodb Chat Message HistoryEntity Memory with SQLite storageMomento Chat Message HistoryMongodb Chat Message HistoryMotÃ¶rhead MemoryMotÃ¶rhead Memory (Managed)Postgres Chat Message HistoryRedis Chat Message HistoryZep MemoryRetrieversText embedding modelsAgent toolkitsToolsVector storesGrouped by providerIntegrationsMemoryDynamodb Chat Message HistoryOn this pageDynamodb Chat Message HistoryThis notebook goes over how to use Dynamodb to store chat message history.First make sure you have correctly configured the AWS CLI. Then make sure you have installed boto3.Next, create the DynamoDB Table where we will be storing messages:import boto3# Get the service resource.dynamodb = boto3.resource("dynamodb")# Create the DynamoDB table.table = dynamodb.create_table(    TableName="SessionTable",    KeySchema=[{"AttributeName": "SessionId", "KeyType": "HASH"}],    AttributeDefinitions=[{"AttributeName": "SessionId", "AttributeType": "S"}],    BillingMode="PAY_PER_REQUEST",)# Wait until the table exists.table.meta.client.get_waiter("table_exists").wait(TableName="SessionTable")# Print out some data about the table.print(table.item_count)    0DynamoDBChatMessageHistoryâ€‹from langchain.memory.chat_message_histories import DynamoDBChatMessageHistoryhistory = DynamoDBChatMessageHistory(table_name="SessionTable", session_id="0")history.add_user_message("hi!")history.add_ai_message("whats up?")history.messages    [HumanMessage(content='hi!', additional_kwargs={}, example=False),     AIMessage(content='whats


additional_kwargs={}, example=False),     AIMessage(content='whats up?', additional_kwargs={}, example=False)]DynamoDBChatMessageHistory with Custom Endpoint URLâ€‹Sometimes it is useful to specify the URL to the AWS endpoint to connect to. For instance, when you are running locally against Localstack. For those cases you can specify the URL via the endpoint_url parameter in the constructor.from langchain.memory.chat_message_histories import DynamoDBChatMessageHistoryhistory = DynamoDBChatMessageHistory(    table_name="SessionTable",    session_id="0",    endpoint_url="http://localhost.localstack.cloud:4566",)Agent with DynamoDB Memoryâ€‹from langchain.agents import Toolfrom langchain.memory import ConversationBufferMemoryfrom langchain.chat_models import ChatOpenAIfrom langchain.agents import initialize_agentfrom langchain.agents import AgentTypefrom langchain.utilities import PythonREPLfrom getpass import getpassmessage_history = DynamoDBChatMessageHistory(table_name="SessionTable", session_id="1")memory = ConversationBufferMemory(    memory_key="chat_history", chat_memory=message_history, return_messages=True)python_repl = PythonREPL()# You can create the tool to pass to an agenttools = [    Tool(        name="python_repl",        description="A Python shell. Use this to execute python commands. Input should be a valid python command. If you want to see the output of a value, you should print it out with `print(...)`.",        func=python_repl.run,    )]llm = ChatOpenAI(temperature=0)agent_chain = initialize_agent(    tools,    llm,    agent=AgentType.CHAT_CONVERSATIONAL_REACT_DESCRIPTION,