Tracing LangChain Applications

LangChain is a versatile library that enhances generative AI applications by facilitating the integration of large language models with external data and decision-making processes. It provides tools for model chaining, conversation management, and task handling, enabling complex applications such as chatbots and interactive agents.

deepeval allows anyone to automatically log traces for Langchain applications in just one line of code. All traces will be sent to Confident AI upon setup and all you need to do is login to Confident AI.

info

LangChain traces are marked with the LangChain icon .

Setting up the LangChain Integration

Frst, make sure you have deepeval and langchain installed.

pip install langchain deepeval

Next, set up the LangChain integration through deepeval. Simply import deepeval and call the trace_langchain function before your LangChain application.

import deepeval

deepeval.trace_langchain()

# Your LangChain Applications
...

Your LangChain application is now equipped to automatically log and transmit traces to Confident AI! In addition to LangChain applications, DeepEval also supports LangChain traces as part of a larger LLM application.

tip

If your LLM application was built using a hybrid approach (eg., a bit of langchain, and bit of llama_index, and maybe a bit of custom LLM API calling), you'll want to consider hybrid tracing instead.

Confident AI supports multiple LangChain trace types. For example, when you invoke an LLM through LangChain, attributes such as token count and prompt template are automatically saved and displayed on the platform. Here are all the supported LangChain trace types:

• Chain
• Embedding
• Llm
• Retriever
• Tool
• Generic

Returning Sources

If your RAG application is returning retrieval documents using LangChain's RunnableParallel, you must use assign the output of your RAG chain to the RunnableParallel object using only the output key.

from langchain_core.runnables import RunnableParallel
...

rag_chain_with_source = RunnableParallel(
    {"context": retriever, "question": RunnablePassthrough()}
).assign(output=rag_chain_from_docs) # use the output key

Example: Tracing your LangChain Application

1. Import and Configure Models

Begin by importing necessary modules and configure LangChain with OpenAI's GPT-3.5 Turbo and embeddings.

from langchain import hub
from langchain_openai import OpenAIEmbeddings, ChatOpenAI
from langchain_chroma import Chroma
from langchain_core.runnables import RunnablePassthrough
from langchain_core.output_parsers import StrOutputParser

llm = ChatOpenAI(model="gpt-3.5-turbo-0125")
embeddings = OpenAIEmbeddings()

2. Building the Vector Store

Load web-based documents, process them into manageable chunks, and index using Chroma for efficient retrieval.

from langchain_community.document_loaders import WebBaseLoader
from langchain_text_splitters import RecursiveCharacterTextSplitter
...

loader = WebBaseLoader(web_paths=("https://example.com/data",))
docs = loader.load()
text_splitter = RecursiveCharacterTextSplitter(
    chunk_size=1000, chunk_overlap=200
)
splits = text_splitter.split_documents(docs)
vectorstore = Chroma.from_documents(documents=splits, embedding=embeddings)

3. Constructing the RAG Chain

Create a Retrieval-Augmented Generation chain that integrates retrieval, formatting, and generative responses.

...

retriever = vectorstore.as_retriever()
prompt = hub.pull("rlm/rag-prompt")

def format_docs(docs):
    return "\n\n".join(doc.page_content for doc in docs)

rag_chain = (
    {"context": retriever | format_docs, "question": RunnablePassthrough()}
    | prompt
    | llm
    | StrOutputParser()
)

4. Tracing your Queries

Trace the RAG chain with DeepEval and run queries asynchronously.

import asyncio
import deepeval
...

deepeval.trace_langchain()

async def main():
    queries = [...]
    responses = [rag_chain.invoke(query) for query in queries]
    return responses

asyncio.run(main())