Retrieval-Augmented Generation (RAG) has become a cornerstone technique for enhancing Large Language Models (LLMs) with domain-specific knowledge. In this guide, you'll learn how to build a RAG application using the langchain-snowflake package with SnowflakeCortexSearchRetriever and ChatSnowflake. You'll then evaluate your RAG's performance using TruLens and Snowflake's AI Observability features.
By combining Snowflake's data platform capabilities with LangChain's flexible framework, you'll create a powerful RAG system that can answer questions based on your own data.
A complete RAG application that can answer questions about sales conversations by retrieving relevant information from a knowledge base and generating contextually appropriate responses. You'll also implement evaluation metrics to measure the quality of your RAG system.
Firstly, to follow along with this quickstart, you can click on build-and-evaluate-rag-with-langchain-and-snowflake.ipynb to download the Notebook from GitHub.
Before we can build our RAG application, we need to prepare our Snowflake environment with data and search capabilities. This involves:
Run the SQL commands in the setup.sql file in your Snowflake environment to:
Notebooks come pre-installed with common Python libraries for data science and machine learning. For this guide, we'll need to install additional packages specific to our RAG implementation:
%pip install --quiet -U langchain-core langchain-snowflake trulens-core trulens-providers-cortex trulens-connectors-snowflake trulens-apps-langchain
To securely connect to Snowflake, we'll configure our credentials as environment variables. This approach keeps sensitive information out of your code and follows security best practices.
import os
os.environ["SNOWFLAKE_ACCOUNT"] = "your_account_identifier"
os.environ["SNOWFLAKE_USER"] = "your_username"
os.environ["SNOWFLAKE_PASSWORD"] = "your_password"
os.environ["SNOWFLAKE_WAREHOUSE"] = "SALES_INTELLIGENCE_WH"
os.environ["SNOWFLAKE_DATABASE"] = "SALES_INTELLIGENCE"
os.environ["SNOWFLAKE_SCHEMA"] = "DATA"
Now we'll create a session that will be used by both the retriever and LLM components:
from langchain_snowflake import create_session_from_env
session = create_session_from_env()
This session will handle authentication and connection management for all Snowflake operations in our RAG application.
The retriever is responsible for finding relevant documents from our knowledge base. We'll use SnowflakeCortexSearchRetriever which leverages Snowflake's Cortex Search capabilities:
from langchain_snowflake import SnowflakeCortexSearchRetriever
# Replace with your actual Cortex Search service
CORTEX_SEARCH_SERVICE = "SALES_INTELLIGENCE.DATA.SALES_CONVERSATION_SEARCH"
retriever = SnowflakeCortexSearchRetriever(
session=session,
service_name=CORTEX_SEARCH_SERVICE,
k=3, # Number of documents to retrieve
auto_format_for_rag=True, # Automatic document formatting
content_field="TRANSCRIPT_TEXT", # Extract content from this metadata field
join_separator="\n\n", # Separator used for multiple documents
fallback_to_page_content=True # Fall back to page_content if metadata field is empty
)
Let's test our retriever to make sure it's working correctly:
import textwrap
docs = retriever.get_relevant_documents("What happened in our last sales conversation with DataDriven?")
for doc in docs:
wrapped_text = textwrap.fill(doc.page_content, width=80)
print(f"{wrapped_text}\n{'-' * 120}")
The retriever should return the most relevant documents from our sales conversations that match the query about DataDriven.
Now we'll set up the language model component using Snowflake's Cortex LLM capabilities:
from langchain_snowflake import ChatSnowflake
# Initialize chat model
llm = ChatSnowflake(
session=session,
model="claude-4-sonnet",
temperature=0.1,
max_tokens=1000
)
We'll create a prompt template that instructs the model to answer questions based on the retrieved context:
from langchain_core.prompts import ChatPromptTemplate
# Create RAG prompt template
rag_prompt = ChatPromptTemplate.from_template("""
Answer the question based on the following context from Snowflake Cortex Search:
Context:
{context}
Question: {question}
Provide a comprehensive answer based on the retrieved context. If the context doesn't contain enough information, say so clearly.
""")
Now we'll chain everything together using LangChain's LCEL (LangChain Expression Language):
from langchain_core.runnables import RunnablePassthrough
# Build RAG chain
rag_chain = (
{"context": retriever, "question": RunnablePassthrough()} | rag_prompt | llm
)
Let's test our RAG chain with a sample question:
response = rag_chain.invoke("What happened in our last sales conversation with DataDriven?")
print(f"{response.content}")
The RAG chain should retrieve relevant context about DataDriven from our knowledge base and generate a comprehensive answer based on that context.
To evaluate our RAG application, we'll use TruLens with Snowflake integration:
from trulens.apps.langchain import TruChain
from trulens.connectors.snowflake import SnowflakeConnector
tru_snowflake_connector = SnowflakeConnector(snowpark_session=session)
app_name = "sales_assistance_rag"
app_version = "cortex_search"
tru_rag = TruChain(
rag_chain,
app_name=app_name,
app_version=app_version,
connector=tru_snowflake_connector,
main_method_name="invoke"
)
We'll create a run configuration and dataset for our evaluation:
import pandas as pd
from trulens.core.run import Run, RunConfig
from datetime import datetime
# Create a dataset of test queries
queries = [
"What happened in our last sales conversation with DataDriven?",
"What is the status of the deal with DataDriven?",
"What is the status of the deal with HealthTech?"
]
queries_df = pd.DataFrame(queries, columns=["query"])
# Create a run configuration
run_name = f"experiment_1_{datetime.now().strftime('%Y%m%d%H%M%S')}"
run_config = RunConfig(
run_name=run_name,
dataset_name="sales_queries",
source_type="DATAFRAME",
dataset_spec={
"input": "query",
},
)
# Create and start the run
run: Run = tru_rag.add_run(run_config=run_config)
run.start(input_df=queries_df)
We'll compute several key metrics to evaluate our RAG system:
import time
# Wait for all invocations to complete
while run.get_status() != "INVOCATION_COMPLETED":
time.sleep(3)
# Compute metrics
run.compute_metrics([
"answer_relevance",
"context_relevance",
"groundedness",
])
These metrics will help us understand:
You can view the evaluation results in Snowflake's AI Observability UI: Open in Snowflake AI Observability
Congratulations! You've successfully built and evaluated a complete RAG application using LangChain and Snowflake. You've learned how to create a retriever using Snowflake Cortex Search, build a RAG chain with LangChain, and evaluate its performance using TruLens and Snowflake's AI Observability features.
This foundation can be extended in numerous ways, such as experimenting with different LLM models, adjusting retrieval parameters, adding conversation memory, or building a user interface with Streamlit.
Documentation:
Happy building with LangChain and Snowflake!