Introduction to RAG (Retrieval-Augmented Generation)
In the Intermediate course, we learned how to use Function Calling (Tools) to fetch live data (like the weather) and hand it to Claude.
But what if the tool you need isn't a simple weather API? What if you need Claude to answer questions based on a 10,000-page employee handbook? You cannot pass 10,000 pages into the API on every single request—it would be incredibly slow and cost thousands of dollars.
The solution to this problem is RAG (Retrieval-Augmented Generation).
What is RAG?
RAG is an architectural pattern that combines an LLM with a specialized database.
Instead of forcing Claude to memorize your 10,000-page handbook, or trying to paste the entire handbook into the prompt, RAG acts like an open-book test.
- Retrieval: When a user asks a question, your system searches the handbook and retrieves only the most relevant paragraphs.
- Augmentation: Your system pastes those few paragraphs into the prompt.
- Generation: Claude reads those paragraphs and generates the final answer.
The RAG Workflow
Imagine a user asks: "What is the company policy on remote work?"
Step 1: The Search (Retrieval) Your Node.js app intercepts the user's question. It does NOT send it to Claude yet. Instead, it queries a specialized database containing your handbook. It asks the database: "Find me the paragraphs most similar to this question." The database returns Chapter 4, Section 2 (which is only 500 words long).
Step 2: The Prompt Assembly (Augmentation) Your Node.js app constructs a payload for Anthropic.
const prompt = `
<role>You are an HR Assistant.</role>
<instructions>
Read the provided company policy below. Answer the user's question using ONLY the provided policy. If the policy doesn't mention it, say "I don't know."
</instructions>
<policy>
${retrievedParagraphs} // This is the 500 words from Chapter 4
</policy>
<question>
What is the company policy on remote work?
</question>
`;
Step 3: The API Call (Generation) You send this assembled prompt to Claude. Claude reads the 500 words, understands the context, and generates a perfect, accurate summary for the user.
Why is RAG the Industry Standard?
- Cost: You are only paying Claude to read 500 words, not 10,000 pages.
- Speed: Less input text means much faster generation times.
- Accuracy: By forcing Claude to look only at the retrieved paragraphs, you drastically reduce hallucinations. Claude isn't guessing; it's reading the specific source material you handed it.
- Updatability: If the remote work policy changes tomorrow, you don't need to re-train the AI. You just update the database. The next time the system searches, it pulls the new paragraph, and Claude gives the new answer.
In the next lesson, we will look at exactly how to build the "Search" part of this workflow using Embeddings and Vector Databases.