Connecting Claude to a Vector Database

We've explored the theory of RAG and Vector Databases. Now let's look at the actual code flow required to wire this entire system together.

This architecture requires two separate API services: an Embedding API (to convert text to numbers) and the Claude API (to generate the final text).

Step 1: Initialize the Clients

In your Node.js backend, you will need the Anthropic SDK, and you will need an SDK for your Vector Database (e.g., Pinecone).

Note: Anthropic recently partnered with Voyage AI to provide state-of-the-art embedding models, but you can also use OpenAI's embedding API. We'll assume a generic getEmbedding() function here.

Step 2: The Search Function

When the user asks a question, we first need to search the database.

async function retrieveKnowledge(userQuestion) {
  // 1. Convert the user's question into a number array
  const questionVector = await getEmbedding(userQuestion);

  // 2. Query the Vector Database for the top 3 most similar paragraphs
  const dbResponse = await pineconeIndex.query({
    vector: questionVector,
    topK: 3, 
    includeMetadata: true // This ensures the database returns the actual text, not just numbers
  });

  // 3. Extract the text strings from the database response
  const retrievedTextArray = dbResponse.matches.map(match => match.metadata.text);
  
  // 4. Join them into one big string
  return retrievedTextArray.join("\n\n"); 
}

Step 3: The Augmentation and Generation

Now that we have the raw text from the handbook, we inject it into the prompt and call Claude.

async function answerQuestionWithRAG(userQuestion) {
  
  // Step A: Get the context
  const contextText = await retrieveKnowledge(userQuestion);

  // Step B: Build the augmented prompt
  const systemPrompt = `
    You are a helpful assistant. Answer the user's question using ONLY the information provided in the <context> tags.
    If the context does not contain the answer, reply "I do not have enough information to answer that."
  `;

  const augmentedUserPrompt = `
    <context>
    ${contextText}
    </context>

    Question: ${userQuestion}
  `;

  // Step C: Call Claude
  const response = await anthropic.messages.create({
    model: 'claude-3-haiku-20240307', // Haiku is perfect for this!
    max_tokens: 500,
    system: systemPrompt,
    messages: [
      { role: 'user', content: augmentedUserPrompt }
    ]
  });

  return response.content[0].text;
}

Reviewing the Architecture

This function represents a production-ready RAG pipeline.

Notice how we told Claude: "If the context does not contain the answer, reply 'I do not have enough information'."

This is the ultimate guardrail against hallucinations. If the Vector Database fails to find a relevant paragraph (because the user asked about dinosaurs instead of company policy), Claude will read the irrelevant context, realize the answer isn't there, and gracefully refuse to answer.

You have successfully confined the AI to only speak the truth found in your database.