Streaming API Responses

If you've ever used the claude.ai web interface, you've noticed that the text appears on your screen word-by-word, exactly like someone is typing it out.

If you use the standard API call we've looked at so far, your app won't do that. Instead, your app will freeze for 5 seconds while Claude generates the entire response, and then the whole block of text will appear instantly.

For a user, waiting 5 seconds staring at a blank screen feels broken. To fix this, you need to use Streaming.

What is Streaming?

Instead of waiting for the kitchen to cook the entire 5-course meal before handing it out the window, streaming is like the kitchen handing you each ingredient the second it is ready.

Under the hood, the API sends the response back to your server in tiny chunks (Server-Sent Events) over an open connection.

How to Stream with the SDK

Anthropic's SDK makes streaming incredibly easy. You just add stream: true to the payload and use the .stream() method.

async function streamResponse() {
  const stream = await anthropic.messages.stream({
    model: 'claude-3-haiku-20240307',
    max_tokens: 500,
    messages: [{ role: 'user', content: 'Write a long story about a dog.' }],
  });

  // This loop runs every time Claude generates a new chunk of text
  for await (const chunk of stream) {
    if (chunk.type === 'content_block_delta' && chunk.delta.text) {
      // Print the chunk to the console without a newline, creating a typing effect
      process.stdout.write(chunk.delta.text);
    }
  }
}

When to Stream vs When to Wait

Use Streaming When:

• You are building a chat UI for a human user. Humans hate waiting. Seeing the text stream in keeps them engaged and makes the app feel lightning-fast.

Wait for the Full Response (No Streaming) When:

• Your code is running in the background. If you are extracting JSON data to save to a database, you can't save half a JSON object. You need the whole thing. In this case, streaming just adds unnecessary complexity. Just wait for the full response, parse the JSON, and save it.