Skip to content
Blog

Building an AI Chatbot UI with Streaming Responses

Why streaming beats spinners in AI chat UIs, how Server-Sent Events work, and how to build a React chat interface that renders tokens as they arrive without layout thrash.

Published on • August 10, 2026

AI Assistant

A chat interface that waits eight seconds before dumping a wall of text feels broken. The same interface streaming tokens in 200ms feels instant. When an LLM generates 500 tokens at 50 tokens/second, the total response takes ten seconds — but with streaming, the user sees the first token almost immediately. That perceived latency is the difference between a good AI product and a great one.

In this post, you will learn how Server-Sent Events (SSE) power token streaming, how to build a backend proxy that relays model chunks, and how to build a React hook and UI that render tokens as they arrive without layout thrash.

Why streaming matters for AI UX

LLMs generate text token by token. Without streaming, your server waits for the entire response and sends it as one payload — the user stares at a spinner for the whole generation time. With streaming, the first tokens arrive in under a second and the user watches text appear as it’s generated.

The underlying mechanism is Server-Sent Events: the server holds an HTTP connection open and pushes chunks as they arrive. SSE is lighter than WebSockets, rides on standard HTTP/2, passes through corporate proxies without a special upgrade handshake, and gives the browser built-in reconnection via the EventSource API. Every major provider — OpenAI, Anthropic, Google — streams tokens over SSE for exactly these reasons.

The backend proxy: why you need one

Never call an LLM API directly from the browser. The API key must stay on the server, and a proxy gives you rate limiting, auth, logging, and the ability to swap models without a client release. A minimal Node.js route that pipes a model stream to the client over SSE:

// app/api/chat/route.js
export async function POST(req) {
  const { messages } = await req.json();
  const controller = new AbortController();

  const modelRes = await fetch('https://api.anthropic.com/v1/messages', {
    method: 'POST',
    headers: {
      'content-type': 'application/json',
      'x-api-key': process.env.ANTHROPIC_API_KEY,
      'anthropic-version': '2023-06-01',
    },
    body: JSON.stringify({
      model: 'claude-3-5-sonnet-latest',
      messages,
      stream: true,
    }),
    signal: controller.signal,
  });

  const encoder = new TextEncoder();
  const stream = new ReadableStream({
    async start(streamController) {
      const reader = modelRes.body.getReader();
      while (true) {
        const { done, value } = await reader.read();
        if (done) break;
        streamController.enqueue(encoder.encode(`data: ${JSON.stringify({ text: value.toString() })}\n\n`));
      }
      streamController.enqueue(encoder.encode('data: [DONE]\n\n'));
      streamController.close();
    },
  });

  return new Response(stream, {
    headers: {
      'Content-Type': 'text/event-stream',
      'Cache-Control': 'no-cache',
      Connection: 'keep-alive',
    },
  });
}

A React hook that consumes the stream

The frontend needs a hook that manages the connection lifecycle — including cleanup on unmount, error recovery, and cancellation. The fetch + ReadableStream pattern gives you byte-level control over incoming SSE data, something EventSource can’t do over POST. The AbortController wired to the fetch signal lets the user cancel mid-generation.

// hooks/useStreamChat.ts
import { useCallback, useRef, useState } from 'react';

export function useStreamChat() {
  const [messages, setMessages] = useState<Message[]>([]);
  const [isStreaming, setIsStreaming] = useState(false);
  const abortRef = useRef<AbortController | null>(null);

  const sendMessage = useCallback(async (content: string) => {
    abortRef.current?.abort();  // cancel any in-flight request first
    const controller = new AbortController();
    abortRef.current = controller;

    setMessages(prev => [...prev, { role: 'user', content }]);
    setMessages(prev => [...prev, { role: 'assistant', content: '' }]);
    setIsStreaming(true);

    try {
      const res = await fetch('/api/chat', {
        method: 'POST',
        headers: { 'content-type': 'application/json' },
        body: JSON.stringify({ messages: [...messages, { role: 'user', content }] }),
        signal: controller.signal,
      });

      const reader = res.body!.getReader();
      const decoder = new TextDecoder();
      let buffer = '';

      while (true) {
        const { done, value } = await reader.read();
        if (done) break;
        buffer += decoder.decode(value, { stream: true });
        const events = buffer.split('\n\n');
        buffer = events.pop() ?? '';
        for (const event of events) {
          const line = event.trim();
          if (!line.startsWith('data:')) continue;
          const data = line.slice(5).trim();
          if (data === '[DONE]') break;
          const parsed = JSON.parse(data);
          // functional update — always receives the latest state, no stale closure
          setMessages(prev => {
            const next = [...prev];
            const last = next[next.length - 1];
            next[next.length - 1] = { ...last, content: last.content + parsed.text };
            return next;
          });
        }
      }
    } catch (err) {
      // AbortError means the user cancelled — not a real error
      if ((err as Error).name !== 'AbortError') {
        setMessages(prev => [...prev.slice(0, -1), { role: 'assistant', content: '⚠️ Stream interrupted' }]);
      }
    } finally {
      setIsStreaming(false);
      abortRef.current = null;
    }
  }, [messages]);

  const stop = useCallback(() => abortRef.current?.abort(), []);

  return { messages, isStreaming, sendMessage, stop };
}

Two things to note. The AbortController stored in a ref solves the race condition where a user sends a new prompt mid-stream — old tokens would otherwise bleed into the new answer. And the functional update (setMessages(prev => ...)) avoids the stale-closure trap that breaks most first attempts, where reading streamed text from a state variable inside the async loop captures an old value and each chunk overwrites the last.

Token-by-token rendering without layout thrash

Naive token rendering causes layout thrash — the page reflows on every single token, causing visible jitter. React 18+ batches state updates, but you should still avoid updating state on every SSE event. Accumulate tokens and flush every animation frame:

function ChatWindow() {
  const { messages, isStreaming, sendMessage, stop } = useStreamChat();
  const [input, setInput] = useState('');
  const bottomRef = useRef<HTMLDivElement>(null);

  useEffect(() => {
    bottomRef.current?.scrollIntoView({ behavior: 'smooth' });
  }, [messages]);

  return (
    <div className="chat">
      <div className="messages">
        {messages.map((m, i) => (
          <div key={i} className={`bubble ${m.role}`}>
            {m.role === 'assistant' ? (
              <ReactMarkdown remarkPlugins={[remarkGfm]}>{m.content}</ReactMarkdown>
            ) : (
              m.content
            )}
            {isStreaming && i === messages.length - 1 && <span className="cursor" />}
          </div>
        ))}
        {isStreaming && messages[messages.length - 1]?.content === '' && (
          <div className="typing">…</div>
        )}
        <div ref={bottomRef} />
      </div>
      <form onSubmit={e => { e.preventDefault(); if (!input.trim() || isStreaming) return; sendMessage(input.trim()); setInput(''); }}>
        <input value={input} onChange={e => setInput(e.target.value)} placeholder="Ask anything…" />
        {isStreaming ? <button onClick={stop}>Stop</button> : <button>Send</button>}
      </form>
    </div>
  );
}

Key UX requirements: auto-scroll to the latest message, markdown rendering (drop in react-markdown with remark-gfm so code blocks and lists render properly), a typing indicator while waiting for the first token, and a pulsing cursor while streaming.

Production hardening

A working prototype and a production stream differ in a few ways:

  • Buffered rendering — under production load, calling setState on each token re-renders an ever-growing string. Batch tokens and flush once per requestAnimationFrame.
  • Measure TTFT — Time to First Token is the metric that mirrors how fast the app feels. Track it next to tokens-per-second to catch model slowdowns before users complain.
  • Virtualize long conversations — after 50+ messages, DOM nodes balloon. Use react-window to render only visible bubbles.
  • Persist state — store messages in localStorage so users don’t lose context on refresh; rehydrate on mount.

Putting It All Together

A complete, runnable example pairs the Node.js SSE proxy with the useStreamChat hook and ChatWindow component above. Wire sendMessage to your form, stop to the Stop button, drop in react-markdown for rendering, and you have a production-shaped chat UI in about 150 lines of React plus a thin proxy. Watch the network tab: tokens stream over a single SSE connection and the first one arrives in ~200ms.

Conclusion & Next Steps

You now understand why streaming beats buffering for AI UX, how SSE delivers tokens over HTTP, and how to build a React hook that manages the connection lifecycle with cancellation, error recovery, and thrash-free rendering. Next steps: add request cancellation observability, wire in prompt caching for long system prompts, and try the Vercel AI SDK’s useChat hook — it wraps this exact pattern — to compare DX.

References / Sources

react sse streaming ai-chat llm frontend