How the chat works
The assistant on this site is an engineering experiment: a chat that runs in the browser, a Worker that talks to the models and a search over the site's own content. This page explains the pieces and the decisions.
The architecture in one diagram
Architecture diagram
The path of a question, from the browser to the providers, and the branches that answer about the site, generate artifacts and stream the response. Tap a layer to see the details.
Main flow
Search, artifacts and streaming
Pick a layer to open its details.
pvsouza-ai Worker
- POST /chat (streaming conversation), POST /title (short title), GET /models (catalog) and GET /health.
- Validates the request origin (CORS) before processing; errors before the stream starts become JSON with a code (origin_not_allowed, invalid_request, upstream_error).
- Provider keys live only as Worker secrets.
AI Gateway
- Cloudflare records latency, tokens, cost and status for every call.
- If the gateway fails with a network error or 5xx, the call is retried once straight to the provider's URL.
- 4xx errors are not retried: they usually mean an invalid key or a rejected request.
- Gemini goes direct, without the gateway's OpenAI-compatible route.
Providers
- Five families: OpenAI, Workers AI (Cloudflare), Cerebras, OpenRouter and Google Gemini.
- The user picks the model in the chat and the Worker routes to the matching provider.
- OpenAI-compatible providers use the gateway URL; Workers AI uses the native binding.
buscar_site
- A tool that queries the public corpus when the question is about Paulo, the site or the chat.
- Combines BM25 (lexical) with cosine similarity over 1024-dimension bge-m3 embeddings.
- The rankings are merged with Reciprocal Rank Fusion (RRF, k=60), without normalizing scores.
- There is no vector database: the corpus fits in memory in the Worker; passages come back numbered and become source chips.
Artifacts
- The model answers with a self-contained HTML or SVG artifact when the request is visual.
- The preview runs in a sandboxed iframe, with no access to the page origin.
- Re-emitting the same artifact creates versions, with a diff between them.
Streaming
- The response arrives over the Vercel AI SDK's UI message stream protocol, on top of Server-Sent Events.
- The Worker writes typed events (text-start, text-delta, text-end, reasoning and finish) and the client renders each chunk.
- The user can stop the generation midway, and the cancellation is forwarded to the provider.
Static site, AI outside it
The site is a Next.js app exported as static files and published on Cloudflare Pages. There is no server or API route on the site: the chat page runs entirely in the browser and talks directly to a separate Cloudflare Worker, published at ai.pvsouza.com. Provider keys live only as Worker secrets.
Worker, gateway and providers
The Worker exposes routes for the streaming conversation, the conversation title, the model catalog and health. Before processing, it validates the request origin.
Calls go through a Cloudflare AI gateway, which records latency, tokens, cost and status per request. If the gateway fails with a network error or a 5xx, the call is retried once straight to the provider's URL; 4xx errors are not retried. The catalog brings together models from five families, and the user picks which one to use in the chat itself.
Streaming
The response streams over the Vercel AI SDK's UI message stream protocol, on top of Server-Sent Events. The client renders each chunk as it arrives, so the text shows up progressively instead of waiting for the whole answer. The user can stop the generation midway, and the cancellation is forwarded to the provider.
Artifacts
When the request is a visual component, the model can answer with a self-contained HTML or SVG artifact. It shows up as a card in the middle of the conversation and opens in a panel, with the preview running in an isolated iframe. A new version of the same artifact becomes a diff instead of replacing the previous one.
Search as a tool: buscar_site
The assistant has a search tool, buscar_site, that queries the site's public corpus. When the question is about Paulo, this site or the chat itself, the model calls the tool with the user's question and the Worker runs retrieval over the corpus chunks.
Retrieval combines two in-memory searches: BM25, which is lexical, and cosine similarity over embeddings precomputed with the multilingual bge-m3 model. The two rankings are merged with Reciprocal Rank Fusion, which adds up the contribution of each position without normalizing scores. There is no managed vector database: the corpus is small and fits in the Worker.
The result goes back to the model as numbered passages, so it answers citing the sources with [N], and to the interface as linked source chips. Passages without a public link are neither numbered nor turned into chips. If the search finds nothing, the model should say it didn't find the information on the site instead of making it up.
What stays out
The corpus is curated and public: no internal data, client names or details that would allow inferring them. The content goes through editorial review and enters the manifest with source, license and checksum before becoming chunks.