Why This Site Needed a Real Chatbot Integration

A few weeks ago I added an Ask AI button to this site.

It worked. Technically.

You typed a question, it sent my corpus to the model, and it replied from my own writing. No personality theater, no fake support voice, no made-up facts. Just grounded answers from /llms-full.txt.

On paper, that sounds fine.

In practice, it felt wrong the moment you used it.

The chat opened as a clean panel. You asked a question. Then nothing moved for a bit. No visible response line, no hint that the system was alive, just a quiet empty area that looked like it had stalled.

It was doing work in the background. It just did not feel like it.

That gap is why I rebuilt it.

Why a blog needs a chatbot at all

Before the UX story, the more basic question deserves an answer: why does a personal site need a chatbot in the first place?

This is not a large archive. Right now it is eleven documents: a handful of long essays and a few project notes. Small enough to read in an evening. Too small, on its own, to justify “AI-powered search” as a category.

But most visitors do not arrive at the front page and read in order. They land on one essay from a share, a search result, or a link inside some other post. From there, the rest of the site is invisible to them. Tags help if the reader already knows the word to search for. Tags do not help with “has he written about X,” when X is not a tag, or when the answer sits split across two essays that never link to each other.

That is the actual gap: not information overload, but information the reader cannot see from where they are standing.

A chatbot grounded in my own writing closes that gap. Ask the specific question, get pointed at the specific answer, or get told plainly that it is not covered. It does not replace reading an essay. It replaces skimming everything hoping one piece happens to match, or leaving without checking at all.

If this site’s only job were to look good, the chatbot would not be worth building. But its job is to be read, and to be useful to someone with one specific question. For that job, even a small, honest corpus needs something at the door that can point.

Once I accepted the chatbot itself was worth having, the next problem showed up immediately: the first version was not good enough to keep that promise.

Working is not the same as believable

The first version was a small custom script wired into my Astro layout. It used direct fetch calls to my serverless endpoint and rendered full answers when they came back.

That design was simple, and simplicity is usually my bias.

But chat has one sharp UX rule: users need immediate evidence of progress. If the interface stays still after submit, people assume failure long before your model has a chance to prove itself.

A normal form can wait. A chatbot cannot.

The old version had three real issues:

1. It returned buffered responses, so there was no token-level progression.
2. It used manual DOM management, which made state transitions brittle.
3. It offered no composable path to richer chat behaviors without writing more one-off UI code.

None of these are dramatic bugs. Together, they create low confidence.

And low confidence is fatal in a chat interface.

Why I moved to AI Elements

I switched the frontend to AI Elements, the component layer built for AI-native UIs.

This was not a design refresh project. It was a trust project.

AI Elements gave me a structured conversation shell, message primitives, prompt input primitives, scroll behavior, and a status-driven submit control that already understands chat states.

That matters because hand-rolled chat UIs mostly fail at the same places:

• ambiguous loading states
• inconsistent message rendering
• fragile control logic as features grow

When those pieces come pre-structured, I can spend effort on product behavior, not repeated widget plumbing.

Why I moved transport to Vercel AI SDK

UI components alone were not enough. The transport layer had to change too.

So I shifted to Vercel AI SDK for the chat loop and streaming flow.

On the client, I now use useChat with the SDK transport. On the server, I stream model output as UI message events instead of waiting for a full response blob.

The visible result is simple:

• users see an assistant row quickly
• streamed text appears progressively
• stop behavior is native, not bolted on

The structural result is bigger:

• request and message flow are normalized
• state management is less custom
• future features have a stable base

This is the classic build-vs-adopt decision. I still like writing small systems from scratch. I do not like rewriting solved problems around streaming semantics and chat state machines.

Grounding stayed the same, and that was important

One thing I did not change was the truth model.

The assistant is still grounded on this site’s own corpus. The server still builds its system prompt around the generated /llms-full.txt content, so answers are constrained to what I have actually published.

I care about that constraint more than any UI effect.

A nicer chatbot that hallucinates confidently is worse than an ugly chatbot that says “I don’t know.” The integration was meant to improve responsiveness and trust, not relax factual boundaries.

The part that looked “empty” and what fixed it

After the integration, one subtle issue remained.

Streaming was technically on, but if the model took a moment before the first token, the panel still looked empty right after submit. No bug in transport, just first-token latency.

That is where perception and implementation diverge.

So I replaced that gap with a pinned assistant waiting row that appears immediately on submit and stays visible until actual assistant text arrives, with rotating status lines tied to the site’s own content context.

The user does not grade your architecture. They grade what the interface does in the first second after they press Enter.

That one state transition changed the feel of the whole system.

What the metrics said (July 8, 2026)

I ran a small production benchmark against the live endpoint (/api/v1/chat/stream) for the same prompt, 12 times:

"Summarize Abhiram writing themes in one sentence"

Two things are clear from this:

1. Transport startup is fast and stable.
2. First-token latency can still be high and variable.

That is exactly why the waiting-state UX mattered. If first token can take 10+ seconds, users need continuous, trustworthy progress feedback from the first click, not a blank panel.

Why this integration was actually necessary

I could have left the original script in place and called it “good enough.” It answered questions. It was cheap to run. It was already deployed.

But “good enough” chat is expensive in a quieter way.

It costs retries. It costs trust. It costs the user’s willingness to ask the second question.

This integration fixed that tax.

Not because the stack is fancier, but because the interaction is clearer:

• submit gives instant feedback
• waiting is visible, not ambiguous
• output arrives progressively
• the chat behaves like a live system, not a static form

If the goal is to help someone get value from your writing, that behavior is not optional polish. It is core product behavior.

The engineering lesson I am keeping

I lead a documentation team, so I naturally optimize for clarity.

This migration reminded me that interface clarity is not just wording. It is timing, state visibility, and honest feedback loops.

In docs, we say “show progress” with headings and structure. In chat, we say it with streamed tokens, pending states, and predictable controls.

Same principle, different medium.

I still like small systems. I still value simple architecture. I still think custom code is often the right call.

But for this problem, integrating a purpose-built UI layer plus a purpose-built streaming transport was the right trade.

Because the real requirement was never “have a chatbot.”

The requirement was: make the chatbot feel alive quickly, stay grounded, and earn trust every turn.