Project ElliotBot: The AI Co-Pilot for This Website

The goal of ElliotBot was to move beyond a simple, decorative chatbot. I wanted to build a genuine AI co-pilot that could understand the full context of the website and perform useful actions for the user. It needed to be more than just a Q&A machine; it had to be a tool for navigation, discovery, and qualified intent-capture, all while demonstrating modern AI safety practices.

ElliotBot is built on a Retrieval-Augmented Generation (RAG) architecture. Instead of putting the whole knowledge base in front of the model, it retrieves only the passages a given question needs - typically around 2,600 tokens, against roughly 52,600 when the entire index was injected on every message.

Retrieval is hybrid. A BM25 index handles lexical matching, which carries most of the weight here: questions are full of proper nouns a general embedding model has no useful prior for - Copilot Studio, Dataverse, AgentLens, project slugs. Alongside it, Gemini embeddings catch paraphrases, where the visitor's words and the page's words differ. The two rankings are fused with Reciprocal Rank Fusion, which combines by position rather than score - necessary because BM25 is unbounded and query-dependent while cosine similarity is bounded, so weighting the raw scores against each other would need retuning every time the corpus changes.

The lexical layer needs no network call, so it is the layer that never fails. Dense retrieval times out at 400 ms and drops out silently; every failure path falls back to lexical order rather than breaking the conversation.

The index is generated from src/i18n/content/ at build time. Every page is split into section-aware chunks of around 800 tokens, each carrying a breadcrumb prefix so a retrieved fragment states its own provenance. Some units are never split - a use case, a newsletter signal, a role in the career timeline - because splitting them separates a claim from the thing it is about.

The interesting part is what the build refuses to do. 3 conditions fail the build outright rather than warning: a content file with no registered extractor, a document that yields almost no text, and any emitted URL that does not resolve against the real App Router tree. That last check exists because an earlier version of this pipeline cited /work/blog/<slug> for a route that is actually /work/blog/<category>/<slug>. The 2-segment form still matches a real route - the category listing - so it returned HTTP 200 and rendered an empty blog page instead of the article. A soft 404 is worse than a hard one here: nothing reports it, no link checker flags it, and the visitor simply finds nothing. Validating emitted URLs against the router at build time is what turns that silence into a failed build.

Because the chunk build needs no API key, it runs inside the Docker build and cannot go stale. Embeddings do need a key - and the key is a Cloud Run runtime secret, unavailable during docker build - so document vectors are generated offline, quantised to int8, and committed. A CI check fails the pull request if they no longer match the content.

The heart of ElliotBot is its system prompt. This detailed set of instructions acts as the bot's 'constitution,' defining its persona, response format, tool definitions with qualification flows, and security boundaries. It's a powerful example of 'prompt engineering,' where carefully crafted instructions guide the AI's behavior without needing to retrain the entire model.

ElliotBot can do more than just talk. It has access to 5 tools that let it perform actions on behalf of the user. 3 of them include mandatory qualification flows - the bot must collect enough context before firing the tool, so it never submits a vague request:

• `navigateTo`: Navigates to any internal page.
• `searchSite`: Natural-language search over the indexed knowledge base.
• `reportBug`: Asks which page has the issue and what you saw before submitting a structured bug report directly to `/api/bug-report` (no email client popup).
• `bookMentorship`: Asks one qualifying question (architecture, governance, career, etc.) before opening the mentorship booking page with a pre-filled topic.
• `contactElliot`: Asks for context (project, consulting, speaking) before navigating to the contact form with the topic pre-filled.

When the AI determines an action is needed, it returns a structured JSON object that the application interprets, complete with a dedicated tool-execution UI showing a spinner and confirmation.

The tools are declared as native function calls rather than described in the prompt. An earlier version asked the model to reply with a JSON object and recovered it client-side with a regular expression - which meant any answer that happened to contain a code sample could be mistaken for a tool call. Responses now stream as framed NDJSON events, so the server states what each piece is instead of the client guessing.

A truly helpful assistant doesn't always wait to be asked. ElliotBot ships with a proactive layer that monitors the user's current route and surfaces a contextual offer after a short dwell. 11 exact routes have curated messages (about, projects, blog, tech stack, mentorship, clinic, experience, open-source, methodology, use cases, and BuyerCompanion), plus prefix matching for dynamic routes like `/work/projects/` and `/work/blog//` - each page gets its own message, deduplicated per-path via sessionStorage so users never see the same offer twice.

The user experience was a primary focus. Small details were implemented to make the interaction feel more natural and responsive:

Security was a top priority in the design of ElliotBot. A multi-layered approach ensures the assistant operates safely and within its intended purpose, protecting both the user and the application from unintended behavior.

The biggest challenge has been the iterative process of prompt engineering - refining the system instructions to be robust, secure, and capable of consistently producing valid JSON tool calls without leaking system details. The qualification-flow pattern emerged from real failures: early versions of `reportBug` fired on "I want to report a bug" with no context, creating useless tickets. Looking ahead, the roadmap includes deeper page-context awareness (passing the current article or project slug as a structured field, not just a title), expanded proactive triggers with A/B-tested copy, and a streaming-aware Message type to replace the current CSS-only border animation hack.