Custom AI chatbot

The brief

The chatbot needed to answer questions from an approved collection of documents rather than rely on general model knowledge. Users needed clear responses, and the team operating the system needed evidence about what happened after launch.

I helped build the product around both of those audiences.

Retrieval quality is product quality

A chatbot can produce fluent text even when it retrieved the wrong passage or lacked the information required to answer. That makes the retrieval layer, citation behavior, and refusal path part of the user experience.

The system was designed to search a curated corpus, assemble relevant context, and keep the final answer connected to the material that supported it. When the available documents could not justify an answer, the workflow needed to preserve that uncertainty.

The approach

The product combined a guided chat surface with operational instrumentation:

• document ingestion and segmentation for retrieval;
• scoped search across the approved corpus;
• prompts that distinguish source material from the user’s question;
• cited responses that make supporting context inspectable;
• guardrails for low-confidence or unsupported requests; and
• a dashboard for conversations, errors, feedback, usage, and overall performance.

This made it possible to improve the system through observed behavior instead of judging it from a handful of demonstration prompts.

Why the admin surface mattered

AI applications create new operational questions. Which questions recur? Where does retrieval miss? Are users giving negative feedback because the source is absent, outdated, or difficult to locate? Did a failure come from the model, an upstream service, or the application?

The administrative experience made those questions visible. It turned the chatbot from a standalone interface into a product the team could monitor and improve.

The takeaway

This project demonstrates my approach to retrieval products: constrain the knowledge boundary, make evidence visible, instrument the complete workflow, and give operators a practical way to learn from real use.