AI Docs Assistant
Code on GitHub, no live demoA chat assistant that answers questions from your own documentation and shows which passages each answer came from. Python and FastAPI back end, Next.js front end, works with Claude, OpenAI or offline.
Architecture
Problem
Documentation bots often sound confident while making things up, and it is hard to see where an answer came from. A team needs an assistant it can trust to stay inside its own content and to admit when the answer is not there.
Solution
Built a retrieval based assistant. The Python back end splits documents by heading, finds the passages that match a question with BM25 search, and streams the answer back as it is written. The Next.js front end shows clickable citations next to every answer. A question with nothing relevant in the docs never reaches the model and gets an honest "could not find it" reply instead of a guess.
Technology
Key Features
- Answers stream live with clickable source citations
- A relevance check keeps off topic questions away from the model
- Works with Claude, OpenAI, or an offline mode that needs no API key
- Short follow up questions use the conversation history
- Input validation, rate limiting and API keys kept on the server
- 58 automated tests, including a retrieval accuracy check on 18 questions
What I Learned
Search quality decides answer quality, so I tested retrieval on its own against real questions before connecting any model. That caught a stemming bug that would have hidden some documents. A browser test also caught a message like "tell me a joke" inheriting the previous question's topic, which now has a regression test.