All projects
RAG
PolicyPilot
Permission-aware RAG assistant
An HR-policy assistant that answers with citations, declines what the handbook does not cover, and never shows an employee a document outside their access groups.
Pythonpgvector (HNSW)Gemini embeddingsFastAPIPostgreSQLDockerpytest
0
permission leaks
held-out
97.1%
citation precision
8/8
unanswerable declined
0.906
answer accuracy
The problem
Generic RAG chatbots invent policies, never say “I don't know”, and can leak manager-only or HR-only documents to anyone who asks.
Architecture
- Question
- Access-group filter
- Vector retrieval
- Grounded answer
- Citations
- Abstain if uncovered
How it works
- Access control enforced inside retrieval, before the model sees any text.
- Every answer carries citations to the exact handbook section.
- Retrieval ablation across full-text, vector, hybrid and reranking chose the default with data.
Safety & reliability
- Poisoned-document tests for prompt injection hidden in the corpus.
- Unanswerable questions are declined instead of guessed.
Results
Retrieval ablation — Recall@1 on the held-out set
Share of questions where the first retrieved chunk is the right one
Full-text search (Postgres FTS)0.562
Vector search (pgvector HNSW)0.906
What I learned
Hybrid search did not beat plain vector search on this corpus. Measuring before choosing avoided extra complexity.