All projects
RAG

PolicyPilot

Permission-aware RAG assistant

An HR-policy assistant that answers with citations, declines what the handbook does not cover, and never shows an employee a document outside their access groups.

Live demo Source code
Pythonpgvector (HNSW)Gemini embeddingsFastAPIPostgreSQLDockerpytest

0

permission leaks

held-out

97.1%

citation precision

8/8

unanswerable declined

0.906

answer accuracy

The problem

Generic RAG chatbots invent policies, never say “I don't know”, and can leak manager-only or HR-only documents to anyone who asks.

Architecture

  1. Question
  2. Access-group filter
  3. Vector retrieval
  4. Grounded answer
  5. Citations
  6. Abstain if uncovered

How it works

  • Access control enforced inside retrieval, before the model sees any text.
  • Every answer carries citations to the exact handbook section.
  • Retrieval ablation across full-text, vector, hybrid and reranking chose the default with data.

Safety & reliability

  • Poisoned-document tests for prompt injection hidden in the corpus.
  • Unanswerable questions are declined instead of guessed.

Results

Retrieval ablation — Recall@1 on the held-out set

Share of questions where the first retrieved chunk is the right one

Full-text search (Postgres FTS)0.562
Vector search (pgvector HNSW)0.906

What I learned

Hybrid search did not beat plain vector search on this corpus. Measuring before choosing avoided extra complexity.