Lab: A RAG Policy Assistant

Build an assistant that answers from a small, fictional HR handbook and cites the supporting rule. The important result is a supported answer or an honest missing-evidence response, not simply an index that was created successfully.

#Prepare the source

Use this synthetic handbook entry; it is not a real company's policy or employment guidance:

textCopy
Document: demo-handbook, version 1
Meals: reimburse up to $50 per person per meal with a receipt.
Annual leave: the fictional example provides 10 days after one year.
Sick leave: the fictional example requires a medical certificate.

Preserve document ID, section and version as metadata. Split at meaningful section boundaries. Inspect the chunks so a condition such as “with a receipt” stays with its rule.

#Retrieve before generating

Create embeddings and an index only with an authorized provider or approved local model. The LangChain RAG chapter includes a concrete indexing example and its execution prerequisites.

First inspect retrieval for “Can I claim an $80 meal?” The meal passage must be returned. Then pass the question and passage to the answer stage with instructions to cite the source and avoid adding policy terms.

#A complete illustrative answer

The demo policy does not cover the full $80. It permits up to $50 per person per meal with a receipt, leaving $30 outside that limit. Source: demo-handbook, meals, version 1. This is a fictional policy example.

This answer is prewritten for teaching, not evidence of a live model run.

#Acceptance cases

Test a covered meal, a missing receipt, an unsupported question about parental leave, an updated policy and a user without access. Separate retrieval failure from an answer that ignores the retrieved rule.

When exposing the retriever as a tool, return source IDs and passages rather than only an unsupported answer string. Apply document permissions before retrieval, update embeddings when the source changes, and remove deleted documents from the index.

Improve with multiple documents, metadata filters and reranking only after the basic evidence chain passes.