Retrieval-Augmented Generation

RAG supplies external source passages before a model answers. It is useful for private documents and changing policies that the model cannot reliably know from training alone.

#Indexing and querying are different paths

The indexing path loads documents, splits them, creates embeddings and stores text with metadata. The query path retrieves authorized passages and supplies them with the user's question to the generator.

textCopy
Index: document -> meaningful chunks -> embeddings -> searchable store
Query: question -> retrieval and access filters -> passages -> answer with sources

This is a conceptual architecture, not a claim that one Google product is required. Managed search, a vector database or a hybrid system can implement parts of it.

#Tune retrieval before prose

Inspect the returned passages for known questions. A large chunk can dilute the answer with irrelevant content; a small chunk can omit an exception. Top-k and overlap are parameters to evaluate, not universal constants.

Store document version, effective date and source ID. Apply permissions before private text enters model context. Updating a source without updating its index can produce a well-cited but obsolete answer.

#Evaluate support

Ask the model to distinguish supported answers from missing evidence. Then check whether the cited passage actually supports each claim. Include conflicting versions, an absent answer and an unauthorized document in the test set.

RAG reduces some grounding problems but does not eliminate hallucinations. The model may ignore the right passage, misread a condition or follow malicious instructions embedded in a document.

For a concrete fictional policy and complete sample answer, open the RAG policy lab.

Vertex AI RAG documentation↗