Retrieval-augmented generation, or RAG, connects a language model to knowledge sources selected by an organization. When a question arrives, the system retrieves related passages and sends them to a model with instructions for composing an answer. This does not train a new model on every document, and it does not make an answer correct by default. The result depends on content quality, permissions, retrieval, instructions, and evaluation.
RAG becomes useful when knowledge changes regularly or a user must inspect the source of an answer: policies, procedures, product manuals, operating documents, contracts, support knowledge, or internal technical material. It needs enterprise design because a search result can expose information that a user is not permitted to see, and an obsolete policy can sound as convincing as a current one.
When is RAG a suitable pattern?
Consider RAG when answers should be grounded in an identifiable collection of text and users need search, summarization, comparison, or explanation with citations. It can also fit when fixed rules cannot cover the variety of questions or when documents change more frequently than a custom model could be retrained and reviewed.
Signals that support its use
- Knowledge is distributed across files, pages, and several systems.
- An approved version and a clear content owner can be identified.
- Results must respect the requesting user’s identity and permissions.
- Answers can be evaluated against reference questions and sources.
- A user needs a link or short quotation to verify the response.
Do not make RAG the default for a calculation or forecast that needs a governed query or statistical model. If a value is a fixed fact in one structured field, a direct API may be simpler and more accurate. RAG also cannot repair unmanaged knowledge. Conflicting, ownerless, or obsolete documents require content governance before indexing.
Architecture: from source to answer
A practical architecture has five areas: source systems, ingestion and indexing, retrieval, generation, and the user experience with monitoring. Keeping them separate allows the team to test each stage and prevents fluent wording from hiding a retrieval failure.
1. Ingestion and preparation
The ingestion pipeline reads a document and its metadata, checks its type, version, owner, validity period, and access classification, then extracts text and divides it into retrievable passages. Each passage should retain a link to its source, page, version, and sensitivity. One fixed chunk size can separate a heading from its paragraph or a table from its explanation, so test chunking approaches against the actual content structure.
2. Indexing and retrieval
Retrieval may combine keyword matching, semantic representations, and structured filters such as department, language, and effective date. Hybrid search is useful when product codes and exact terms coexist with natural-language questions. A reranking stage can place closer passages first, but its contribution to latency and cost should be measured.
3. Generation and citations
The generation layer supplies the question, permitted passages, and instructions such as: answer from the context, say when information is insufficient, and associate each conclusion with a source. The system should not ask the model to improvise when evidence is missing. The interface should display a useful document name, version, and link rather than an internal identifier the reader cannot interpret.
Authorization must happen before retrieval
A serious architectural error is to build one unrestricted index and hide results only after the model has read them. Access filters should be applied during retrieval using the user’s identity, groups, and document attributes. When a membership or document classification changes, the index needs a traceable update. Sensitive passages should not be copied into logs solely for monitoring convenience.
Define a threat model. Can a user place hostile instructions in a question? Can an indexed document contain text that tries to override system instructions? Could a file title or URL reveal a confidential topic? Treat retrieved content as untrusted data, separate it from system instructions, and constrain any tool or action available to the response.
Access-control checklist
- Synchronize identity groups with index filters and test them on a schedule.
- Apply least privilege to source connectors and service accounts.
- Remove or invalidate an indexed document when it is removed at the source.
- Mask sensitive fields that are not required before sending context to a model.
- Record source identifiers and decisions without storing full text unnecessarily.
- Test isolation between departments, projects, languages, and guest roles.
Content governance matters more than index size
Before adding a source, assign an owner, approval status, review interval, and expiry rule. Metadata should distinguish a draft from a published version and a current policy from an expired one. If sources conflict, apply a documented priority rule or show the conflict rather than blending two positions. A review and deletion cycle limits the accumulation of stale material that is difficult to recognize after launch.
Multilingual collections require deliberate testing. Arabic questions may mix English product terms, use different forms of letters and numbers, or rely on tables extracted from scanned PDF files. Build a bilingual evaluation set from realistic user questions and do not assume retrieval performance in one language transfers to another.
Evaluate retrieval and generation separately
Start with reference questions that identify the passages the system should retrieve and the characteristics of an acceptable answer. Measure whether the correct source appears, whether it ranks near the top, and whether disallowed material is excluded. Then assess the response: is it supported by the passages, is the citation accurate, and does it state clearly when information is unavailable?
Tests that should be included
- A clear question with one answer in an approved document.
- An ambiguous question that should trigger a clarification request.
- Two documents that conflict or have different effective dates.
- A question with no answer in the available sources.
- An attempt to access a document outside the user’s permission.
- Text inside a document that tries to direct the model or call a tool.
- Arabic, English, and mixed-language questions using technical terms.
Repeat the evaluation when changing embeddings, chunking, instructions, filters, reranking, or the language model. In production, monitor empty retrievals, cited sources, latency, request cost, and user feedback while preserving the privacy of operational logs.
Move to production with a bounded scope
Start with one or two sources that have clear ownership and permissions, a limited user group, and questions that reviewers can assess. Display citations, provide a reporting control, and retain a conventional search route. Expand only after the team understands failure cases and has updated content, filters, and evaluations. This makes RAG an operated knowledge service rather than a conversation demonstration.
Pre-production checklist
- Approved sources, owners, review schedules, and expiry rules are documented.
- Chunking, metadata, and retrieval were tested on representative content.
- Authorization is enforced before context reaches the language model.
- Citations open the correct source and version for the requesting user.
- The evaluation set covers absence, conflict, denial, injection, and both languages.
- Monitoring, deletion, rollback, and incident processes have named owners.
If enterprise knowledge is spread across documents and systems, begin by assessing the sources, permissions, and user task before selecting tools. Discuss a RAG or knowledge-integration path with Tech Wonder through the English contact page.