Design a Production RAG System — System Design Practice | AmanAI Lab

Sheet›Design a Production RAG System

MSFTAmazonGoogleAnth

45:00

Hard

The Problem

Problem

Design a production RAG (Retrieval-Augmented Generation) system for an enterprise that needs to answer questions over 1 million internal documents (PDFs, Word files, Slack messages, wikis).

Requirements:

•Answer any question grounded in company knowledge
•Responses must cite source documents with page numbers
•Queries answered within 3 seconds end-to-end
•System must handle 10,000 queries/day
•Data must stay secure (no data sent to public APIs without encryption)

What you'll be assessed on

The interviewer will dig into chunking strategy, embedding model choice, hybrid search architecture, re-ranking, and how you evaluate quality at scale.

Scale & Constraints

▸1M documents (avg 5K words each) ≈ 50B tokens total
▸10K queries/day at peak load
▸P99 query latency ≤ 3 seconds (retrieve + generate)
▸New documents indexed within 5 minutes of upload
▸Answer faithfulness ≥ 95% on held-out eval set (RAGAS)
▸Security: enterprise data must be encrypted in transit and at rest

Must Cover0/11

Hints (if stuck)

💡 Contextual Retrieval (Anthropic): prepend chunk-specific context before embedding for 35-49% better retrieval.

💡 Hybrid search consistently beats dense-only by 5-15% recall@10.

💡 Re-rankers are 10-100× more accurate but too slow for retrieval — only re-rank top-50.

💡 Cache query embeddings and results: repeated queries are common in enterprise (30-40%).

💡 Latency budget: retrieval ≤ 500ms, re-rank ≤ 200ms, generation ≤ 2s.

0 words · auto-saved

Back to sheet

Mock interviewTake quizFlashcards

Problem

Design a production RAG (Retrieval-Augmented Generation) system for an enterprise that needs to answer questions over 1 million internal documents (PDFs, Word files, Slack messages, wikis).

Requirements:

•Answer any question grounded in company knowledge

•Responses must cite source documents with page numbers

•Queries answered within 3 seconds end-to-end

•System must handle 10,000 queries/day

•Data must stay secure (no data sent to public APIs without encryption)

What you'll be assessed on