Design a ChatGPT-Like Product — System Design Practice | AmanAI Lab

Sheet›Design a ChatGPT-Like Product

OpenAIMSFTGoogleMeta

45:00

Hard

The Problem

Problem

Design a consumer chatbot product (like ChatGPT) from scratch that can serve 10 million daily active users.

The product should support:

•Multi-turn conversations with memory
•Streaming token output
•File uploads (PDF, images, code)
•Tool use (web search, code execution)
•Multiple model tiers (fast cheap model and a slower premium model)

What you'll be assessed on

This is a full-stack ML systems design question. The interviewer will probe your thinking on conversation state, streaming architecture, tool orchestration, abuse prevention, and cost management.

Scale & Constraints

▸10M DAU, average 5 messages per session
▸50M messages/day peak
▸99.9% availability (max ~8.7h downtime/year)
▸P95 time-to-first-token ≤ 500ms
▸Conversation history maintained per user across sessions
▸File uploads up to 50MB
▸Code execution sandbox must be isolated

Must Cover0/11

Hints (if stuck)

💡 SSE (Server-Sent Events) is simpler than WebSocket for one-directional streaming.

💡 Store conversation history in Redis (hot) + PostgreSQL (durable) with a 30-day TTL.

💡 Context window limits require a strategy: summarise old turns, use sliding window, or RAG over history.

💡 E2B or Modal provide secure sandboxed code execution with container isolation.

💡 Content moderation runs in parallel with generation — don't block on it.

0 words · auto-saved

Back to sheet

Mock interviewTake quizFlashcards

Problem

Design a consumer chatbot product (like ChatGPT) from scratch that can serve 10 million daily active users.

The product should support:

•Multi-turn conversations with memory

•Streaming token output

•File uploads (PDF, images, code)

•Tool use (web search, code execution)

•Multiple model tiers (fast cheap model and a slower premium model)

What you'll be assessed on