How would you scale from 10K to 1M users without rewriting?

"The architecture shouldn't change — the infrastructure scales.:

• (1)Stateless agents on Cloud Run/GKE — auto-scale horizontally.
• (2)Queue-based ingestion via Pub/Sub — decouples request rate from processing rate.
• (3)Provisioned throughput on Vertex AI for predictable latency under load.
• (4)Sharded vector indices — partition by tenant or region.
• (5)Regional deployment — deploy in 3 regions, route by user geography.
• (6)Cache layers become critical — semantic cache hit rate determines your cost scaling. The key insight: at 10K users you can afford to be synchronous. At 1M, everything must be async with graceful degradation."