Event-Driven Async
Event-driven async architectures for AI: message queues, webhook patterns, and asynchronous agent orchestration.
Last updated
After this section you can
- Draw the sync/stream/queue line with real thresholds — p95 under ~30s stays synchronous, above that goes behind a queue — and defend it with load-balancer idle timeouts (60s ALB default) and tail-latency math
- Run the provider rate-limit arithmetic: worker concurrency × per-call tokens vs your tier's RPM/ITPM/OTPM, identify the binding constraint, and state the 50%-headroom rule
- Diagnose queue health with Little's law: compute ρ = λ/μ, predict backlog growth (36,000 messages after one hour at ρ = 1.25), and explain why capacity plans target ρ ≤ 0.7
- Size a worker fleet, queue, and DLQ end to end for a concrete workload (200K tickets/month, 90s runtime): 90 workers, 480s visibility timeout, maxReceiveCount 4, DLQ alarm at depth 50
Event-Driven & Async Architectures for AI Systems
AI systems frequently involve long-running operations — LLM inference, multi-step agent workflows, retrieval-augmented generation, and tool execution chains. Event-driven and asynchronous architectures are essential for building responsive, scalable, and resilient AI applications. This section covers patterns for handling the unique concurrency challenges of AI workloads.
The Numbers Behind the Decisions
The patterns in this section are only half the answer — the other half is arithmetic. Three calculations come up in nearly every async-systems interview and every production design review: where the sync/async line actually sits, whether your worker fleet fits inside your provider’s rate limits, and whether your queue is stable. Run them before you pick an architecture, not after the first page.