How One Thread Serves Many Users
If an asyncio web server runs on a single thread with a single event loop,
how can it serve thousands of simultaneous users without blocking? What is
the eventual bottleneck for database-heavy endpoints?
How it stays responsive:
Each request is handled by its own coroutine and gets its own database
session/connection. When a coroutine awaits on I/O (a database response,
an external API), it suspends and hands control back to the event loop,
which immediately advances other ready coroutines. Because the work is
I/O-bound, the loop spends its time starting and resuming many requests
rather than blocking on any single one. This yields high concurrency
across requests on a single thread — no parallelism required.
The bottleneck for DB-heavy endpoints:
The connection pool. Sessions borrow a connection from a fixed-size pool
(e.g. pool_size=5, max_overflow=10 → up to 15 connections). When all
connections are in use, additional requests wait for one to be returned.
This is healthy back-pressure, but it caps how many DB operations can be
in flight at once. Scaling further means a larger pool (bounded by what the
database can handle), read replicas, caching to reduce query volume, or more
server instances.