🐍 Backend with Python · Chapter 9

Async Python: when it wins and when it lies

async def makes waiting cheap, not work fast: one blocking call inside it and your whole server queues behind a single request.

Study this lesson free in the app →

The basics

Async Python lets one thread juggle many tasks that are waiting: on the network, the database, a timer. While one waits, another runs.

The problem it solves: a backend spends most of its life waiting on I/O. One thread per request wastes memory and caps you at the thread count. Async lets a single thread keep thousands of requests in flight.

The mental model: one cook, many pots. The cook (the event loop) never stands watching water boil. At every await a task says "wake me when this is ready" and the cook moves to the next pot. The catch: the cook is alone. If a task chops onions for ten seconds without saying await, every pot waits.

Explained

Three downstream calls of 300 ms each. Awaited one after another: 0.90 s. Through asyncio.gather: 0.30 s. That's the win.

Now put one time.sleep(0.3) inside an async def handler and send ten requests: ~3 s, one after another. Same keyword, opposite result. That's the lie.

Async doesn't make code faster. It makes waiting overlap, and only when every wait is an await.

One cook, twelve pots

A dhaba near Lonavala at lunch rush has one cook and a twelve-burner range.

He puts dal on, sets rice to boil, starts the chai, and while all three bubble he takes the next order. He never stands staring at a pot; he checks whichever one is ready. One cook serves forty tables because cooking is mostly waiting.

Then an order comes for chutney ground by hand on the sil-batta: fifteen minutes of solid grinding. He can't walk away from it. The dal burns, the chai boils over, forty tables wait for one chutney.

Two fixes, both obvious in a kitchen: hand the grinding to a helper (a thread or process), or buy ground chutney that only needs warming (an async library that awaits instead of blocking).

Cook = event loop. Walking away from a pot = await. The sil-batta = a blocking call.

What await actually does

Calling fetch() on an async def doesn't run it. It returns a coroutine object, a paused function. It runs when you await it or wrap it in a task (asyncio.gather, asyncio.create_task).

When a running coroutine reaches await something_not_ready, it hands control back to the event loop, which runs the next ready task. When the socket has data or the timer fires, the loop resumes it where it left off.

time →     0.0s                 0.3s
pricing    [send]~~~~waiting~~~~[done]
eta        [send]~~~~waiting~~~~[done]
offers     [send]~~~~waiting~~~~[done]
loop       runs ▲ ▲ ▲  idle      ▲ ▲ ▲

Three consequences:

1. Concurrency, not parallelism. One thread, one CPU core. Two tasks never run Python code at the same instant.
2. Switches happen only at await. Code between two awaits runs uninterrupted, which is why a blocking call stalls every other task.
3. Everything in the chain must be async. requests, psycopg2 and time.sleep block. Their async counterparts: httpx.AsyncClient, asyncpg or psycopg 3's async mode, SQLAlchemy's AsyncSession, asyncio.sleep.

Watch gather overlap the waits

Measured on a laptop, several runs: ~0.90 s vs ~0.30 s. The speed-up equals how many waits you can overlap, not anything about the CPU.

Ten requests, two kinds of sleep

import asyncio, time

async def bad_handler():
    time.sleep(0.3)                     # blocking: the loop can't switch away
    return "done"

async def good_handler():
    await asyncio.sleep(0.3)            # yields: the loop serves others
    return "done"

async def serve(handler, n=10):
    t = time.perf_counter()
    await asyncio.gather(*(handler() for _ in range(n)))
    return time.perf_counter() - t

print(f"time.sleep    x10: {asyncio.run(serve(bad_handler)):.2f}s")
print(f"asyncio.sleep x10: {asyncio.run(serve(good_handler)):.2f}s")

FastAPI: async def is a promise you can break

import asyncio, time
import httpx
from fastapi import FastAPI

app = FastAPI()

@app.get("/async-blocking")
async def async_blocking():
    time.sleep(0.2)                     # a sync driver call hiding in async def
    return {"ok": True}

@app.get("/sync-def")
def sync_def():
    time.sleep(0.2)                     # FastAPI runs plain def in a threadpool
    return {"ok": True}

@app.get("/async-awaiting")
async def async_awaiting():
    await asyncio.sleep(0.2)            # a real async driver call
    return {"ok": True}

async def hit(path, n=10):
    transport = httpx.ASGITransport(app=app)
    async with httpx.AsyncClient(transport=transport, base_url="http://t") as c:
        t = time.perf_counter()
        await asyncio.gather(*(c.get(path) for _ in range(n)))
        return time.perf_counter() - t

for path in ("/async-blocking", "/sync-def", "/async-awaiting"):
    print(f"{path:<16} 10 requests: {asyncio.run(hit(path)):.2f}s")

Measured, 10 concurrent requests each:

/async-blocking  10 requests: 2.06s   (2.68s on a second run)
/sync-def        10 requests: 0.23s
/async-awaiting  10 requests: 0.20s

FastAPI runs a plain def endpoint in a threadpool (40 threads by default via AnyIO), so a blocking call there blocks one thread, not the loop. An async def endpoint runs on the loop, so a blocking call inside it blocks everyone.

The rule I use: async def only if every slow thing inside is awaited. Calling a sync SDK you can't replace? Write def, or wrap the call in await asyncio.to_thread(...). A plain def endpoint is boring, and that's fine.

When async doesn't help at all

CPU-bound work has no waiting to overlap:

import asyncio, time
from concurrent.futures import ProcessPoolExecutor

def count_primes(limit: int) -> int:
    return sum(all(n % d for d in range(2, int(n**0.5) + 1)) for n in range(2, limit))

async def as_async(limit):
    return count_primes(limit)          # no await inside: nothing to overlap

async def on_loop():
    return await asyncio.gather(*(as_async(300_000) for _ in range(4)))

async def in_processes():
    loop = asyncio.get_running_loop()
    with ProcessPoolExecutor(max_workers=4) as pool:
        return await asyncio.gather(*(loop.run_in_executor(pool, count_primes, 300_000)
                                      for _ in range(4)))

if __name__ == "__main__":
    t = time.perf_counter(); [count_primes(300_000) for _ in range(4)]
    print(f"plain loop     : {time.perf_counter() - t:.2f}s")
    t = time.perf_counter(); asyncio.run(on_loop())
    print(f"asyncio.gather : {time.perf_counter() - t:.2f}s")
    t = time.perf_counter(); asyncio.run(in_processes())
    print(f"4 processes    : {time.perf_counter() - t:.2f}s")

Measured on a 10-core laptop: plain loop ~1.22 s, asyncio.gather ~1.16 s (no gain: there's no await inside), four processes ~0.44 s. Threads wouldn't help either on standard CPython; the GIL lets one thread run Python bytecode at a time.

At 100× traffic the other lie shows up: async makes it easy to start 10,000 calls at once. Your DB pool has 15 connections (be-006), the partner API rate-limits you at 50 rps, and every unbounded gather becomes a stampede. Cap concurrency with asyncio.Semaphore, put a timeout on every await (asyncio.wait_for), and read be-014 for backpressure.

Where async genuinely wins: many slow, concurrent I/O waits per process, such as fan-out to several services, websockets, long-polling, thousands of idle keep-alive connections.

How async code lies to you

• Blocking calls in async def: requests, time.sleep, psycopg2, open().read() on a slow disk, bcrypt or scrypt. Each one freezes the loop.
• Forgetting await: you get a coroutine object, not the result, and a RuntimeWarning: coroutine ... was never awaited in the logs.
• Unbounded gather over 10,000 ids: you DoS your own database. Use a Semaphore.
• No timeouts: one hung partner call holds its task forever. asyncio.wait_for(coro, 2.0).
• One failure cancels the picture: by default gather raises the first exception. Use return_exceptions=True when partial results are acceptable.
• Mixing sync DB sessions into async code: use SQLAlchemy's AsyncSession with an async driver, or stay sync.
• "Async = faster": for a CRUD app at 30 rps on Postgres, sync FastAPI with a threadpool is simpler and just as fast.

Words to know

In two minutes

• Async gives one thread many concurrent tasks; the event loop switches only at await.
• It wins when work is mostly waiting on I/O: three 300 ms calls take ~0.9 s awaited in turn, ~0.3 s with gather.
• Calling an async function returns a coroutine; nothing runs until it is awaited or scheduled as a task.
• A blocking call inside async def (time.sleep, requests, a sync driver) freezes every task: 10 × 0.3 s became ~3 s.
• CPU-bound work gains nothing from async; use processes (measured ~1.2 s → ~0.44 s with four).
• In FastAPI, plain def runs in a threadpool, so it's the safe default when anything inside is sync.
• Cap concurrency with a Semaphore and put a timeout on every await, or async makes stampedes easy.

Test yourself

There are 9 questions, 3 puzzles and 5 recall cards for this lesson in the app — the recall cards come back on later days, just before you would forget.