Cheat Sheet: Coding Round

🎧Listen to this page3:24

Read the transcript

1. When you’ll see this and how to run the applied build

Host: So let’s set the scene for this one. You’re forty-five to sixty minutes into an interview, and the prompt lands: build a rate limiter, a retry path, a batcher, or a worker pool. This is the cheat sheet for exactly that moment, and it also doubles as the map for a debug-and-extend round, which we’ll touch on too. The question is, once that prompt lands, what do you actually do first?

Guest: You state the invariant before you write a single line of code. Something like ‘no more than N in flight, and a rejected request consumes no capacity’ — that sentence is what everything you write afterward gets judged against. From there it’s signature and data model, types on the public boundary only, then the smallest happy path that holds the invariant. Then, unprompted, you handle timeout, cancellation, and shutdown — that’s the step that actually separates candidates. You close with one test that races concurrent calls instead of a sequential one, and then you name out loud what you left out: bounded not unbounded, in-process not distributed, and what would force the other choice.

2. Debug-and-extend, plus the non-negotiables underneath any of it

Host: Okay, that’s the build side. What changes when they hand you broken code instead and say ‘fix this’?

Guest: The sequence inverts but stays just as strict: reproduce the failure before you read a single line, then find the invariant and the exact spot where it stops holding — not the whole file, that spot. Change the smallest thing that restores it, add the test that would’ve caught it originally, and say out loud what you deliberately left untouched. And underneath both formats sit the same non-negotiables — release the semaphore in finally so a slot frees on success, failure, or cancellation; re-raise CancelledError or your shutdown just hangs; jitter your backoff or retries synchronize into a storm; bound fan-out inside the loop when the caller controls input size; flush batches on size or deadline so a lone request never starves; and claim before you execute, atomically, or two concurrent submissions both run. Miss any one of those and it doesn’t matter how clean the rest of your code is — that’s the line between passing and failing.

3. The red flags that sink an otherwise good answer

Host: So even with all that right, what actually tanks an otherwise solid answer? What are the tells you’re watching for as an interviewer?

Guest: The classics: a blocking sleep or a synchronous network call sitting inside an async function, which freezes every coroutine, not just one. Acquiring the semaphore outside a try block so an exception leaks the slot forever, a broad except around an awaited call that quietly swallows cancellation, or firing off a full gather over caller-sized input with no bound. Add a sequential ‘concurrency test’ that proves nothing, spinning up a background task with no reference kept so it vanishes mid-flight, in-process state described as if it survives replicas, and worst of all, silence — this round scores reasoning you say out loud, not just code you type. If any of that sounds familiar, go compare your work against the finished files in the async-ai-gateway, durable-agent-task-engine, and dynamic-batching-inference labs — that’s the fastest way to see exactly where the gap is.

Generated from this page by Claude Sonnet 5 on 2026-08-23, spoken by Kokoro-82M running locally. Two synthetic voices, not a recorded conversation. Every claim is drawn from this page — where it differs from the text above, the text is correct.

Use This When

You are about to be asked to build a limiter, a retry path, a batcher, or a worker pool in 45–60 minutes. Also usable for a debug-and-extend round — the second sequence below.

Depth lives in Track: AI Systems Coding, Module 1, and asyncio.

The Sequence

Applied build

1. State the invariant first. “No more than N in flight, and a rejected request consumes no capacity.” Everything after is judged against this sentence.
2. Signature and data model. Types on the public boundary; nothing ceremonial inside.
3. Happy path, smallest thing that holds the invariant.
4. Timeout, cancellation, shutdown. Unprompted. This is the separator.
5. One test that fails if the invariant breaks — concurrent calls that would race, not a sequential call.
6. Name what you left out. Bounded not unbounded, in-process not distributed, and what would force the other choice.

Debug and extend

Reproduce before reading → find the invariant and where it stops holding → change the smallest thing → add the test that would have caught it → say what you deliberately did not change.

Quick Reference

Non-negotiables

Rule	Why
async with the semaphore, or release in finally	The slot must free on success, failure, and cancellation
Re-raise CancelledError	Swallowing it makes the task uncancellable — shutdown hangs, deadlines stop working
Jitter the backoff	random.uniform(0, backoff); fixed backoff synchronizes retries into a storm
Bound the fan-out	The concurrency bound goes inside the loop when input size is caller-controlled
Flush on size or deadline	A batcher without a deadline starves a lone request
Claim before execute, atomically	Otherwise two concurrent submissions both run

Concurrency model — async for I/O with async-native libraries · threads only for blocking libraries with no async equivalent (no CPU gain, GIL) · processes for CPU-bound work.

Numbers — 1 thread runs Python bytecode per interpreter · default executor min(32, cpu_count + 4) · Queue(maxsize=0) is unbounded · CancelledError inherits BaseException since 3.8 · TaskGroup and asyncio.timeout() need 3.11+.

Finished versions to compare against — rate_limit.py, gateway.py, resilience.py, draining.py in async-ai-gateway; store.py, worker.py in durable-agent-task-engine; batcher.py in dynamic-batching-inference.

Red Flags

• time.sleep(), requests, or a CPU loop inside async def — freezes every coroutine, not one.
• acquire() outside try, so an early exception leaks the slot forever.
• except Exception wrapped around an await, swallowing cancellation.
• asyncio.gather() over caller-sized input — unbounded fan-out.
• A “concurrency test” that calls the function sequentially and proves nothing.
• create_task() with no reference kept, letting the task be collected mid-flight.
• In-process state described as if it were correct across replicas.
• Ten minutes of silence. The round scores reasoning you make audible.