Thread pool starvation means the ThreadPool has work (new HTTP requests and async continuations) and no free worker, because existing workers are blocked. A 504 Gateway Timeout is the reverse proxy giving up while your API is still queued — often while CPU looks idle. The usual cause is sync-over-async: .Result, .Wait(), or .GetAwaiter().GetResult() on a Task.
Healthy await Starved by .Result
[W1] await SQL → W1 FREE [W1] .Result → W1 BLOCKED
[W2] await SQL → W2 FREE [W2] .Result → W2 BLOCKED
[W3] serves next request [W3] .Result → W3 BLOCKED
queue: empty queue: 400 Kestrel requests
CPU: low, by design CPU: still low — nobody is computing
gateway: 504
New to this → stay here. Merging a PR → wrong vs right. On-call / interview → production · diagnose · if an interviewer asks.
Read first if this is new: Task vs Thread and async/await in ASP.NET Core.
Terms used here: Kestrel = ASP.NET Core’s web server. Continuation = code after await that needs a worker when I/O finishes. Hill-climbing = the runtime adds ThreadPool workers slowly (about one or two per second) when the queue is backed up. p95 = the 95th-percentile request duration.
Wrong vs right
I would reject a PR that blocks on EF or HttpClient on the request path.
// Wrong — worker blocked until EF finishes
[HttpGet("{id:guid}")]
public ActionResult<OrderDto> Get(Guid id)
{
var order = _orders.GetByIdAsync(id).Result;
return Ok(order);
}
// Right
[HttpGet("{id:guid}")]
public async Task<ActionResult<OrderDto>> GetAsync(Guid id, CancellationToken ct)
{
var order = await _orders.GetByIdAsync(id, ct);
if (order is null) return NotFound();
return Ok(order);
}
Under one user in staging, .Result “works.” Under Monday morning traffic you run out of workers. SQL is not the bottleneck.
Sync-over-async: Starvation is not a classic deadlock
ASP.NET Core does not install a request SynchronizationContext. Classic UI deadlock (continuation needs the same blocked thread) usually does not happen. .Result is still wrong — it occupies a pool thread until I/O completes.
Comparison: classic deadlock versus thread pool starvation in ASP.NET Core.
| Symptom | Classic deadlock (UI / old ASP.NET) | Starvation (ASP.NET Core) |
|---|---|---|
| Sync context | Present; continuation needs the captured thread | None — ASP.NET Core does not install one |
| One request | Can hang forever | Usually finishes, just slowly |
| Many requests | Hang | Queue length ↑, p95 ↑, 504s, CPU idle |
| Fix | Don’t block the context thread | Don’t block pool threads on async I/O |
ConfigureAwait(false) does not forgive .Result.
How it showed up in production
The clinic portal was healthy at 7:40am. At 8:05, Angular login and the schedule grid started returning 504s. SQL was fine. CPU sat around 12%. Scaling out bought twenty minutes. The dump showed blocked workers.
A healthcare service layer was “sync because the original interface was sync.” Every EF call became GetAwaiter().GetResult(). Controllers were async Task — so a code search for .Result in *Controller.cs was clean.
App Insights showed request duration, not SQL duration. The SQL dependency was 80ms. The HTTP request was 12 seconds, then the gateway timed out.
I have seen the same shape on marketplace checkout when a pricing helper blocked on HttpClient, and on SaaS exports that mixed Wait() “just in the CSV mapper.”
The continuation deadlock-with-yourself
Even without a sync context, this is still a pool deadlock under load:
- Request A’s worker calls
.ResultonGetByIdAsync - EF completed on I/O; the continuation needs a worker
- Every worker is in step 1
- Injection is too slow; Kestrel’s queue expires; 504
Every cashier is waiting for the bagger, and the bagger is waiting for a cashier. Nobody is using CPU.
Do not wrap .Result in Task.Run
// Still two pool workers for one SQL wait — not a last resort
public OrderDto GetById(Guid id) =>
Task.Run(() => _orders.GetByIdAsync(id).Result).GetAwaiter().GetResult();
Microsoft’s ASP.NET Core best-practices page: do not use Task.Run to fake async. The request is already on a pool thread. That fight is Task.Run vs await.
If the leftover sync interface cannot move this sprint, keep the block off the request path — a dedicated hosted worker you own, not Kestrel. Do not “fix” it with more App Service instances first.
Diagnose before you scale the farm
1. Counters, not vibes
dotnet-counters monitor System.Runtime --process-id <pid>
What a starved box looks like:
[System.Runtime]
cpu-usage 12 % ← idle, not a compute fire
threadpool-thread-count 48 ← climbing slowly (hill-climbing)
threadpool-queue-length 312 ← this is the smoking gun
If CPU is 90% you have a compute problem. Starvation looks like a quiet box with an angry queue.
2. Parallel Stacks / dump
dotnet-dump collect -p <pid>
dotnet-dump analyze <dump>
> parallelstacks
> syncblk
You want stacks parked in Monitor.Wait, Task.Wait, ManualResetEventSlim, or SemaphoreSlim.Wait (the sync overload). Visual Studio Parallel Stacks on a dump is the same picture with better grouping.
.NET 9: WaitHandleWait events fire when a worker blocks on .Result, .Wait(), lock, Monitor.Enter, or SemaphoreSlim.Wait.
3. Search the call graph, not just controllers
.Result
.Wait(
GetAwaiter().GetResult()
.GetResult()
Thread.Sleep(
Hiding places: FluentValidation adapters, AutoMapper resolvers, IAuthorizationHandler, health checks, “temporary” sync facades, SOAP clients, and Task.WhenAll(...).Wait().
Analyzer: Microsoft.VisualStudio.Threading.Analyzers (VSTHRD002, VSTHRD103).
Pass the token — CancellationToken in ASP.NET Core.
Common mistakes
- “It works on my machine” with one user
- Blocking in
ConfigureServices/ startup on async I/O async voidfire-and-forget that still hits.Resultinside- Treating 504 as “the gateway is flaky”
- CPU-bound loops on the request thread (different bug: you want CPU high; that is saturation, not starvation)
What this is not
Merge checklist: async and await in ASP.NET Core. Wrapping the block in Task.Run: Task.Run vs await. Task vs OS thread: Task vs Thread. Topic map: async & threading hub.
If an interviewer asks
What thread-pool starvation is; what .Result does; why async does not create a thread; why scalability ≠ speed; how an idle-CPU slow API is diagnosed; how you find the blocking call.
Strong answer: names injection rate, queue length, and a dump of Wait/Result — not “async is broken.”
. Bring a 30-second dotnet-counters capture and one dump.