Stop letting LLM calls blow up your server
Most LLM tooling optimizes execution. async-bulkhead-llm decides whether a request should run at all.
Browse
Journal
Search and share individual articles. Every post has a stable URL under /journal/{slug}.
all
#503#admission-control#backpressure#concurrency#express#fail-fast#fetch#llm#loadlens#nodejs#overload#overload-protection#queues
When a bounded queue is the right call
Fail-fast gets the good press. But there are real cases where letting requests wait is the better engineering decision.
Your fetch calls don't know when to stop
Outbound HTTP calls have no built-in concurrency limit. That's fine until a downstream dependency slows down and your service drowns in open connections.
p-limit doesn't tell you why it said no
Most concurrency libraries silently queue your work. A bulkhead rejects it, tells you why, and lets your system stay responsive under load.