Your preparation
0 of 0 safeguards readyMake the next decision with evidence
Recover connection capacity without creating a reconnect storm, then identify the callers, transactions, or pool settings that consumed it.
Capture before evidence disappears
- Capture active and waiting sessions, application name, user, client, query, transaction age, lock state, pool metrics, and server connection limits.
- Graph deploys, traffic, latency, retries, worker count, autoscaling, failovers, and database CPU or memory around the first exhaustion.
- Preserve representative slow queries and blocked transaction chains before terminating sessions.
Decisions that change the response
| Question | Act when | Action |
|---|---|---|
| Kill sessions? | Idle-in-transaction, abandoned, or noncritical sessions block recovery. | Terminate the smallest verified set and throttle reconnecting callers. |
| Raise the limit? | Database memory and process capacity support it and the pool demand is legitimate. | Apply a bounded increase; do not mask a leak or unbounded autoscaling. |
Proof that recovery worked
- Pool wait, active connections, transaction age, and query latency remain stable under peak traffic.
- Application restarts do not produce a reconnect storm or exceed the budget.
- Connection budgets across services, jobs, administrators, and failover capacity reconcile with the server limit.
Controls to put in place
- Set explicit pool sizes, acquire timeouts, transaction timeouts, and connection budgets per workload.
- Alert on wait time, utilization, long transactions, and connection growth rate.
- Load-test autoscaling and database failover with realistic pool behavior.
Exhaust a staging pool with controlled sessions. Identify the caller, protect admin capacity, terminate safe sessions, throttle reconnects, and prove the repaired pool survives load.
Use database specialists when locks, replication, failover, or memory limits complicate recovery, or when terminating sessions could corrupt business operations.
What this means
A traffic spike, connection leak, slow query, long transaction, or oversized pool can consume every connection. Adding more application instances can make it worse.
Warning signs
- Errors report pool timeout or too many connections.
- Active connections remain at the configured maximum.
- Query duration, lock waits, or idle transactions increase.
- Scaling the application causes faster failure.
Recover now
First 15 minutes
- Capture connection, query, lock, and pool metrics.
- Stop the leaking release, worker, or expensive endpoint.
- Reduce application concurrency before increasing the database limit.
- Terminate only clearly abandoned sessions using a controlled list.
Today
- Fix connection lifecycle, slow queries, locks, or pool sizing.
- Add bounded queues and timeouts at callers.
- Reintroduce traffic gradually while watching saturation.
- Reconcile failed or partially completed writes.
Verify recovery
- Connection use has headroom under normal and peak load.
- No long-lived unexplained transactions remain.
- Application errors and backlog return to baseline.
- Scaling does not multiply connections beyond capacity.
Prepare now
Access
- Emergency database inspection uses a reserved administrative path.
Backups and evidence
- Pool, connection, query, transaction, and lock metrics are available.
Contacts and ownership
- Pool limits and database capacity have named owners.
Practice
- A load test has demonstrated bounded connection use.
Common mistakes
- Restarting everything and losing the evidence.
- Increasing the database maximum without memory analysis.
- Scaling application replicas during a connection storm.