Your preparation
0 of 0 safeguards readyIncident worksheet
Make the next decision with evidence
Identify missed executions and their business consequences, restore scheduling, and backfill only work that remains safe and useful.
Evidence→Decision→Action→Proof
Capture before evidence disappears
- Capture schedule definition, time zone, last start and success, next run, lock, leader, trigger events, job version, and scheduler health.
- List every missed interval, expected input, expiry, dependent job, customer effect, financial event, and external side effect.
- Preserve deploy, permission, clock, quota, provider, credential, queue, and configuration changes around the first missed run.
Decisions that change the response
| Question | Act when | Action |
|---|---|---|
| Backfill every run? | Each interval remains valid, idempotent, ordered, and within downstream limits. | Replay with explicit interval IDs and rate limits; otherwise consolidate or skip expired work. |
| Run manually? | The exact code, parameters, identity, and duplicate protection are known. | Record the manual execution and disable the overlapping scheduled run. |
Proof that recovery worked
- The scheduler fires on time through a real execution, not only a displayed next-run value.
- Every missed interval is completed, consolidated, skipped, or manually resolved with evidence.
- Dependent jobs and external systems reconcile without duplicate notifications, charges, or exports.
Controls to put in place
- Alert on expected completion and business output, not only scheduler process health.
- Use idempotent interval keys, visible run history, leases, and bounded catch-up.
- Document time zone, owner, safe manual command, dependencies, and backfill policy.
Tabletop drill
Disable a staging schedule for three intervals. Detect the missing output, choose the safe backfill set, run it with interval IDs, and verify dependent work.
Escalate when
Involve finance, privacy, or customer operations when missed work affects payouts, tax, retention, deletion, invoices, or promised communications.
What this means
The absence of work may produce no error. The business often discovers it only after backups are stale, certificates expire, invoices fail, or data diverges.
Warning signs
- A heartbeat or expected artifact is late.
- The scheduler reports success but the intended result is absent.
- Credentials, runtime, time zone, or deployment environment changed.
- Backlog or stale-data age rises on a daily boundary.
Recover now
First 15 minutes
- Identify the last successful result, not only the last invocation.
- Preserve scheduler and job logs plus recent configuration changes.
- Assess whether missed runs can be replayed safely.
- Prevent overlapping manual and automatic execution.
Today
- Fix schedule, credentials, runtime, ownership, or dependency.
- Replay missed periods with idempotency and bounded batches.
- Reconcile outputs against the system of record.
- Add a result-based heartbeat and stale-output alert.
Verify recovery
- A new run produces the intended result.
- Missed periods are completed or explicitly waived.
- Duplicate side effects did not occur.
- Failure and absence alerts reach an owner.
Prepare now
Access
- Scheduler credentials have an owner and expiry alert.
Backups and evidence
- Each run records start, finish, result count, and failure.
Contacts and ownership
- Critical jobs have expected frequency and maximum lateness.
Practice
- A job has been disabled, detected, and replayed in a safe test.
Common mistakes
- Monitoring process exit but not business output.
- Running every missed period simultaneously.
- Ignoring time-zone and daylight-saving behavior.
Sources
Last reviewed July 19, 2026Guidance changes. Confirm provider-specific actions in the linked official sources.