Skip to Content
Understand worker leases

Understand worker leases

The worker is the only process that claims analysis, artifact, application, and scheduled monitoring jobs. A lease prevents two workers from completing the same attempt. Once a lease expires, the old worker cannot commit that stale attempt.

Retryable failures use bounded backoff. After three consecutive monitoring failures—or an unrecoverable storage or evidence-integrity failure—the registration is degraded. Degraded, paused, or inactive monitoring is never presented as currently verified.

Check Health, restore the failed dependency, and allow the next eligible lease to be claimed. Do not refresh a page expecting it to run work.

A job is delayed or exhausted

  1. Open Health and check worker reachability without changing the job.
  2. On the gate, inspect the latest attempt, lease state, attempt count, and safe error code. Do not start a duplicate task while a lease is active.
  3. Restore the unavailable database, DataHub, inference, artifact, or sandbox dependency named by the attempt.
  4. Leave the gate page alone while the worker claims the next eligible retry. If the attempt is exhausted, use the task-specific retry shown on the gate.

Expected result: worker health is available and a new attempt becomes completed or reaches the next explicit review state. For monitoring, the gate also returns to Monitoring active with a recent Last verified time.

Last updated on