Core Concepts
Run lifecycle
Follow task runs and job runs from snapshot and dispatch through statuses, failure policies, stopping, resuming, and retention.
Run lifecycle
Two kinds of run
A Run is one execution of one task — whether started standalone or as part of a job. A Job Run is one execution of a Job's DAG and contains one task run per node it executes. They have separate status vocabularies, and the distinction matters when reading the UI or the API.
Task run lifecycle
A run is created as pending, then moves to running when its subprocess starts. Before the subprocess is spawned, the server snapshots everything the run needs — the task version, decrypted connector configurations, and argument values — into one resolved YAML, so a run is unaffected by edits made to the task or connectors while it executes.
The subprocess's exit decides the terminal status:
| Status | Meaning |
|---|---|
success |
Process exited with code 0 |
failed |
Process exited with a non-zero code; the error message records the exit code |
completed |
Process was stopped by a user; the error message reads "Stopped by user" |
Throughout execution, the run's stdout/stderr stream into the run log store and its metrics are recorded; both are visible live in the UI.
Leases and the run reaper
Every active run holds a lease that is renewed while its process is alive — by a server timer for local runs, by executor heartbeats for remote ones. If the owning process dies without reporting (a crashed executor, a killed container), the lease expires and the run reaper marks the run failed within about 90 seconds, so runs cannot hang in running forever.
Job run lifecycle
A job run starts running and executes its DAG in dependency order: each task node becomes a task run when its depends_on predecessors have finished. Per-task statuses inside a job run are pending, running, success, failed, cancelled, and skipped.
When a task fails, the template's on_failure policy decides what happens:
| Policy | Effect |
|---|---|
pause (default) |
Stop dispatching new tasks; the job run becomes paused and can be resumed |
abort |
Stop dispatching; the job run becomes failed |
continue |
Keep running tasks whose dependencies succeeded; report at the end |
A job run whose tasks all finish becomes completed. The terminal job run statuses are completed, failed, and cancelled.
Stopping
Stop on a running job run cancels its in-flight task runs (they finish as cancelled), marks not-yet-started tasks skipped, and finalizes the job run as completed. Stopping a paused job run finalizes it the same way rather than leaving it resumable.
Resuming
Resume is available on paused and failed job runs. Failed and cancelled tasks are reset to pending and the DAG continues from there — tasks that already succeeded are not re-executed. The same resume semantics apply to CLI job runs via envoy job resume and a state file.
Reruns and retention
Runs are immutable history: rerunning means starting a new run of the same task or Job, never mutating an old one. Old runs are cleaned up by the retention settings in envoy-server.yaml (runs.retention), which age out logs on a schedule and can optionally delete the run records themselves — see Retention.
Related
- Job Runs — reading run pages, logs, and statuses in the UI
- Live run monitoring — websocket updates, executors, and observability
- Tasks, jobs, and templates — the definitions runs are made from