# Tasks, jobs, and templates

> Source: https://docs.clonepartner.com/concepts/tasks-jobs-and-templates/

<!-- sources: .cursor/rules/RULE.mdc, src/task-parser.ts, src/job-parser.ts, references/envoy/jobs.md, references/envoy/task-structure.md -->

# Tasks, jobs, and templates

## Why three layers

Envoy keeps pipeline logic strictly separate from deployment-specific configuration. The logic — what to read, how to transform it, where to write it — is written once as portable YAML. The configuration — which actual database, which credentials, which account — is supplied where the pipeline is deployed. Three layers make that split explicit.

```mermaid From portable YAML to a concrete run
flowchart LR
  T1[Task A] --> JT[Job Template]
  T2[Task B] --> JT
  JT -->|Create Job: bind Connectors + arguments| J[Job]
  C1[Connector: prod CRM] --> J
  C2[Connector: prod DB] --> J
  J -->|Run| JR[Job Run]
```

## Task

A **Task** is one streaming pipeline: an ordered list of steps that records flow through. Tasks declare **connector slots** — a slot name plus a connector `type`, never credentials — along with optional `variables`, `arguments`, and output settings. Because nothing environment-specific appears in the YAML, the same task runs on a laptop via the CLI, in a test deployment, and in production, unchanged.

Tasks live as YAML files in the `tasks/` directory or in the server's `tasks` table, where every save creates a new entry in `task_versions`.

## Job Template

A **Job Template** composes tasks into a directed acyclic graph. Each task node has a `key`, the `task` it points to, optional `depends_on` edges, and optional per-node `arguments`. The template's other keys:

- **`connectors`** declares the template-level connector slots, and each task node's `connector_mapping` wires the task's slot names to those template names. Many tasks can share one template connector.
- **`arguments`** declares create-time parameters. Their values are substituted into `${VAR}` placeholders when a Job is created and baked into the Job's snapshot — right for things fixed at creation, like an account ID or a migration job ID.
- **`job_config`** declares runtime-editable settings with defaults — field mappings, sync options, stage toggles. Operators can change these on the Job after creation, and each run snapshots the values it ran with.
- **`on_failure`** sets the failure policy for runs: `pause` (default), `abort`, or `continue`.

Templates live as YAML in the `jobs/` directory or the `job_templates` table.

:::callout{type="tip"}
Choosing between `arguments` and `job_config`: if changing the value should require creating a new Job, it is an argument; if operators should be able to adjust it between runs of the same Job, it is job config. See [Template arguments](/building/template-arguments/) and [Job config](/building/job-config/).
:::

## Job

A **Job** is an instantiated template: a persistent record that binds each template connector name to an actual stored [Connector](/concepts/connectors-and-slots/) by ID, carries the argument-substituted snapshot of the template, and holds the current `job_config` values. Jobs are created in the UI with the **Create Job** dialog (or via the API) and run any number of times. Each execution is a **Job Run**, which fans out into one task run per DAG node.

Jobs, job runs, and their per-task links live in the `jobs`, `job_runs`, and `job_run_tasks` tables; the individual task executions live in `runs`.

## The layers at a glance

| Layer | Contains | Stored in | Environment-specific? |
| --- | --- | --- | --- |
| Task | Steps, connector slots, variables | `tasks/` files or `tasks` + `task_versions` | No |
| Job Template | Task DAG, connector mapping, arguments, job config defaults | `jobs/` files or `job_templates` | No |
| Job | Connector bindings, argument values, editable job config | `jobs` table | Yes |
| Job Run / Run | Execution state, logs, metrics | `job_runs`, `job_run_tasks`, `runs` | Yes |

## Standalone task runs

A task does not need a template to run. The UI's **Run Now** on a task (and the equivalent API call) binds Connectors to the task's slots directly for that one run, and the CLI does the same with a `connectors.yaml` file. Templates earn their keep when tasks need ordering, shared connectors, arguments, or operator-editable configuration.

## Related

- [Connectors and slots](/concepts/connectors-and-slots/) — how slot binding works in each path
- [Run lifecycle](/concepts/run-lifecycle/) — what happens when a Job or task actually runs
- [Job templates and DAGs](/building/job-templates-and-dags/) — authoring templates
- [Job template reference](/reference/job-templates/) — the full template schema