Building Pipelines
Template arguments
Collect create-time job values with typed argument widgets, substitute them into the job snapshot, and keep secrets out.
Template arguments
Template arguments collect values when a Job is created. Envoy substitutes them into the job template snapshot before the Job is stored. Use them for identities and structural choices that should remain fixed for that Job: an integrated account ID, source tenant, migration job name, object type, or target workspace.
Use job config instead for values operators should edit between runs.
Declare arguments
arguments:
account_id:
type: string
description: Account used by all source tasks
required: true
widget: integrated_account
connector_slot: truto
integration: salesforce
include_archived:
type: boolean
description: Include archived records
default: false
max_records:
type: number
description: Test limit
default: 100The parser (JobArgumentSchema in src/job-parser.ts) accepts these fields per argument:
| Field | Type | Description |
|---|---|---|
type |
string | string, number, boolean, string[], or number[]. Defaults to string. |
required |
boolean | Whether the value must be provided when the Job is created. |
description |
string | Shown next to the input in the Create Job dialog. |
default |
any | Default value when none is provided. |
enum |
array | Restrict allowed values to a fixed list. |
widget |
string | UI hint for the Create Job dialog: integrated_account or mongo_collection_record. |
integration |
string | With widget: integrated_account, filter accounts by integration slug. |
connector_slot |
string | With picker widgets, the template connector slot whose mapping supplies the connector. |
The integrated_account widget renders a searchable account picker that lists accounts from the mapped connector; mongo_collection_record renders a searchable record picker over a MongoDB collection (configured with mongo_collection, mongo_value_field, mongo_label_field, and related fields — see the arguments schema). Both pickers use server-side search with cursor pagination, so operators never paste raw UUIDs.
Substitute values
Use ${NAME} in template fields that must be resolved before the Job is stored:
tasks:
extract:
task: extract-tickets
connector_mapping:
source: truto
arguments:
integrated_account_id: ${account_id}
include_archived: ${include_archived}Substitution is textual: when the Job is created, every ${NAME} placeholder in the template YAML is replaced with the string form of the provided value, and the resolved YAML is stored as the Job's snapshot. Place a placeholder as the complete scalar value. An unquoted numeric or boolean replacement is then parsed as that YAML type; quoted replacements remain strings. Avoid embedding arrays or other structured values inside longer strings, and inspect the resolved Job before running it.
Task expressions then read the bound value from arguments:
config: |
{
"connector": "source",
"method": "list",
"args": {
"integrated_account_id": arguments.integrated_account_id,
"resource": "tickets",
"unified_model": "ticketing"
}
}Keep ownership clear
There are two argument layers:
- template arguments are the Job creation form;
- task arguments are the values a particular Task accepts at runtime.
The template explicitly maps the first layer into each task node. Reusing the same name does not bind them automatically. Task-node arguments: values are delivered verbatim — writing arguments.foo: arguments.foo ships the literal string to the Task; only ${VAR} placeholders are substituted.
Updating a template
Existing Jobs keep their original substituted snapshot. Updating a template argument default or choice list affects future Jobs only. If a fixed identity must change, create a new Job from the template. Do not hide a structural identity in editable job_config.
Security and validation
- Never collect passwords, API keys, tokens, or connection strings as arguments.
- Store secrets in Connector configuration or a configured secret provider.
- Treat connector option labels as display data and IDs as stored values.
- Required arguments are validated when the Job is created; an empty or missing value rejects the request.
- Use a bounded test Job before scheduling a newly selected tenant or workspace.
Related
- Arguments schema — every argument field, including the Mongo picker options
- Job config: forms and field mapping — runtime-editable values, the contrast to arguments
- Job templates and DAGs — where arguments live in the template