Start typing to search.

Building Pipelines

Job config: forms and field mapping

View Markdown

Model runtime-editable forms and field mappings with job_config, read values from task JSONata, and rely on per-run snapshots.

Job config: forms and field mapping

job_config is a runtime-editable configuration schema declared at the top level of a job template. Use it for settings an operator may change after a Job is created: migration stages, dry-run switches, notifications, batching choices, and field mappings.

It is distinct from template arguments: arguments are create-time values baked into the Job snapshot via ${VAR} substitution and fixed once the Job exists; job_config stays editable, so each Job can diverge from its template.

Lifecycle

  1. A job template declares the job_config schema and defaults.
  2. Creating a Job copies that schema into the Job snapshot, and defaults seed the Job's initial values.
  3. Operators edit values on the Job Details page or through dashboard input widgets.
  4. Starting a Job Run snapshots the current values onto runs.job_config, so every run has an auditable copy of the configuration it used.
  5. Tasks read the snapshot in JSONata as job_config.<key>.

Editing a Job's configuration after a run starts does not change that run.

Schema rules

Each top-level key under job_config: becomes both the stored key and the JSONata key. Keys must match ^[A-Za-z0-9_-]+$. Only two entry types are allowed — form and field_mapping — reusing the dashboard input-widget shapes from src/dashboard-parser.ts. Do not put id or config_key inside an entry; the map key supplies both.

Define a form

A form entry stores one object keyed by field key:

job_config:
  sync_options:
    type: form
    title: Sync options
    description: Settings applied to the next run
    fields:
      - key: dry_run
        label: Preview only
        type: boolean
        default: true
      - key: batch_size
        label: Batch size
        type: number
        required: true
        default: 100
      - key: stage
        label: Current step
        type: select
        required: true
        default: connect_accounts
        options:
          - { label: Connect accounts, value: connect_accounts }
          - { label: Migrate tickets, value: migrate_tickets }

Stored value for this entry:

{
  "dry_run": true,
  "batch_size": 100,
  "stage": "connect_accounts"
}

Each field supports these keys (FormField in src/dashboard-parser.ts):

Key Type Description
key string Required. The key the value is stored under.
label string Display label.
type string text, number, boolean, select, or multi_select.
required boolean The field must be present when saving.
default any Seeds the Job's initial value.
format string email or url, validated on save.
options array Inline choices as { label, value } for select / multi_select.
options_source object Connector-backed choices (an OptionSource — must call a read-only method).
depends_on string[] Sibling field keys; options refetch and the value resets when a dependency changes.
hide_if_expr string JSONata; hides the field when it evaluates true.

A form entry may also declare validation_expr (JSONata run against the value before persisting; a non-empty [{ field, message }] result rejects the save) and transform_expr (reshapes the value just before it is written).

Define a field mapping

A field_mapping entry stores an array of { source_field, target_field, transform? } rows. Use it when an operator maps source fields to destination fields:

job_config:
  ticket_field_map:
    type: field_mapping
    title: Ticket field mapping
    allow_transform: true
    default:
      - { source_field: subject, target_field: title }
    source:
      options_source:
        connector: zendesk
        method: proxy_list
        args: { resource: ticket_fields }
      label_expr: $.title
      value_expr: $.key
      search: { on_frontend: true }
    target:
      options_source:
        connector: gorgias
        method: proxy_list
        args: { resource: ticket-fields }
      label_expr: $.label
      value_expr: $.name
      search: { on_frontend: true }

Entry keys (DashboardFieldMappingWidget in src/dashboard-parser.ts):

Key Type Description
source / target object Required. Each an OptionSource supplying the pickable fields for that side.
default array Seeds initial mapping rows; must be an array.
allow_transform boolean Enables the per-row JSONata transform editor (operator-only).
validation_expr string JSONata run against the value before persisting.
cardinality string one_to_one (default, unique targets) or many_to_one.
extra_columns array Typed per-row fields: { key, label?, type, options?, required? } with type text, select, or boolean.
status_expr string JSONata per row → { label, tone } badge.
row_action string Action widget id rendered per row on a dashboard.

Connector-backed option sources must use read-only methods; writes are rejected. Saves are validated structurally: source_field and target_field must be non-empty strings, target_field values must be unique (unless cardinality: many_to_one), and the value is capped at 1000 rows and 256 KB of JSON.

Where operators edit values

  • The Job Details page renders the schema-backed configuration editor for every key declared in the job template snapshot — see Jobs and job templates.
  • Dashboard form and field_mapping widgets can read and write the same keys for guided or customer-facing flows — see Authoring dashboards.

For automation, read GET /api/jobs/{id}/config (returns { schema, config }) before writing, then save one key with PUT /api/jobs/{id}/config/{key} and body { value }. Values are validated against the Job's schema on save.

Read job_config from a task

Every step expression can reference job_config.<key>:

- name: write
  type: call_method
  run_if: "job_config.sync_options.dry_run = false"
  config: |
    {
      "connector": "destination",
      "method": "upsert",
      "args": {
        "table": "tickets",
        "data": input
      }
    }

A form value is an object, so nested reads like job_config.sync_options.batch_size work directly. A field_mapping value is an array of rows; the map_fields step expects an object keyed by source field, so convert the rows in the step config, with a static fallback so a freshly created Job is runnable before anyone edits the mapping:

- name: map
  type: map_fields
  config: |
    {
      "mapping": $exists(job_config.ticket_field_map)
        ? job_config.ticket_field_map{
            source_field: {
              "field": target_field,
              "transform": transform
            }
          }
        : {
            "subject": { "field": "title" }
          }
    }

Avoid these mistakes

  • Do not store credentials or private tokens in job_config.
  • Do not use job_config for an identity that must be fixed at Job creation — use template arguments.
  • Do not assume a later edit changes historical runs; each run keeps its own snapshot.
  • Do not use connector write methods to populate option lists — only read-only methods are accepted.