Masking, loops, and error handling

These features affect data safety and execution lifetime. Configure explicit bounds and test failure behavior before using them on production-scale data.

PII masking

Task-level masking controls Envoy's JSONata masking functions:

masking:
  enabled: ${MASK_PII}
  seed: ${MASK_SEED}

Functions include:

• $mask(value, type?, options?);
• $maskEmail(value, options?);
• $maskPhone(value);
• $maskName(value);
• $maskText(value);
• $maskingEnabled().

When masking is disabled, these functions return the original value. When enabled, masking is deterministic for the same value and seed so test relationships can remain joinable.

Keep the seed in deployment secret management. Changing it changes generated tokens.

Deterministic masking is not cryptographic anonymization. Low-entropy values can still be guessed, and masked data remains subject to the organization's test-data policy.

Task loops

A top-level loop repeats the complete main pipeline within one task run:

loop:
  sleep_ms: 30000
  max_iterations: 100
  max_duration_ms: 3600000

sleep_ms is required and must be at least 10,000 milliseconds.

Continuation steps

loop:
  sleep_ms: 15000
  check_num_times: 3
  steps:
    - name: remaining
      type: call_method
      config:
        connector: staging
        method: count
        args:
          table: tickets
          where:
            status: pending
    - name: continue
      type: transform
      config: "steps.remaining.output > 0"

The final loop-step output is coerced to boolean. check_num_times requires consecutive false checks before exit.

Loop-step JSONata can read:

• variables.envoy_iteration;
• variables.envoy_last_iteration_at;
• variables.envoy_false_count.

until_empty

loop:
  sleep_ms: 15000
  until_empty:
    from_step: fetch_one_pending

Envoy re-executes the named main step in isolation as a continuation check. Make it a lightweight existence query such as a count or limit: 1, not the full extraction.

Use output: none or ndjson. A looping task with output: json accumulates results.

Step error handling

error_handling: fail

• fail stops the task and is the default.
• ignore logs the error and lets other items continue.
• ignore_rest skips the remaining pipeline for the affected item.

Use ignore only with an explicit reconciliation path. A failed step does not produce a valid success response.

Guarded continuation

After an ignored write:

- name: mark_complete
  type: call_method
  skip_if: "$not(steps.create.output.id)"
  config: |
    { ... }

Record a separate error state when the create output is absent. Do not write a destination mapping or success marker from an empty result.

Cancellation and limits

A loop exits on its configured cap, a false continuation condition, or task cancellation. Stopping it does not reverse completed connector calls.

Always set max_iterations or max_duration_ms for a bounded operational workflow. Use a schedule for discrete auditable runs when one long-lived run is unnecessary.

See Task structure and Debugging.