Documentation by ClonePartner

Start typing to search.

Task Reference

filter

View Markdown

Keep or drop records in the stream based on a boolean JSONata predicate evaluated against each input record.

filter

Keeps or drops each item based on a boolean JSONata predicate — the first-class way to remove items from a stream.

When to use it

Use filter whenever some items should not continue down the pipeline. Do not use skip_if/run_if for this (they skip a step but pass the item through unchanged), and do not rely on a transform returning undefined (which also passes the item through).

Configuration

The config must be a single JSONata expression string. An object config fails validation with Filter step config must be a JSONata boolean expression string.

Key Type Required Default Description
config string Yes — Boolean JSONata predicate evaluated per item with { input, arguments, steps, variables, job_config }. Truthy keeps the item; falsy drops it.

Example

name: active-users
connectors:
  source:
    type: csv
steps:
  - name: read_users
    type: call_method
    config:
      connector: source
      method: read
      args:
        resource: users
 
  - name: only_active
    type: filter
    config: 'input.status = "active"'

Behavior notes

  • Truthy → the item passes through unchanged. The filter never modifies the item.
  • Falsy → the item is dropped — nothing is emitted downstream. This is a real drop, not a passthrough.
  • A predicate that resolves to JSONata "nothing" drops the item. The step wraps your expression as (<expression>) ? true : false, so a missing field or a null result counts as falsy instead of triggering the base passthrough behavior for null configs.