# validate_source

> Source: https://docs.clonepartner.com/reference/steps/validate-source/

<!-- sources: src/steps/validate-source-step.ts, src/steps/types.ts, references/envoy/step-types.md -->

# validate_source

Validates each source row against a primary-key and required-column contract, either annotating rows as they stream or producing one summary report for the whole batch.

## When to use it

Use `validate_source` at the front of a migration pipeline to catch broken exports — null primary keys, missing columns — before any record reaches the destination. To drop rows on arbitrary conditions rather than this fixed contract, use a [filter](/reference/steps/filter/) step.

## Configuration

| Key | Type | Required | Default | Description |
|---|---|---|---|---|
| `primary_key` | string | Yes | — | Column that must be present and non-empty on every row. `null`, missing, and `''` all count as a null primary key. |
| `required_columns` | string[] | No | — | Columns that must exist on each row (presence check, not a value check). |
| `reject_null_pk` | boolean | No | `true` | Row mode only: when not set to `false`, invalid rows are dropped from the stream. Set `false` to keep them, annotated. |
| `mode` | string | No | `report` | `row` for streaming per-row validation, `report` for a single batch summary. |

## Example

```yaml
name: validate_contacts_export
connectors:
  source:
    type: csv
steps:
  - name: read_rows
    type: call_method
    config:
      connector: source
      method: read
      args:
        resource: contacts
  - name: validate
    type: validate_source
    spool: true
    config:
      primary_key: contact_id
      required_columns:
        - email
        - name
```

## Behavior notes

### `row` mode

Each row is checked and annotated with a `_validation` object: `{ valid, null_pk, missing_columns }`. Invalid rows (null primary key or missing required columns) are dropped unless `reject_null_pk: false`, in which case they pass through with `valid: false` so a later step can route them. Memory use is constant — rows stream through one at a time.

### `report` mode (default)

The step consumes the whole batch and emits a single report:

```json
{
  "status": "ok",
  "messages": [],
  "data": {
    "row_count": 1200,
    "null_pk_count": 0,
    "missing_columns": [],
    "column_count": 14
  }
}
```

- `status` is `error` when any message is present; messages are `Missing required columns: <names>` and `<n> row(s) have null/empty primary key`.
- `missing_columns` reports the first row's missing set; `column_count` is the widest row seen.
- Report mode requires **array input**: set `spool: true` on this step (or an upstream step) so all rows arrive as one batch. Non-array input fails with `validate_source report mode requires array input`. Only counters are kept in memory, not the rows.

## Related

- [filter](/reference/steps/filter/) — drop rows on arbitrary conditions
- [ensure_table](/reference/steps/ensure-table/) — bootstrap staging tables after validation passes
- [Migration playbook](/building/migration-playbook/) — where source validation fits in a migration