# Testing and debugging tasks

> Source: https://docs.clonepartner.com/building/testing-and-debugging/

<!-- sources: references/envoy/debugging.md, src/envoy.ts, src/logger.ts, src/task-parser.ts -->

# Testing and debugging tasks

This guide covers the workflow for getting a task from "compiles" to "verified": validate the YAML, run with debug output against a bounded input, then inspect structured logs. Debug in layers — find the first incorrect step before touching anything downstream.

## Prerequisites

- The `envoy` CLI and a `connectors.yaml` with the connectors your task needs.
- A task file under `tasks/`.

## The two output systems

Envoy separates two kinds of diagnostic output. Do not confuse them with the task's data output, which goes to stdout.

| System | Trigger | Destination | Format |
|---|---|---|---|
| Debugging | `--debug` flag or per-step `debug: true` | stderr | Human-readable step input/config/output |
| Logging | `--log-level` flag | `logs/envoy.log` | Structured Pino JSON |

## Steps

1. Validate the task before running it. Validation checks structure and compiles expressions — it does not call connectors or evaluate expressions against live records:

   ```bash
   envoy validate my-task
   ```

   A failing validation prints the issues and exits non-zero.

2. Run the task with debug output enabled:

   ```bash
   envoy my-task --debug --config connectors.yaml
   ```

   Every step prints its input, config, and output to stderr:

   ```
   [DEBUG read_users] INPUT  → undefined
   [DEBUG read_users] CONFIG → {"connector":"csv_source","method":"read",...}
   [DEBUG read_users] OUTPUT → [{"id":"1","first_name":"Alice",...},...]
   ```

   Payloads are truncated at roughly 2000 characters per field. Do not parse debug stderr as pipeline data.

3. To watch a single step instead of the whole pipeline, set `debug: true` on that step. It works without the global flag:

   ```yaml
   - name: format
     type: transform
     debug: true
     config: '{ "name": input.first_name }'
   ```

4. Raise the structured log level when you need more detail in the log file:

   ```bash
   envoy my-task --log-level debug --config connectors.yaml
   ```

   Then inspect `logs/envoy.log` — each line is one JSON object.

## Log levels

`--log-level` accepts `trace`, `debug`, `info`, `warn`, `error`, and `silent` (default `info`). What gets logged at each level:

| Event | Level | Fields |
|---|---|---|
| Task started | info | `task`, `steps` |
| Task completed | info | `task`, `items`, `durationMs` |
| Task failed | error | `task`, `err`, `durationMs` |
| Step started | info | `step` |
| Step completed | info | `step`, `items`, `durationMs` |
| Step failed (ignored) | warn | `step`, `err`, `errorDetail`, `input`, `config` |
| Truto API call | info | `method`, `resource`, `count` |
| SQLite/MongoDB/CSV/YAML call | debug | `method`, `table`/`collection`/`resource`, count |
| Condition skip | debug | `step` |

Error entries include enriched detail — `statusCode`, `body`, `response`, `code`, and `cause` from API and driver errors. `silent` disables logging entirely.

## Inspecting the pipeline with `console`

The `console` step prints items to stderr and passes them through unchanged, so you can drop it between two steps to see what flows across that boundary. It is unrelated to the task-level `output` key. See [console](/reference/steps/console/).

Earlier step output is also available in any JSONata config as `steps.<name>.output`, which helps distinguish a source problem from a transform problem: bound the source to one known record, then compare each step's output against what you expected.

## Debugging in the server and UI

Tasks run through the server keep their logs per Run. Open the Run's detail page to read them — see [Tasks and task runs](/using/tasks-and-runs/) for where Runs live in the UI and [Live run monitoring](/operations/live-run-monitoring/) for following a Run while it executes.

## Verify it worked

A task is verified when:

- `envoy validate my-task` exits 0 with no errors.
- A `--debug` run against one known record shows the expected output at every step.
- `logs/envoy.log` shows `Task completed` with the expected item count and no `Step failed` warnings you cannot explain.

:::callout{type="warning"}
Validation only compiles expressions. Evaluation-time errors — a missing field, a wrong runtime type — appear only when a real item flows through the step. Always follow a clean validation with a bounded real run before running at scale.
:::

## Related

- [console](/reference/steps/console/) — print items mid-pipeline without changing them
- [Error handling, retries, and loops](/building/error-handling/) — how ignored step failures are logged
- [Live run monitoring](/operations/live-run-monitoring/) — follow server-side Runs in real time
- [Tasks and task runs](/using/tasks-and-runs/) — where Run logs appear in the UI