Documentation by ClonePartner

Start typing to search.

Getting Started

Quickstart: build your first task

View Markdown

Author a minimal task YAML with connector slots and steps, create it in the task editor, and run it against real connectors.

Quickstart: build your first task

Goal

Write a task that reads users from a CSV file, reshapes each record with JSONata, and upserts the result into SQLite — then create it in the web UI and run it. Along the way you will meet the three ideas every Envoy task is built on: connector slots, steps, and per-record streaming.

Prerequisites

  • A running Envoy instance and an account with the admin or superadmin role
  • A csv Connector and a sqlite Connector created in the deployment (Managing connectors)
  • A users.csv file with id, first_name, and last_name columns, readable by the CSV Connector's configured directory

The task YAML

name: csv-to-sqlite
description: Read users from CSV, transform, and upsert into SQLite
 
connectors:
  csv_source:
    type: csv
  db:
    type: sqlite
 
steps:
  - name: setup_table
    type: call_method
    config:
      connector: db
      method: execute
      args:
        statements: |
          CREATE TABLE IF NOT EXISTS users (
            id TEXT PRIMARY KEY,
            full_name TEXT NOT NULL
          )
 
  - name: read_users
    type: call_method
    config:
      connector: csv_source
      method: read
      args:
        resource: users
 
  - name: transform
    type: transform
    config: |
      {
        "id": input.id,
        "full_name": input.first_name & " " & input.last_name
      }
 
  - name: upsert_user
    type: call_method
    config: |
      {
        "connector": "db",
        "method": "upsert",
        "args": {
          "table": "users",
          "data": input,
          "conflict_columns": ["id"]
        }
      }

Reading it top to bottom:

  • connectors declares two typed slots, csv_source and db. A slot is a name and a type — no paths, no credentials. Which actual CSV directory and SQLite file get used is decided when the task runs, not when it is written. See Connectors and slots.
  • steps is the pipeline. Records stream through it one at a time: read_users emits one record per CSV row, and each record flows through transform and upsert_user individually.
  • The transform step's config is a JSONata expression evaluated per record with {input, arguments, steps, variables} as context — input is the current record. Here it builds a new object with full_name concatenated from two source fields.
  • The upsert_user step shows the second config style: when a step's config is given as a string, the whole config is a JSONata expression, which lets you inject the current record ("data": input) into the method arguments. The static YAML mapping style (setup_table, read_users) is for configs that don't depend on the record.

Create it in the web UI

  1. Go to Tasks in the sidebar and select New Task.
  2. Fill in the Name field (for example csv-to-sqlite) and optionally a Description.
  3. Paste the YAML into the editor and select Create Task.

The server validates the YAML on save. If something is wrong — an unknown step type, a step referencing an undeclared connector slot — the editor shows a "Validation failed" banner listing each issue, and the "More actions" menu offers Fix with AI to hand the issues to the built-in assistant.

Run it

On the task's detail page, select Run Now. Because the task declares connector slots, the Run Configuration dialog asks you to pick a stored Connector for each slot — choose your CSV Connector for csv_source and your SQLite Connector for db — and starts the run.

The run page streams the task's output and logs live.

Or use the CLI

The same YAML runs unchanged from the command line. Put connector configurations in a connectors.yaml file whose names match the slot names, then:

envoy validate csv-to-sqlite
envoy csv-to-sqlite --config connectors.yaml

See the CLI quickstart for the full walkthrough including the connectors.yaml format.

Verify it worked

Query the users table in the SQLite database — with Data Explorer against the db Connector, or any SQLite client — and confirm one row per CSV record with full_name populated.