Documentation by ClonePartner

Start typing to search.

CLI Reference

connectors.yaml format

View Markdown

Define connector configurations for CLI runs in connectors.yaml: entry structure, slot matching, and environment-variable secrets.

connectors.yaml format

connectors.yaml supplies connector configurations to CLI task and job runs. Tasks and job templates declare connector slots with a type only; this file provides the credentials, paths, and URLs for each slot at run time. parseConfigFile in src/config-parser.ts parses it, and the CLI reads it through the --config/-c flag:

envoy csv-to-sqlite --config connectors.yaml
envoy job run jobs/kitchen-sink.yaml -c connectors.yaml

The CLI does not read Connector records stored by the control plane, and the server does not use a CLI connectors.yaml — the two configuration paths are separate.

Structure

One top-level connectors: object. Each entry name must match a connector slot name from the task (or a template connector name for job runs), and each entry has:

Key Type Required Description
type string Yes Connector type (must match the slot's declared type).
config object No Type-specific configuration. Defaults to {}. 
connectors:
  input_csv:
    type: csv
    config:
      directory: ./data/input
 
  output_db:
    type: sqlite
    config:
      path: ./data/output.db
 
  source:
    type: postgres
    config:
      host: ${PGHOST}
      user: ${PGUSER}
      password: ${PGPASSWORD}
      database: ${PGDATABASE}

Per-type config keys (for csv, sqlite, postgres, and the rest) are documented in the Connector Reference.

Slot resolution

Before a run starts, resolveConnectorConfigs matches each slot the task declares against the entries in this file:

  1. The entry is looked up by name — slot source requires an entry named source.
  2. The entry's type must equal the slot's declared type.
  3. ${VAR} references in config are resolved from environment variables.

Failures stop the run before any task step executes:

Message Cause
Config file not found: <path> The --config path does not exist.
Config file must have a connectors object The file lacks a top-level connectors: map.
Connector "<name>" must be an object / must have a type An entry is malformed or omits type.
Connector slot "<slot>" (type: <type>) is not provided in <file> The task declares a slot with no matching entry name.
Connector "<slot>" type mismatch: task expects "<a>" but got "<b>" Entry name matches but the type differs.
Environment variable not found: <VAR> A ${VAR} reference has no value in the environment.

Environment variables for secrets

Any string value in config may embed ${VAR} references; they are replaced with the environment variable's value at resolution time, and a missing variable fails the run. ${secret://...} URIs are left untouched — they belong to the server's secret-provider layer, not the CLI.

connectors:
  source:
    type: postgres
    config:
      host: ${PGHOST}
      password: ${PGPASSWORD}
      database: ${PGDATABASE}