Documentation by ClonePartner

Start typing to search.

Connector Reference

CSV

View Markdown

Configure the csv connector to stream records from CSV files in a local directory, with delimiter, escape, and resume options.

CSV

The csv connector streams records from CSV files under a base directory on the local filesystem.

When to use it

Use csv to ingest flat-file exports that live on disk (or on a volume mounted into the deployment). It is read-only; to write CSV output, use a storage connector's write method with format: csv (Azure Blob Storage, S3 and R2) or a task's batch output.

Configuration

Key Type Required Default Description
directory string Yes — Base directory for CSV files
escape string No " Escape character (defaults to double-quote)
delimiter string No , Field delimiter/separator. Use \t for tab-delimited files.

Methods

Method Arguments Returns Behavior
read resource (required), from_record, delimiter, separator array Streams records from CSV files under resource, one record at a time with backpressure

resource is resolved relative to directory. It can be either:

  • a subdirectory — every *.csv file inside it is read in sorted filename order, or
  • a single file path ending in .csv.

Example

name: import_contacts
connectors:
  source:
    type: csv
steps:
  - name: read_contacts
    type: call_method
    config: |
      { "connector": "source", "method": "read", "args": { "resource": "contacts" } }

The matching connectors.yaml entry for CLI runs:

source:
  type: csv
  config:
    directory: /path/to/csv-exports
    delimiter: comma

Behavior notes

  • Records are parsed with the header row as column names, empty lines skipped, and BOM handling enabled. Cell values that are exactly NULL or null become JSON null.
  • delimiter (and its alias separator) accept named values tab (or \t), comma, semicolon, and pipe, or any literal character. A per-call delimiter overrides the connector-level one.
  • from_record resumes parsing from a specific record number — useful for restarting a partially completed import.
  • If resource does not exist, read yields nothing rather than failing.
  • The connector supports file uploads from the connector editor; only .csv files are accepted, and upload paths are validated to stay inside directory.