Documentation by ClonePartner

Start typing to search.

Connector Reference

Azure Blob Storage

View Markdown

Configure the azure-blob connector and use its methods to list, read, write, upload, and delete blobs in Azure Storage containers.

Azure Blob Storage

The azure-blob connector reads, writes, and manages blobs in an Azure Storage account, with optional parsing of blob content as JSON, YAML, or CSV.

When to use it

Use azure-blob when task data lives in Azure Storage containers — exported files to ingest, or output files to publish. For AWS S3, Cloudflare R2, or other S3-compatible stores, use the S3 and R2 connector instead.

Configuration

Key Type Required Default Description
connection_string string Yes — Azure Storage account connection string (stored as a secret)
container string Yes — Default blob container name

Every method accepts a container argument that overrides the default container for that call.

Methods

Method Arguments Returns Behavior
list prefix, container array Streams one object per blob (name, size, contentType, lastModified) matching the prefix
read blob (required), format, container, escape, from_record, delimiter, separator array Reads a blob and parses it: text (default), json, yaml, or csv; CSV is streamed record by record
write blob (required), data (required), format, container, content_type object Serializes data as text (default), json, yaml, or csv and uploads it to the blob
delete blob (required), container void Deletes the blob
upload blob (required), body (required), content_type, metadata, container object Uploads raw bytes; body is a string, or base64 for binary content
head blob (required), container object Returns blob metadata without downloading the content
download blob (required), encoding, container object Downloads blob content as text (default) or base64

Example

name: import_exported_orders
connectors:
  source:
    type: azure-blob
steps:
  - name: read_orders
    type: call_method
    config: |
      {
        "connector": "source",
        "method": "read",
        "args": { "blob": "exports/orders.csv", "format": "csv" }
      }

The matching connectors.yaml entry for CLI runs:

source:
  type: azure-blob
  config:
    connection_string: ${AZURE_STORAGE_CONNECTION_STRING}
    container: envoy-data

Behavior notes

  • list and CSV-format read return async iterators — downstream steps receive records with backpressure instead of a buffered array.
  • CSV parsing reads the header row as column names, skips empty lines, and coerces the literal cell values NULL and null to JSON null.
  • delimiter accepts the named values tab (or \t), comma, semicolon, and pipe as well as a literal character; separator is an alias. The CSV escape character defaults to a double quote.
  • from_record resumes CSV parsing from a specific record number, which supports restartable imports.
  • The connector supports file uploads from the connector editor (any extension).
  • The connection test validates credentials, checks that the configured container exists, lists one page of blobs, and runs a write-and-delete probe.