Start typing to search.

Overview

Introduction to Envoy

View Markdown

Learn how Envoy defines, runs, schedules, and monitors portable data pipelines.

Envoy is a self-hosted data movement and workflow product built around YAML definitions and streaming pipelines. It provides a web control plane, API, CLI, and executor for connections, tasks, jobs, schedules, dashboards, users, and run monitoring.

Envoy is developed and supported by ClonePartner. Customer builds and deployment assistance may come through a ClonePartner delivery channel, but the product documented throughout this site is Envoy.

Envoy can run entirely inside infrastructure you control. A deployment may use a local SQLite state database, PostgreSQL, or Microsoft SQL Server. Connector secrets are stored through a pluggable secret-provider layer, and data tasks execute in isolated subprocesses on a local or remote executor.

What you can build

Use Envoy to:

  • move records between files, databases, object storage, and SaaS APIs;
  • transform and validate data with JSONata;
  • compose several tasks into a dependency graph;
  • bind the same portable definitions to different connections in each environment;
  • run work manually, on a cron schedule, or at a future time;
  • monitor run status, logs, and metrics;
  • publish job-bound dashboards for operators or selected external viewers.

The core model

flowchart LR task["Task<br/>one streaming pipeline"] --> template["Job template<br/>DAG of tasks"] template --> job["Job<br/>template + bound connectors"] job --> run["Job run<br/>one execution"] connector["Connector<br/>stored configuration"] --> job executor["Envoy executor"] --> run
Envoy core model

Tasks declare connector slots by type, but never carry credentials. A job template maps those slots across its task graph. A job binds real connector records to the template, and each run snapshots the configuration it uses.

Read Architecture for the complete flow and Glossary for the product vocabulary.

Choose a starting point

Technical reference

This guide explains product workflows and operating decisions. Detailed YAML schemas, connector methods, step behavior, and authoring edge cases live in the technical reference:

Those reference pages are reviewed and maintained with this site and should be treated as the source of truth for exact schemas.