Control plane and executors

Envoy separates orchestration from data processing. The control plane is the persistent application service; executors are workers that run task subprocesses.

Local mode

With local execution, the server starts task subprocesses on its own host:

executor:
  type: local

This topology suits development or a single trusted machine. The server and task processes share the same filesystem and network boundary.

HTTP mode

With HTTP execution, the server dispatches work to registered executors:

executor:
  type: http
  config:
    callback_url: http://control-plane:3000
    executor_key: ${ENVOY_EXECUTOR_KEY}

Start an executor with the envoy executor command. Its flags, from src/envoy.ts:

envoy executor \
  --control-plane http://control-plane:3000 \
  --port 9090 \
  --max-concurrent 10 \
  --advertise-url http://envoy-executor:9090

ENVOY_CONTROL_PLANES may provide comma-separated control-plane URLs; they are merged with --control-plane values. At least one URL is required, otherwise the command exits with Error: at least one --control-plane URL or ENVOY_CONTROL_PLANES is required. See Commands and flags for the full CLI.

Authentication

The executor process requires ENVOY_EXECUTOR_KEY in its environment and refuses to start without it. Every request in both directions — registration, heartbeats, log and completion callbacks — carries this shared key. It is separate from the application encryption key, user sessions, API keys, and connector credentials.

Use one executor key per pool of control planes and executors, and rotate it as a coordinated deployment change; mismatched values prevent registration and callbacks.

Registration, heartbeats, and leases

An executor registers with each control plane at startup (POST /api/executors/register with its URL, capacity, and labels) and then sends heartbeats every 15 seconds (POST /api/executors/heartbeat) reporting active tasks and the run IDs it is executing.

On the control plane (src/server/api/executors.ts):

• each reported run ID renews that run's lease;
• run IDs the executor reports that are already terminal in the database are returned as cancelRunIds, and the executor kills those zombie processes (fencing);
• the Executors page lists URL, health, active tasks versus capacity, labels, and last heartbeat.

A run lease expires 60 seconds after the last renewal (LEASE_TTL_MS, four missed heartbeats). The run reaper then fails abandoned running or pending work with the message Executor became unavailable — run interrupted — it never silently marks work successful. See Run and executor issues.

Callbacks

Executors push results back to the callback_url:

• POST /api/executor-callbacks/runs/:id/logs — log chunks, appended to the configured run log store;
• POST /api/executor-callbacks/runs/:id/complete — final status, exit code, and completion time.

If a callback cannot be delivered, the executor spools logs to disk (.envoy/.log-spool) and journals completions in a local SQLite database (data/executor-journal.db, override with ENVOY_EXECUTOR_JOURNAL), retrying until the control plane returns.

Executor-side log backends

The control plane tells the executor per task where to write logs. The executor supports four backends (src/executor/server.ts): http (callback to the control plane), sqlite, mssql, and victoria_logs. Direct database or VictoriaLogs writes avoid routing high-volume log traffic through the control plane; the store must then be reachable from the executor network.

Advertised URL

The executor's advertised URL must be reachable from the control plane. Do not advertise:

• localhost when services run in different containers;
• a browser-facing public URL when internal service discovery is intended;
• a hostname that resolves only on the executor itself.

In Compose, advertise the executor service or container DNS name. In Azure Container Apps, use the internal application FQDN.

Capacity

--max-concurrent limits active tasks on one executor. It is one of three separate levels:

• executor --max-concurrent bounds simultaneous task processes on that worker;
• server runs.max_concurrent bounds active runs managed by the control plane;
• per-step concurrency bounds simultaneous record operations inside one task.

Size all three against CPU, memory, destination rate limits, and database connection pools.

Labels

envoy executor \
  --control-plane http://control-plane:3000 \
  --labels='{"region":"eu","workload":"migration"}'

Labels appear on the Executors page and are available to executor selection. Do not treat labels as an authorization boundary.

Restart safety

Restarting an executor terminates its task subprocesses. Before an executor deployment:

1. check activeRunsByLease in GET /api/health or the UI;
2. wait for work to drain;
3. deploy the executor;
4. confirm it re-registers and resumes heartbeats.

The remote deploy script refuses to restart an executor while active leases exist unless WAIT_FOR_DRAIN=true is set. Control-plane-only restarts are less disruptive, but callbacks and heartbeats still require the server to return; the executor spools and retries in the meantime.

Unregistering

Admins can unregister an executor in the UI. This removes the registry entry, but a live executor re-registers on its next heartbeat. Stop or reconfigure the process if it should stay removed.

Related

• Commands and flags — the full envoy CLI including executor
• Server configuration — the executor section and runs.max_concurrent
• Run and executor issues — queued runs, lost executors, lease expiry
• Remote Docker — the hardened two-container topology