Troubleshooting
Common errors
Match common Envoy startup, permission, validation, and expression errors to their causes and concrete resolutions.
Common errors
This page covers errors you meet across the product. Look first at the run's logs, then rerun with --debug (or per-step debug: true) for step input/config/output on stderr, and check the structured server log at logs/envoy.log. Connector-specific failures are in Connector and account issues; dispatch and executor failures in Run and executor issues.
API errors share one JSON shape from the error middleware:
{ "error": "FORBIDDEN", "message": "Insufficient permissions" }with codes such as FORBIDDEN (403), CONFLICT (409), DB_QUERY_TIMEOUT (503), and INTERNAL_ERROR (500).
Server exits at startup with a config error
Symptom: The control plane exits immediately. Stderr shows one of:
Server config file not found: <path>encryption_key is required in server config. Set ENVOY_ENCRYPTION_KEY env var or provide it directly.Encryption key "default" is not valid base64. Generate one with: openssl rand -base64 32store.type is "postgres" but no postgres.connection_string is provided(or the equivalentmssql/victoria_logsmessage)
Cause: loadServerConfig validates the YAML before the server starts; any missing file, unresolvable ${VAR} reference, invalid encryption key, or backend cross-check failure aborts boot.
Resolution:
- Confirm the
--configpath (orENVOY_CONFIG_PATHin Docker) points at the persisted YAML. - Confirm every
${VAR}referenced in the YAML is set in the process environment — includingENVOY_ENCRYPTION_KEYon every restart, not only first boot. - Generate a valid key with
openssl rand -base64 32if the key message appears. - Add the backend section the cross-check names. Every rule is listed in Server configuration.
API request fails with FORBIDDEN
Symptom: A request returns 403 with { "error": "FORBIDDEN", "message": "Insufficient permissions" }.
Cause: The requireRole middleware rejected the caller's role. Mutating endpoints — connector tests, metrics configuration, imports, run deletion — require superadmin or admin; viewers get read paths only.
Resolution:
- Check the signed-in user's role under Users.
- Have a superadmin or admin perform the action, or raise the account's role if policy allows.
Create fails with CONFLICT
Symptom: Creating a resource returns 409 with { "error": "CONFLICT", "message": "A record with that <field> already exists" }.
Cause: A unique constraint (for example a task or connector name) already holds that value.
Resolution: Rename the new resource, or update the existing record instead of creating a duplicate.
Task validation fails with "Unknown connector type"
Symptom: envoy validate or the task editor reports Unknown connector type: '<type>'. Available: <list>.
Cause: The task's connectors block names a type that is not registered in this build. Customer builds compile in a selected connector set, and catalog types exist only when the deployment configures a catalog.
Resolution:
- Compare the type against the
Available:list in the message — check spelling and case. - If the type should exist, the deployment needs a build that includes it or a catalog that registers it. See Customer builds and catalog.
Task validation fails with "Invalid JSONata expression"
Symptom: Validation reports Invalid JSONata expression: <parser message> on a step's config, run_if, skip_if, or batch.
Cause: The expression does not parse — commonly an unbalanced bracket, a stray comma, or YAML indentation cutting the expression short.
Resolution:
- Read the parser message; it includes the failing position.
- Check the YAML block scalar (
config: |) captured the whole expression. - Validate again before running. See JSONata reference.
Static config contains a JSONata reference
Symptom: Validation reports Static config contains a JSONata reference ('<leaf>'). Static object configs are passed verbatim and never evaluated — convert this step's config to a JSONata string (config: |).
Cause: A step's config is a YAML object, and one of its string values references input, steps, or another JSONata root. Object configs are passed to the step as-is; only string configs are evaluated as JSONata.
Resolution: Rewrite the step's config as a JSONata string using config: |, producing the same object with the references evaluated.
Warning: $callMethod buffers a streaming method
Symptom: Validation warns $callMethod with method '<method>' returns a collection and is fully buffered into memory, defeating streaming. Use a call_method step as the pipeline source; reserve $callMethod for bounded lookups (find_one, count).
Cause: $callMethod(...) inside an expression collects the entire result set in memory before the expression continues, which defeats the streaming pipeline for collection-returning methods.
Resolution: Move the collection fetch into a call_method step so records stream with backpressure, and keep $callMethod for bounded lookups.
Trigger did not run
Symptom: A scheduled Job did not start at its expected time.
Cause: The Trigger is disabled, the cron expression or time zone is wrong, the target Job is gone, or a previous triggered run of the same Job is still in flight.
Resolution:
- Confirm the Trigger is enabled and its displayed next-run time is what you expect.
- Check for another in-flight run of the same Job — see the scheduler entry in Run and executor issues.
- Confirm scheduler and executor health before re-enabling.
Related
- Connector and account issues — test failures and credential errors
- Run and executor issues — queued, stuck, and interrupted runs
- Testing and debugging tasks —
--debug, validation, and structured logs