Start typing to search.

Operations

Store run logs in VictoriaLogs

View Markdown

Configure VictoriaLogs as the run-log backend, plan tenant and retention settings, verify ingestion and queries, and diagnose failures.

Envoy can persist task-run logs in VictoriaLogs instead of local files or the application database. Use it when control-plane replicas need a shared, scalable log backend.

This backend stores run logs, not the main application log or Envoy database state.

Prerequisites

  • A reachable VictoriaLogs endpoint.
  • Network policy allowing Envoy to insert and query logs.
  • Any reverse-proxy authentication configured outside the URL.
  • Tenant account and project IDs when using multi-tenancy.
  • A retention and deletion policy agreed with operations.

Configure the backend

runs:
  log_store: victoria_logs
 
victoria_logs:
  url: https://victoria-logs.example.com
  account_id: 0
  project_id: 0
  decolorize: true
  extra_fields:
    environment: production
    service: envoy

url is required when runs.log_store is victoria_logs. Account and project IDs default to 0. Extra fields are attached to inserted records and are useful for environment-level filtering.

Do not put credentials in the URL. Configure authentication at an approved proxy or secret-aware infrastructure layer.

Data model and live tailing

Envoy writes JSON lines and uses run_id as a stream field. It assigns strictly increasing timestamps and serializes writes per run so cursor-based live tailing does not miss out-of-order inserts.

With decolorize: true, terminal color codes are removed from the message field during ingestion.

Verify the integration

  1. Start Envoy and confirm configuration validation succeeds.
  2. Run a small task that emits several log lines.
  3. Open the task run and verify stored logs.
  4. Keep the page open and verify live lines arrive in order.
  5. Reload after completion and confirm the same lines are queryable.
  6. Test a failed run and a long message.
  7. Verify environment fields in VictoriaLogs.

Retention and deletion

Envoy's run-retention process can request log deletion, but VictoriaLogs deletion must be explicitly enabled by its administrator. If deletion is disabled or forbidden, Envoy treats removal as best-effort and the backend's own retention policy remains authoritative.

Align:

  • Envoy run retention;
  • VictoriaLogs retention;
  • backup and legal-hold requirements;
  • expected deletion behavior.

Deleting an Envoy run record does not guarantee immediate physical deletion from an external log backend.

Troubleshooting

  • Insert errors: verify URL, proxy authentication, network policy, and VictoriaLogs health.
  • Query errors: verify tenant IDs and read permissions.
  • Live logs stop: reload the run to distinguish WebSocket trouble from backend query failure.
  • Duplicate environment data: confirm all replicas use the same extra-field keys and intended values.
  • Logs remain after retention: verify VictoriaLogs deletion is enabled or shorten backend retention.

Monitor VictoriaLogs capacity and query latency independently. See Live run monitoring for browser update behavior.