Start typing to search.

Building Pipelines

Authoring dashboards

View Markdown

Design job-bound dashboard pages in YAML, choose widgets, scope data at the server, and publish controlled public views.

Authoring dashboards

Envoy Dashboards are YAML-defined, job-bound pages. Author one to combine run status, curated data, operator guidance, configuration forms, and controlled actions around a single Job.

A dashboard template belongs to a job template. When a Job is created, every active dashboard template is snapshotted onto the Job as a dashboard instance, with ${VAR} placeholders substituted from the Job's arguments. Editing the template affects future Jobs, not existing instances.

Dashboard YAML structure

The top-level key may be dashboard: or a bare object. The parser (parseDashboardYaml in src/dashboard-parser.ts) accepts these top-level fields:

Key Description
title / description Display metadata rendered in the page header.
navbar_ctas Optional public-header actions: { label, href, variant?, icon?, text? }.
refresh_interval_seconds Dashboard-level polling interval.
scope / default_scope Named read-only filter profiles injected into widget calls.
shared_queries Named read-only connector calls resolved once per refresh.
allow_public_input Allows input widgets to be edited via the public share link (also gated by a server switch).
pages Multi-page layout: each page is { id, title, widgets[], public? }.
widgets Legacy single-page list, normalized into one implicit page.

Widget ids must be unique across the whole document, and a document cannot declare both pages and top-level widgets. See Dashboard YAML for the full schema and Widgets overview for every widget type — this page does not repeat per-widget fields.

A worked example

A two-page dashboard attached to a migration Job Template: a stat, a bounded table, and a form that writes job_config:

dashboard:
  title: Ticket migration
  description: Monitor progress and review exceptions.
  refresh_interval_seconds: 30
  allow_public_input: false
 
  pages:
    - id: overview
      title: Overview
      widgets:
        - id: tickets_total
          type: stat
          title: Tickets
          unit: tickets
          data:
            connector: mongo
            method: count
            args:
              collection: zendesk_tickets
 
        - id: recent_errors
          type: table
          title: Recent errors
          data:
            connector: mongo
            method: find
            args:
              collection: migration_errors
              filter: { severity: error }
              limit: 50
              sort: { created_at: -1 }
          columns:
            - { field: created_at, label: Time, format: datetime }
            - { field: message, label: Message, format: truncate }
 
    - id: settings
      title: Settings
      public: false
      widgets:
        - id: sync_options
          type: form
          title: Sync options
          config_key: sync_options
          fields:
            - { key: batch_size, label: Batch size, type: number, required: true, default: 100 }
            - { key: notify, type: boolean, default: false }

Choose the right widget

  • stat for one number; progress for a current/total ratio.
  • bar_chart, line_chart, or pie_chart for bounded grouped results.
  • table for a small fixed result set (calls are clamped to 500 rows).
  • explorer for pageable, filterable, exportable data.
  • task_progress for the latest run of one job task key — no connector call, safe for public pages.
  • steps for a process stage; people for ownership and contacts.
  • heading for section labels (variant: section) or instructional callouts (variant: callout).
  • form and field_mapping for job configuration.
  • action for a confirmed server-side operation.

Reuse expensive reads

Put a read-only call in shared_queries when several widgets need the same result. Each refresh resolves a needed shared query once; widgets reference it with shared_query and derive their values with JSONata (the result is bound as data in the expression context). Keep chart and stat inputs aggregated and bounded at the connector — do not fetch raw collections merely to count or group them in the browser.

Scope data at the server

Every data widget executes a read-only connector method. Writes are rejected at widget execution time, and SQL select calls accept only SELECT or WITH ... SELECT statements. Scope what viewers can see with author-controlled filters they cannot remove:

  • An explorer widget's data.base_query is server-enforced; viewer filters, sort, and pagination are always ANDed into it and cannot override its keys.
  • dashboard.scope profiles map connector slots to filter fields (literal or JSONata) that are merged into read-only calls; default_scope applies a profile to widgets that omit scope, and scope: null on a widget opts out.
- id: failed_tickets
  type: explorer
  title: Failed tickets
  data:
    connector: mongo
    resource: zendesk_tickets
    base_query:
      filter: { severity: error }
  default_sort: { updated_at: -1 }
  page_size: 25
  features: [filter, sort, columns, export]
  columns:
    - { field: id, label: ID, sortable: true }
    - { field: subject, label: Subject }
    - { field: updated_at, label: Updated, sortable: true }

When columns is declared it is also the allowlist for viewer filters and projections. Viewers may only sort on columns marked sortable: true; sort is always applied server-side.

Dashboard widgets refresh on an interval, so an unindexed filter becomes a repeated collection scan. On MongoDB-backed dashboards, create a compound index matching the fields every widget filters on, and mark only indexed fields sortable. For a migration dashboard that filters hot collections by job and account:

db.zendesk_tickets.createIndex(
  { truto_migration_job_id: 1, truto_integrated_account_id: 1 },
  { name: 'dashboard_job_ia' },
)

Verify with .explain('executionStats') that totalDocsExamined is close to the matched count, not the full collection size.

Add editable configuration

Define the canonical schema and defaults in the job template's job_config: block, then add dashboard form or field_mapping widgets when the same keys need a guided or customer-facing editor. Both widget types persist their value under a config_key on the Job, and tasks read it as job_config.<key> — see Job config: forms and field mapping.

Add actions last

An action widget runs entirely server-side: viewers submit form values only, and the server resolves the action from the stored YAML. kind is run_task, job_run, or connector_method. Every action opens a confirmation dialog (confirm_message) and writes an audit row; in-flight dedupe and cooldown_seconds prevent double-fires.

- id: run_validation
  type: action
  label: Run validation
  confirm_message: Run validation against the current configuration?
  action:
    kind: job_run
    tasks: [validate]
    skip_deps: false
  cooldown_seconds: 30

kind: job_run does not accept arguments_expr or args — the Job runs with its existing arguments and job config.

Public dashboards

A Dashboard can be shared through a per-dashboard share token. What an anonymous viewer gets is deliberately narrow, verified in src/server/api/public-dashboards.ts:

  • Viewing. GET /api/public/dashboards/:token returns a sanitized definition: no connector call configs, no shared_queries or scope, no action dispatch internals. Pages with public: false are omitted, and widget data routes only execute widgets on public pages. Viewing is on by default and can be disabled server-wide with ENVOY_PUBLIC_DASHBOARDS_ENABLED=false (503).
  • Input widgets. The public config, options, and save endpoints are gated twice: the server kill switch ENVOY_PUBLIC_DASHBOARD_INPUT_ENABLED defaults off (unset, empty, 0, and false all return 503), and the dashboard's parsed allow_public_input must be true (otherwise 403). Public saves are validated like operator saves and recorded with updated_by: 'public'.
  • Actions. Public execution requires the server kill switch ENVOY_PUBLIC_DASHBOARD_ACTIONS_ENABLED and allow_public: true on the widget, on a public page. Public action requests have an extra rate limit, and client-supplied row context is ignored on the public path.
  • All public routes are rate limited per token.

Test before sharing

  1. Save the dashboard template and create a test Job, so snapshot behavior is exercised.
  2. Open every page as an authenticated user and verify data scope with records inside and outside the intended boundary.
  3. Save each input widget and confirm the Job's configuration changed.
  4. Trigger each action and inspect its run and audit status.
  5. If sharing, open the public URL in a private browser session and confirm only the intended pages and widgets appear.