Building Pipelines
Authoring dashboards
Design job-bound dashboard pages in YAML, choose widgets, scope data at the server, and publish controlled public views.
Authoring dashboards
Envoy Dashboards are YAML-defined, job-bound pages. Author one to combine run status, curated data, operator guidance, configuration forms, and controlled actions around a single Job.
A dashboard template belongs to a job template. When a Job is created, every active dashboard template is snapshotted onto the Job as a dashboard instance, with ${VAR} placeholders substituted from the Job's arguments. Editing the template affects future Jobs, not existing instances.
Dashboard YAML structure
The top-level key may be dashboard: or a bare object. The parser (parseDashboardYaml in src/dashboard-parser.ts) accepts these top-level fields:
| Key | Description |
|---|---|
title / description |
Display metadata rendered in the page header. |
navbar_ctas |
Optional public-header actions: { label, href, variant?, icon?, text? }. |
refresh_interval_seconds |
Dashboard-level polling interval. |
scope / default_scope |
Named read-only filter profiles injected into widget calls. |
shared_queries |
Named read-only connector calls resolved once per refresh. |
allow_public_input |
Allows input widgets to be edited via the public share link (also gated by a server switch). |
pages |
Multi-page layout: each page is { id, title, widgets[], public? }. |
widgets |
Legacy single-page list, normalized into one implicit page. |
Widget ids must be unique across the whole document, and a document cannot declare both pages and top-level widgets. See Dashboard YAML for the full schema and Widgets overview for every widget type — this page does not repeat per-widget fields.
A worked example
A two-page dashboard attached to a migration Job Template: a stat, a bounded table, and a form that writes job_config:
dashboard:
title: Ticket migration
description: Monitor progress and review exceptions.
refresh_interval_seconds: 30
allow_public_input: false
pages:
- id: overview
title: Overview
widgets:
- id: tickets_total
type: stat
title: Tickets
unit: tickets
data:
connector: mongo
method: count
args:
collection: zendesk_tickets
- id: recent_errors
type: table
title: Recent errors
data:
connector: mongo
method: find
args:
collection: migration_errors
filter: { severity: error }
limit: 50
sort: { created_at: -1 }
columns:
- { field: created_at, label: Time, format: datetime }
- { field: message, label: Message, format: truncate }
- id: settings
title: Settings
public: false
widgets:
- id: sync_options
type: form
title: Sync options
config_key: sync_options
fields:
- { key: batch_size, label: Batch size, type: number, required: true, default: 100 }
- { key: notify, type: boolean, default: false }Choose the right widget
statfor one number;progressfor a current/total ratio.bar_chart,line_chart, orpie_chartfor bounded grouped results.tablefor a small fixed result set (calls are clamped to 500 rows).explorerfor pageable, filterable, exportable data.task_progressfor the latest run of one job task key — no connector call, safe for public pages.stepsfor a process stage;peoplefor ownership and contacts.headingfor section labels (variant: section) or instructional callouts (variant: callout).formandfield_mappingfor job configuration.actionfor a confirmed server-side operation.
Reuse expensive reads
Put a read-only call in shared_queries when several widgets need the same result. Each refresh resolves a needed shared query once; widgets reference it with shared_query and derive their values with JSONata (the result is bound as data in the expression context). Keep chart and stat inputs aggregated and bounded at the connector — do not fetch raw collections merely to count or group them in the browser.
Scope data at the server
Every data widget executes a read-only connector method. Writes are rejected at widget execution time, and SQL select calls accept only SELECT or WITH ... SELECT statements. Scope what viewers can see with author-controlled filters they cannot remove:
- An explorer widget's
data.base_queryis server-enforced; viewer filters, sort, and pagination are always ANDed into it and cannot override its keys. dashboard.scopeprofiles map connector slots to filter fields (literal or JSONata) that are merged into read-only calls;default_scopeapplies a profile to widgets that omitscope, andscope: nullon a widget opts out.
- id: failed_tickets
type: explorer
title: Failed tickets
data:
connector: mongo
resource: zendesk_tickets
base_query:
filter: { severity: error }
default_sort: { updated_at: -1 }
page_size: 25
features: [filter, sort, columns, export]
columns:
- { field: id, label: ID, sortable: true }
- { field: subject, label: Subject }
- { field: updated_at, label: Updated, sortable: true }When columns is declared it is also the allowlist for viewer filters and projections. Viewers may only sort on columns marked sortable: true; sort is always applied server-side.
Dashboard widgets refresh on an interval, so an unindexed filter becomes a repeated collection scan. On MongoDB-backed dashboards, create a compound index matching the fields every widget filters on, and mark only indexed fields sortable. For a migration dashboard that filters hot collections by job and account:
db.zendesk_tickets.createIndex(
{ truto_migration_job_id: 1, truto_integrated_account_id: 1 },
{ name: 'dashboard_job_ia' },
)Verify with .explain('executionStats') that totalDocsExamined is close to the matched count, not the full collection size.
Add editable configuration
Define the canonical schema and defaults in the job template's job_config: block, then add dashboard form or field_mapping widgets when the same keys need a guided or customer-facing editor. Both widget types persist their value under a config_key on the Job, and tasks read it as job_config.<key> — see Job config: forms and field mapping.
Add actions last
An action widget runs entirely server-side: viewers submit form values only, and the server resolves the action from the stored YAML. kind is run_task, job_run, or connector_method. Every action opens a confirmation dialog (confirm_message) and writes an audit row; in-flight dedupe and cooldown_seconds prevent double-fires.
- id: run_validation
type: action
label: Run validation
confirm_message: Run validation against the current configuration?
action:
kind: job_run
tasks: [validate]
skip_deps: false
cooldown_seconds: 30kind: job_run does not accept arguments_expr or args — the Job runs with its existing arguments and job config.
Public dashboards
A Dashboard can be shared through a per-dashboard share token. What an anonymous viewer gets is deliberately narrow, verified in src/server/api/public-dashboards.ts:
- Viewing.
GET /api/public/dashboards/:tokenreturns a sanitized definition: no connector call configs, noshared_queriesorscope, no action dispatch internals. Pages withpublic: falseare omitted, and widget data routes only execute widgets on public pages. Viewing is on by default and can be disabled server-wide withENVOY_PUBLIC_DASHBOARDS_ENABLED=false(503). - Input widgets. The public config, options, and save endpoints are gated twice: the server kill switch
ENVOY_PUBLIC_DASHBOARD_INPUT_ENABLEDdefaults off (unset, empty,0, andfalseall return 503), and the dashboard's parsedallow_public_inputmust betrue(otherwise 403). Public saves are validated like operator saves and recorded withupdated_by: 'public'. - Actions. Public execution requires the server kill switch
ENVOY_PUBLIC_DASHBOARD_ACTIONS_ENABLEDandallow_public: trueon the widget, on a public page. Public action requests have an extra rate limit, and client-suppliedrowcontext is ignored on the public path. - All public routes are rate limited per token.
Test before sharing
- Save the dashboard template and create a test Job, so snapshot behavior is exercised.
- Open every page as an authenticated user and verify data scope with records inside and outside the intended boundary.
- Save each input widget and confirm the Job's configuration changed.
- Trigger each action and inspect its run and audit status.
- If sharing, open the public URL in a private browser session and confirm only the intended pages and widgets appear.
Related
- Dashboard YAML — the full schema
- Widgets overview — every widget type and its fields
- Job config: forms and field mapping — the keys input widgets write
- Dashboards and sharing — the operator view, editing, and share links