Skip to content

Latest commit

 

History

History
322 lines (244 loc) · 12.8 KB

File metadata and controls

322 lines (244 loc) · 12.8 KB
name kibana-vega
description Create Vega and Vega-Lite visualizations with ES|QL data sources in Kibana. Use when building custom charts, dashboards, or programmatic panel layouts beyond standard Lens charts.
metadata
author version
elastic
0.1.0

Kibana Vega

Create and manage Kibana dashboards and Vega visualizations with ES|QL data sources.

Overview

Vega is a declarative visualization grammar for creating custom charts in Kibana. Combined with ES|QL queries, it enables highly customized visualizations beyond standard Kibana charts.

Important Version Requirement: This skill strictly supports ES|QL data sources and requires Serverless Kibana or version 9.4+ (SNAPSHOT). It will not work reliably on older versions or with older Lucene/KQL data source definitions.

Quick Start

Environment Configuration

Kibana connection is configured via environment variables. Run node scripts/kibana-vega.js test to verify the connection. If the test fails, suggest these setup options to the user, then stop. Do not try to explore further until a successful connection test.

Option 1: Elastic Cloud (recommended for production)

export KIBANA_CLOUD_ID="deployment-name:base64encodedcloudid"
export KIBANA_API_KEY="base64encodedapikey"

Option 2: Direct URL with API Key

export KIBANA_URL="https://your-kibana:5601"
export KIBANA_API_KEY="base64encodedapikey"

Option 3: Basic Authentication

export KIBANA_URL="https://your-kibana:5601"
export KIBANA_USERNAME="elastic"
export KIBANA_PASSWORD="changeme"

Option 4: Local Development with start-local

For local development and testing, use start-local to quickly spin up Elasticsearch and Kibana using Docker or Podman:

curl -fsSL https://elastic.co/start-local | sh

After installation completes, Elasticsearch runs at http://localhost:9200 and Kibana at http://localhost:5601. The script generates a random password for the elastic user, stored in the .env file inside the created elastic-start-local folder.

To configure the environment variables for this skill, source the .env file and export the connection settings:

source elastic-start-local/.env
export KIBANA_URL="$KB_LOCAL_URL"
export KIBANA_USERNAME="elastic"
export KIBANA_PASSWORD="$ES_LOCAL_PASSWORD"

Then run node scripts/kibana-vega.js test to verify the connection.

Optional: Skip TLS verification (development only)

export KIBANA_INSECURE="true"

Basic Workflow

# Test connection
node scripts/kibana-vega.js test

# Create visualization directly from stdin (no intermediate file needed)
echo '<json-spec>' | node scripts/kibana-vega.js visualizations create "My Chart" -

# Get visualization spec for review/modification
node scripts/kibana-vega.js visualizations get <vis-id>

# Update visualization from stdin
echo '<json-spec>' | node scripts/kibana-vega.js visualizations update <vis-id> -

# Create dashboard
node scripts/kibana-vega.js dashboards create "My Dashboard"

# Add visualization with grid position
node scripts/kibana-vega.js dashboards add-panel <dashboard-id> <vis-id> --x 0 --y 0 --w 24 --h 15

# Apply a complete layout from stdin
echo '<layout-json>' | node scripts/kibana-vega.js dashboards apply-layout <dashboard-id> -

Note: Use - as the file argument to read JSON from stdin. This enables direct spec creation without intermediate files.

Minimal Vega Spec with ES|QL

IMPORTANT: Always use proper JSON format (not HJSON with triple quotes) to avoid parse errors.

{
  "$schema": "https://vega.github.io/schema/vega-lite/v6.json",
  "title": "My Chart",
  "autosize": { "type": "fit", "contains": "padding" },

  "config": {
    "axis": { "domainColor": "#444", "tickColor": "#444" },
    "view": { "stroke": null }
  },

  "data": {
    "url": {
      "%type%": "esql",
      "query": "FROM logs-* | STATS count = COUNT() BY status | RENAME status AS category"
    }
  },

  "mark": { "type": "bar", "color": "#6092C0" },
  "encoding": {
    "x": { "field": "category", "type": "nominal" },
    "y": { "field": "count", "type": "quantitative" }
  }
}

ES|QL Data Source Options

| Property | Description | | --------------------------- | ------------------------------------------ | --------- | | %type%: "esql" | Required. Use ES | QL parser | | %context%: true | Apply dashboard filters | | %timefield%: "@timestamp" | Enable time range with ?_tstart/?_tend |

Examples

Stdin Examples

# Create visualization directly from JSON
echo '{"$schema":"https://vega.github.io/schema/vega-lite/v6.json",...}' | \
  node scripts/kibana-vega.js visualizations create "My Chart" -

# Update visualization
echo '{"$schema":...}' | node scripts/kibana-vega.js visualizations update <id> -

# Apply layout directly
echo '{"panels":[{"visualization":"<id>","x":0,"y":0,"w":24,"h":10}]}' | \
  node scripts/kibana-vega.js dashboards apply-layout <dash-id> -

Dashboard Layout Design

Grid System

Kibana dashboards use a 48-column grid:

Width Columns Use Case
Full 48 Timelines, heatmaps, wide charts
Half 24 Side-by-side comparisons
Third 16 Three-column layouts
Quarter 12 KPI metrics, small summaries

Above the Fold (Critical)

Primary information must be visible without scrolling.

Resolution Visible Height Layout Budget
1080p ~30 units 2 rows: h:10 + h:12
1440p ~40 units 3 rows: h:12 + h:12 + h:12

Height guidelines:

  • h: 10 — Compact bar charts (≤7 items), fits above fold
  • h: 12-13 — Standard charts, timelines
  • h: 15+ — Detailed views, use below fold

Layout Pattern: Operational Dashboard

┌───────────────────────┬───────────────────────┐  y:0
│  Current State A      │  Current State B      │  h:10 (compact)
├───────────────────────┴───────────────────────┤  y:10
│         Primary Timeline                      │  h:12 (main trend)
├ ─ ─ ─ ─ ─ ─ ─ FOLD ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ┤  y:22 (1080p fold)
│         Secondary Timeline                    │  h:12 (below fold OK)
├───────────────────────┬───────────────────────┤  y:34
│  Complementary 1      │  Complementary 2      │  h:10
└───────────────────────┴───────────────────────┘

Creating Layouts

Option 1: Add panels with positions

# Row 1: Two compact half-width charts (above fold)
node scripts/kibana-vega.js dashboards add-panel $DASH $VIS1 --x 0 --y 0 --w 24 --h 10
node scripts/kibana-vega.js dashboards add-panel $DASH $VIS2 --x 24 --y 0 --w 24 --h 10

# Row 2: Full-width timeline (above fold)
node scripts/kibana-vega.js dashboards add-panel $DASH $VIS3 --x 0 --y 10 --w 48 --h 12

# Row 3: Below fold content
node scripts/kibana-vega.js dashboards add-panel $DASH $VIS4 --x 0 --y 22 --w 48 --h 12

Option 2: Apply layout file

Create layout.json:

{
  "title": "My Dashboard",
  "panels": [
    { "visualization": "<vis-id-1>", "x": 0, "y": 0, "w": 24, "h": 10 },
    { "visualization": "<vis-id-2>", "x": 24, "y": 0, "w": 24, "h": 10 },
    { "visualization": "<vis-id-3>", "x": 0, "y": 10, "w": 48, "h": 12 },
    { "visualization": "<vis-id-4>", "x": 0, "y": 22, "w": 48, "h": 12 }
  ]
}

Apply it:

node scripts/kibana-vega.js dashboards apply-layout <dashboard-id> layout.json

Design Checklist

  1. Above the fold: Primary info in top ~22 height units (1080p)
  2. Compact heights: Use h:10 for bar charts with ≤7 items
  3. Prioritize: Most important info top-left
  4. Group: Related charts side-by-side for comparison
  5. Timelines: Full width (w:48), h:12 for compact
  6. Below fold: Complementary/detailed panels OK to scroll

Guidelines

  1. Use JSON, not HJSON triple-quotes''' multi-line strings cause parse errors in Kibana; use single-line queries with escaped quotes \"

  2. Rename dotted fieldsroom.name breaks Vega (interpreted as nested path); use ES|QL RENAME room.name AS room

  3. Don't set width/height — use autosize: { type: fit, contains: padding }

  4. Set labelLimit on axes — horizontal bar chart labels truncate; use axis: { "labelLimit": 150 }

  5. Sort bars by value — pre-sort in ES|QL with SORT field DESC and use sort: null in encoding (preserves data order); avoid sort: "-x" in layered specs (bar + text labels) as it causes "conflicting sort properties" warnings

  6. Time axis: no rotated labels — use axis: { "labelAngle": 0, "tickCount": 8 }, let Vega auto-format dates

  7. Descriptive titles replace axis titles — good title/subtitle makes axis titles redundant; use title: null on axes

  8. Use color sparingly — color is a precious visual attribute; use a single default color (#6092C0) for bar charts where position already encodes value; reserve color encoding for categorical distinction (e.g., multiple lines in a time series)

  9. Dark theme compatibility — always include config to avoid bright white borders:

    "config": {
      "axis": { "domainColor": "#444", "tickColor": "#444" },
      "view": { "stroke": null }
    }

CLI Commands

# Dashboards
node scripts/kibana-vega.js dashboards list [search]
node scripts/kibana-vega.js dashboards get <id>
node scripts/kibana-vega.js dashboards create <title>
node scripts/kibana-vega.js dashboards delete <id>
node scripts/kibana-vega.js dashboards add-panel <dash-id> <vis-id> [--x N] [--y N] [--w N] [--h N]
node scripts/kibana-vega.js dashboards apply-layout <dash-id> <file|->

# Visualizations (use - for stdin instead of file)
node scripts/kibana-vega.js visualizations list [vega]
node scripts/kibana-vega.js visualizations get <id>
node scripts/kibana-vega.js visualizations create <title> <file|->
node scripts/kibana-vega.js visualizations update <id> <file|->
node scripts/kibana-vega.js visualizations delete <id>

Full Documentation

Common Issues

Error Solution
"End of input while parsing an object" Don't use HJSON ''' triple-quotes; use JSON with single-line queries
Labels show "undefined" Rename dotted fields: RENAME room.name AS room
Bars invisible / not rendering Remove complex scale.domain, use simpler color schemes
Y-axis labels truncated Add axis: { "labelLimit": 150 } to encoding
Panels stacked vertically Use --x --y --w --h options or apply-layout command
"width/height ignored" Remove dimensions, use autosize
Bright white borders on dark theme Add config: { "view": { "stroke": null }, "axis": { "domainColor": "#444", "tickColor": "#444" } }
"401 Unauthorized" Check KIBANA_USERNAME/PASSWORD
"conflicting sort properties" Don't use sort: "-x" in layered specs; pre-sort in ES|QL and use sort: null
"404 Not Found" Verify dashboard/visualization ID