Automated Kibana Dashboard Generator for API Monitoring

This project provides a fully automated solution for generating Kibana dashboards directly from a Swagger/OpenAPI specification. The script analyzes each controller (API scope) defined in the specification and produces a dedicated dashboard containing detailed monitoring panels for every endpoint. Each panel visualizes the API’s operational metrics, including the distribution of returned HTTP status codes, enabling teams to observe performance patterns, detect anomalies, and quickly identify reliability issues.

This tool eliminates the need for manual dashboard creation, ensures consistent observability across all services, and keeps monitoring aligned with API changes. It is especially useful for microservice environments where maintaining accurate dashboards manually is time-consuming and error-prone.

Components

1. swagger_kibana_analyzer.py - Analyzes existing dashboards to find unmonitored endpoints
2. generate_dashboards.py - Generates new dashboards and visualizations for API monitoring
3. swagger_parser.py - Parses Swagger/OpenAPI specifications
4. endpoint_comparator.py - Compares and analyzes API endpoints
5. models.py - Data models for API endpoints

Features

Analysis Features

• Swagger/OpenAPI Support: Reads both Swagger 2.0 and OpenAPI 3.0 JSON files (local files or URLs)
• Kibana Integration: Connects to Kibana via REST API to retrieve dashboards and visualizations
• Smart URL Extraction: Uses regex patterns to find API URLs in dashboard queries and visualizations
• Endpoint Comparison: Intelligently compares Swagger endpoints with dashboard URLs using path normalization
• Unmonitored Endpoint Detection: Identifies API endpoints not covered by existing dashboards

Dashboard Generation Features

• Automatic Dashboard Creation: Generates Kibana dashboards grouped by API tags
• Status Code Monitoring: Creates visualizations for each HTTP status code (200, 400, 500, etc.)
• Controller-based Organization: Groups endpoints by Swagger controller into separate dashboards
• Elasticsearch Query Integration: Uses your specific log format and field structure
• Batch Processing: Processes all endpoints and creates comprehensive monitoring setup

General Features

• Flexible Authentication: Supports username/password and API key authentication for Kibana
• Detailed Reporting: Provides comprehensive JSON output with analysis results
• Modular Design: Separated components for easy customization and extension

Installation

1. Install Python dependencies:

pip install -r requirements.txt

Usage

Generating New Dashboards

# Generate dashboards for all API endpoints with api key
python generate_dashboards.py https://api.example.com/swagger.json http://localhost:5601 --api-key your-key

# With custom index pattern
python generate_dashboards.py swagger.json http://localhost:5601 --api-key your-key --index-pattern "api-logs-*"

# Save generation results
python generate_dashboards.py swagger.json http://localhost:5601 --api-key your-key --output generation_results.json

Save Results to File

python swagger_kibana_analyzer.py swagger.json http://localhost:5601 --api-key your-key --output results.json

Verbose Output

python swagger_kibana_analyzer.py swagger.json http://localhost:5601 --api-key your-key --verbose

Command Line Arguments

• swagger_file: Path to the Swagger/OpenAPI JSON file or URL (e.g., https://api.example.com/swagger.json)
• kibana_url: Kibana URL (e.g., http://localhost:5601)
• --username: Kibana username (optional if using API key)
• --password: Kibana password (optional if using API key)
• --api-key: Kibana API key (alternative to username/password)
• --output: Output file for results in JSON format (optional)
• --verbose, -v: Enable verbose logging

Output Format

The script outputs a JSON report with the following structure:

{
  "total_endpoints": 150,
  "monitored_endpoints": 120,
  "unmonitored_endpoints": 30,
  "dashboard_summary": {
    "API Monitoring Dashboard": [
      "/api/users",
      "/api/orders",
      "/api/products"
    ],
    "Error Tracking Dashboard": [
      "/api/auth/login",
      "/api/auth/logout"
    ]
  },
  "unmonitored_endpoint_details": [
    {
      "path": "/api/admin/settings",
      "method": "GET",
      "operation_id": "getAdminSettings",
      "summary": "Get admin settings",
      "tags": ["admin"]
    }
  ]
}

How It Works

1. Swagger Parsing

• Reads Swagger/OpenAPI JSON files
• Extracts all API endpoints with their methods, paths, and metadata
• Handles both Swagger 2.0 (basePath) and OpenAPI 3.0 (servers) formats

2. Kibana Analysis

• Connects to Kibana using the Saved Objects API
• Retrieves all dashboards and their associated visualizations
• Extracts queries and configurations from visualizations

3. URL Extraction

Uses multiple regex patterns to find API URLs in dashboard content:

• HTTP(S) URLs containing /api/
• Relative API paths starting with /api/
• JSON fields like "path" and "url"
• Configuration fields like endpoint

4. Intelligent Comparison

• Normalizes paths by removing query parameters and trailing slashes
• Converts dynamic path segments (IDs, UUIDs) to generic placeholders
• Performs fuzzy matching to account for different URL formats

Authentication

API Key

--api-key your-api-key

To create an API key in Kibana:

1. Go to Stack Management → API Keys
2. Click "Create API Key"
3. Set appropriate privileges
4. Copy the generated key

Troubleshooting

Connection Issues

• Verify Kibana URL is accessible
• Check authentication credentials
• Ensure Kibana API is enabled

No Dashboards Found

• Verify user has permission to read dashboards
• Check if dashboards exist in the specified Kibana instance

Missing API URLs

• The script uses regex patterns to find URLs
• Complex query structures might not be detected
• Consider customizing the url_patterns in DashboardAnalyzer

Customization

Path Normalization

Customize the normalize_path method in EndpointComparator to handle specific path patterns in your API.

Requirements

• Python 3.7+
• requests library
• Access to Kibana instance
• Valid Swagger/OpenAPI JSON file