Dremio Diagnostic Collector (DDC) is a command-line tool that gathers logs, configuration, and runtime diagnostics from a Dremio cluster and packages them into a single archive you can hand to Dremio Support or analyze yourself.
DDC connects to each Dremio node over Kubernetes, SSH, or locally, and streams files directly to your machine — nothing is staged on the remote nodes. It offers two collection modes: standard, a safe and passive collection of logs and configuration for routine reviews, and diagnosis, a deep, active JVM investigation for incident response. The result is a timestamped .tar.gz containing server logs, queries, configuration, system tables, and OS/JVM info — plus, in diagnosis mode, JVM diagnostics such as JFR recordings and thread dumps.
Download the latest release binary:
- Unzip the binary
- Open a terminal and change to the directory where you unzipped your binary
- Run the command
./ddc help. If you see the DDC command help, you are good to go.
The easiest way to use DDC is the interactive TUI. Just run the binary with no subcommand:
ddcThe TUI walks you through every decision step by step, so you don't have to memorize flags:
- Transport — Kubernetes, SSH, Local, or Local-K8s
- Collection mode —
standard(routine) ordiagnosis(full incident investigation) - Paths — log / config / RocksDB directories, pre-filled with autodetected values
- What to collect — log types, JVM diagnostic tools, and (for diagnosis) API-based collections
- Node selection — pick which coordinators/executors to collect from (Kubernetes & SSH)
- API access (diagnosis only) — Dremio endpoint and PAT token, when API-based collection (KV store report / problematic job profiles) is enabled
Sensible defaults are filled in at each step, and most fields explain themselves inline, so you can usually accept the suggestions and move on.
Diagnosis mode is gated behind a support password. In the TUI, selecting
diagnosisprompts for a password (support) before continuing;standardmode is available to everyone. The non-interactivediagnosissubcommands are likewise labelled "Support only".
The rest of this README documents the non-interactive ddc collect … form — the same commands the TUI generates — for when you already know what you want or need to script DDC.
| Mode | Purpose | Default behavior |
|---|---|---|
standard |
Routine collection / performance review | Lightweight: server logs (1 day), queries.json (30 days), tracker.json, vacuum logs, config, OS info. Sequential. |
diagnosis |
Active incident investigation [SUPPORT ONLY] | All log types (3 days), GC logs, hs_err crash dumps, system tables, and (with a PAT) problematic job profiles. JVM tools (JFR, jstack, top, async-profiler) are opt-in via --diag-* flags — off by default. Parallel collection. |
standardis completely passive and safe — it only reads and copies existing log and config files. It does not touch the running Dremio process and will not affect a live cluster. Use it for routine collection and performance reviews.diagnosisinteracts directly with the live Dremio JVM (JFR, jstack, thread dumps, async-profiler, heap dumps, etc.). These tools attach to the running instance and can add significant load, destabilize, or even crash the cluster. Rundiagnosisonly when Dremio Support specifically asks you to, ideally during a maintenance window.
All settings are configured via CLI flags. There is no ddc.yaml configuration file.
Prefer the interactive TUI (
ddcwith no subcommand) unless you are scripting — it builds the commands below for you.
DDC collects diagnostics from Dremio clusters using a streaming transport — files are transferred directly from each node to your local machine through a pipe, with no intermediate staging on the remote node.
- Kubernetes: DDC uses the Kubernetes API to stream file contents from each pod via
cat. - SSH: DDC opens an SSH session to each node and streams file contents via
catover the SSH channel. - Local: DDC collects diagnostics directly on the current host (no remote transport). Useful for standalone Dremio installations.
- Local-K8s: DDC runs from inside a Dremio coordinator pod, collecting local files plus Kubernetes cluster info via the API. Useful when you cannot reach the cluster from outside.
Remote JVM collection: JVM diagnostics (jcmd for JFR, jstack for thread dumps, top for process snapshots) are executed remotely on each Dremio node. Async-profiler is streamed as a binary to the remote node via stdin and executed in place. All results are streamed back — no binaries are left behind.
Run collections non-interactively with ddc collect <transport> <mode> [flags] — the same commands the TUI generates. These are useful for scripting, CI/CD pipelines, and runbooks.
DDC v4 uses a subcommand-based CLI:
ddc collect <transport> <mode> [flags]
Where <transport> is k8s, ssh, local, or local-k8s, and <mode> is standard or diagnosis.
DDC connects via the Kubernetes API and streams logs and files from each Dremio pod, then archives the results locally.
For Kubernetes deployments (relies on a kubeconfig at $HOME/.kube/config or $KUBECONFIG):
ddc collect k8s standard --namespace mynamespaceddc collect k8s diagnosis --namespace mynamespaceA PAT is only used for two diagnosis-mode collectors — problematic job profiles and the KV store report — and both are opt-in, so you must enable them explicitly. System tables, WLM, and queries-performance data are read from RocksDB and need no PAT. Requires Dremio admin privileges.
export DDC_PAT_TOKEN="your-token-here"
ddc collect k8s diagnosis --namespace mynamespace \
--collect-problematic-profiles --collect-kvstore-report --dremio-pat-token "$DDC_PAT_TOKEN"System tables and WLM are collected from RocksDB by default in both modes — no PAT required. Standard mode does not use a PAT at all.
Specify executors with the -e flag and coordinators with the -c flag. Specify SSH user and SSH key.
ddc collect ssh standard --coordinator 10.0.0.19 --ssh-user myuserddc collect ssh standard --coordinator 10.0.0.19 --executors 10.0.0.20,10.0.0.21,10.0.0.22 --ssh-user myuserRequires Dremio admin privileges.
ddc collect ssh diagnosis --coordinator 10.0.0.19 --executors 10.0.0.20,10.0.0.21,10.0.0.22 --sudo-user dremio --ssh-user myuser --dremio-pat-token "$DDC_PAT_TOKEN"Collect diagnostics directly on the Dremio host (no SSH or Kubernetes required):
ddc collect local standard
ddc collect local diagnosis --days 5To collect from inside a Dremio coordinator pod and also gather Kubernetes cluster info via the API:
ddc collect local-k8s standard
ddc collect local-k8s diagnosis --kubeconfig /path/to/kubeconfigIn diagnosis mode, --days and --start-date control which log files are collected across all log types.
# Collect last 5 days (counting back from now)
ddc collect k8s diagnosis --namespace mynamespace --days 5
# Collect 3 days starting from a specific date
ddc collect ssh diagnosis --coordinator 10.0.0.19 --ssh-user myuser --start-date 2026-03-20 --days 3If you are running DDC from Windows, always run in a shell from the C: drive prompt.
This is because of a limitation of kubectl (see kubernetes/kubernetes#77310).
These flags apply to ddc collect <transport> <mode> (non-interactive mode). In interactive mode (ddc with no subcommand), most settings are prompted.
SSH (ddc collect ssh ...):
| Flag | Description |
|---|---|
-c, --coordinator |
Coordinator IP(s), comma-separated |
-e, --executors |
Executor IP(s), comma-separated |
-s, --ssh-key |
SSH private key file path |
-u, --ssh-user |
SSH user for login |
-b, --sudo-user |
Sudo user for privileged commands (e.g. jcmd) |
--ssh-strict-host-keys |
Enable strict host key checking (default: false) |
Kubernetes (ddc collect k8s ...):
| Flag | Description |
|---|---|
-n, --namespace |
K8s namespace |
-x, --context |
K8s context to use |
--kubeconfig |
Path to kubeconfig file (overrides $KUBECONFIG and ~/.kube/config) |
--detect-label-selector |
K8s label selector to identify Dremio coordinator/executor pods (default: role=dremio-cluster-pod) |
-l, --container-log-label-selector |
K8s label selector to filter which pods' container logs are collected (default: empty = all namespace pods) |
-d, --enable-kubectl |
Use kubectl CLI instead of embedded K8s API client |
--collect-container-logs |
Collect K8s container logs (default: enabled for diagnosis) |
--nodes |
Collect from specific nodes only (comma-separated) |
--exclude-nodes |
Exclude specific nodes (comma-separated) |
Local (ddc collect local ...):
| Flag | Description |
|---|---|
--dremio-home |
Dremio installation directory (default: /opt/dremio) |
--local-log-dir |
Log directory on this node (autodetected if not specified) |
Local-K8s (ddc collect local-k8s ...):
| Flag | Description |
|---|---|
--dremio-home |
Dremio installation directory (default: /opt/dremio) |
--local-log-dir |
Log directory on this node (autodetected if not specified) |
--kubeconfig |
Path to kubeconfig file used when in-cluster config is unavailable |
These flags are only registered on the diagnosis subcommands, and the PAT is used only for the two REST-API collectors — the KV store report (--collect-kvstore-report) and problematic job profiles (--collect-problematic-profiles). Standard mode does not use a PAT (its system tables and WLM data come from RocksDB).
| Flag | Description |
|---|---|
--dremio-pat-token |
Dremio PAT token for API-based collection (env: DDC_PAT_TOKEN) |
--dremio-endpoint |
Dremio REST API endpoint (e.g. http://localhost:9047) |
--allow-insecure-ssl |
Allow insecure SSL connections to the Dremio REST API (default: true) |
| Flag | Description |
|---|---|
--days |
Number of days to collect (default: 3, applies to all log types) |
--start-date |
Start of date range (date-only, e.g. 2026-03-20). Defaults to now minus --days |
All --diag-* tools are opt-in (default false) in both CLI and TUI.
| Flag | Default | Description |
|---|---|---|
--diag-jfr |
false | Collect Java Flight Recorder recording |
--diag-jstack |
false | Collect jstack thread dumps |
--diag-top |
false | Collect top process snapshots |
--diag-async-profiler |
false | Collect async-profiler recording |
--diag-heap-dump |
false | Collect heap dump |
--diag-time-seconds |
60 | Duration in seconds for all diagnostic tools |
N/A means the flag is not registered for that mode. (The reflection log is collected in diagnosis mode and skipped in standard mode, but is not exposed as a CLI flag.)
| Flag | Default (standard) | Default (diagnosis) |
|---|---|---|
--collect-server-logs |
true | true |
--collect-queries-json |
true | true |
--collect-queries-perf-json |
true | true |
--collect-tracker-json |
true | true |
--collect-vacuum-log |
true | true |
--collect-meta-refresh-log |
false | true |
--collect-gc-logs |
N/A | true |
--collect-acceleration-log |
N/A | true |
--collect-access-log |
N/A | true |
--collect-hive-deprecated-log |
N/A | true |
--collect-hs-err-files |
N/A | true |
| Flag | Default | Description |
|---|---|---|
--server-logs-num-days |
1 | Days of server logs to collect |
--tracker-json-num-days |
1 | Days of tracker.json to collect |
--vacuum-log-num-days |
1 | Days of vacuum.json to collect |
--queries-json-num-days |
30 | Days of queries.json to collect |
--queries-perf-num-days |
30 | Days of queries-performance data to collect |
WLM, system tables, queries-performance data, and cluster stats are read from the coordinator's RocksDB store and need no PAT. Only the KV store report and problematic job profiles use the REST API and require a PAT (diagnosis mode only). Note: WLM cluster-usage export is opt-in via --collect-wlm-cluster-usage because it scans every job profile and can be slow on large catalogs (#338).
| Flag | Default (standard) | Default (diagnosis) | Needs PAT |
|---|---|---|---|
--collect-wlm |
true | true | no |
--collect-wlm-cluster-usage |
false | false | no |
--system-tables |
default list | default list | no |
--collect-kvstore-report |
N/A | false | yes |
--collect-problematic-profiles |
N/A | false | yes |
The default --system-tables list is version,options,roles,membership,privileges,reflections,materializations,refreshes,reflection_dependencies.
| Flag | Description |
|---|---|
--coordinator-log-dir |
Coordinator log directory (autodetected if not set) |
--executor-log-dir |
Executor log directory (autodetected if not set) |
--dremio-conf-dir |
Dremio configuration directory (autodetected if not set) |
--dremio-rocksdb-dir |
Dremio RocksDB directory (autodetected if not set) |
| Flag | Description |
|---|---|
--output-file |
Name and location of the diagnostic tarball |
--collection-threads |
Concurrent node collection (0 = mode default: 5 standard, 20 diagnosis) |
--collector-timeout |
Per-collector timeout (default: 10m standard, 20m diagnosis) |
--progress=json |
Machine-readable NDJSON progress output for CI/CD |
--skip-version-check |
Skip update check at startup |
--disable-free-space-check |
Skip disk space check |
ddc connects via ssh or kubectl and collects a series of logs and files
for dremio, then puts those collected files in an archive
examples:
for interactive mode just run:
ddc
for non-interactive collection use the collect subcommand with a transport:
for kubernetes deployments:
ddc collect k8s standard --namespace mynamespace
ddc collect k8s diagnosis --namespace mynamespace --dremio-pat-token $DDC_PAT_TOKEN
for ssh based communication to VMs or Bare metal hardware:
ddc collect ssh standard --coordinator 10.0.0.19 --executors 10.0.0.20,10.0.0.21,10.0.0.22 --ssh-user myuser --ssh-key ~/.ssh/mykey --sudo-user dremio
for local collection (on this host):
ddc collect local standard
ddc collect local diagnosis
for local collection inside a Kubernetes pod:
ddc collect local-k8s standard
Usage:
ddc [flags]
ddc [command]
Available Commands:
collect Run non-interactive collection with provided flags
version Print the version number of DDC
help Help about any command
This is the main reason to prefer the TUI: before collection starts, the final screen shows the exact equivalent ddc collect … command that reflects all of your choices. Copy it once and you can:
- re-run the identical collection non-interactively — in CI/CD pipelines, cron jobs, or runbooks
- replay it across multiple clusters without clicking through the TUI again
- share it with a colleague or attach it to a support ticket so the collection is reproducible
An abbreviated example of what the TUI prints:
ddc collect k8s standard --namespace mynamespace- Read the FAQ for common questions on setting up DDC
- Read the official Dremio Support page for more details on DDC
- Read the DDC Diagnostic Tarball Contents to know what is saved by a DDC tarball
- Read the Architecture Docs for design decisions, patterns, and development history