Skip to content

Commit 43f1b29

Browse files
author
lin.zhang
committed
feat: add job command, custom headers, runs enrichment, and misc improvements
- Add 'job' command group with download, analyze, and status subcommands - Support custom HTTP headers in profiles (--header KEY=VALUE, header.* keys) - Enrich runs detail with instance fields (owner, duration, vc_code, timestamps) - Simplify runs deps/task deps to use task_id from detail and config APIs - Add User-Agent header (cz-cli/<version>) to all Studio requests - Use cz_mcp region_config for environment inference with fallback - Preserve skip_update flag across auto-update state writes - Remove deprecated sql-status command (merged into job status) - Remove published_content from runs detail (use task content instead) - setup.sh: auto-source rc file after PATH update, no manual step needed - Bump version to 0.5.4
1 parent e14c43e commit 43f1b29

37 files changed

Lines changed: 706 additions & 441 deletions

File tree

CHANGELOG.md

Lines changed: 12 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -8,12 +8,24 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
88
## [Unreleased]
99

1010
### Added
11+
- **Auto-update for standalone binaries**: PyInstaller binaries now check GitHub releases for newer versions after each command. Updates are downloaded and installed automatically via `install.sh`. Checks are throttled to once per 4 hours. Output goes through loguru debug level for normal commands (visible with `--debug`), and prints directly to stderr for `--help`. Disable with `CZ_SKIP_UPDATE=1` env var or set `"skip_update": true` in `~/.clickzetta/.update_state.json`.
12+
- **Integration tests for PyInstaller binary**: `tests/integration/runner.py` now supports running test scenarios against the standalone binary via subprocess. Set `CZ_CLI_BIN=/path/to/cz-cli` to use binary mode; unset uses in-process CliRunner as before. New Makefile target: `make test-integration-bin`.
13+
- **macOS ad-hoc codesign in CI**: GitHub Actions workflow recursively signs all `.so`/`.dylib` files and the main binary with `codesign --force --sign -` before packaging. Linux builds also ensure `chmod +x`.
14+
- **Quarantine removal in install.sh**: `xattr -r -d com.apple.quarantine` is run after unzip to prevent macOS Gatekeeper blocking downloaded binaries.
15+
- **Auto-detect system proxy in install.sh**: Installer detects macOS system proxy (via `networksetup`) and probes common local proxy ports (7890, 7897, 1087, 8080) to speed up GitHub downloads in restricted network environments.
16+
- **`CZ_MIRROR` support in install.sh**: Users can set `CZ_MIRROR` env var to use a custom download mirror for release assets.
17+
- **`setup.sh` / `setup.bat` bundled in zip**: Users who manually download the zip can run `./setup.sh` (macOS/Linux) or `setup.bat` (Windows) to install to `~/.clickzetta/bin/`, configure PATH, remove quarantine, and register skills.
18+
- **PATH written to both zshrc and bashrc**: `install.sh` and `setup.sh` now write PATH entries to both `~/.zshrc` and `~/.bashrc` (if exists) for zsh users, with dedup to avoid repeated entries.
19+
- **Timestamped install.sh logs**: Key steps (version fetch, binary download, skills download, install) now print `[HH:MM:SS]` timestamps and file sizes for download diagnostics.
1120
- Add per-command usage examples to `--help`, `ai-guide` JSON, and `SKILL.md`: all commands across `sql`, `profile`, `schema`, `table`, `workspace`, `task`, `runs`, and `executions` are annotated with structured `examples` lists (via `command.examples = [...]`) that automatically render in `--help` epilog, the ai-guide payload, and the generated `SKILL.md`.
1221
- Add integration test scenario YAML files for `sql` (`sql_basic.yaml`), `profile` (`profile_management.yaml`), `table`/`schema`/`workspace` (`table_schema_workspace.yaml`), and `runs` (`runs_management.yaml`) under `tests/integration/cases/`.
1322
- `cz_cli/skills/cz-cli/scripts/` directory added as skill binary delivery location; `build_fat_multi_platform.sh` now copies the platform binary there after each build so skills can run `cz-cli` without a separate `pip install`.
1423
- Added **Rule 0** to `SKILL.md` and `SKILL.template.md`: AI Agents must detect an unconfigured profile before any connection command and interactively guide the user through `profile create` using the `AskUserQuestion` tool — not plain-text prompts.
1524

1625
### Changed
26+
- **CI build dependencies**: `pip install -r requirement.txt` replaced with `pip install -e .` in GitHub Actions to ensure `clickzetta-mcp-server` (cz_mcp) is installed from `pyproject.toml` dependencies.
27+
- **requirement.txt synced with pyproject.toml**: Added `clickzetta-mcp-server` and `pygments>=2.17.0` to match `pyproject.toml` dependencies.
28+
- **PyInstaller hidden imports hardened**: `cz-cli.spec` now includes `collect_submodules` for `cz_mcp`, `cz_skills`, and their third-party dependencies (`mcp`, `loguru`, `httpx`, `aiohttp`, `sqlparse`, `jsonpath_ng`, `pydantic`, `yaml`, `requests`, `numpy`, `pandas`) as defensive measure.
1729
- `SKILL.template.md` top-level installation block updated: binary-first (`scripts/<platform>-<arch>/cz-cli`) with `pip3 install cz-cli -U` as fallback, removing the unconditional pip-required message.
1830
- `.gitignore` refined: `cz_cli/skills/cz-cli/scripts/*/` ignores binary payloads while `SKILL.md` and `scripts/.gitkeep` remain trackable.
1931
- Integration test execution order: integration test scenarios are now implemented before per-command examples to ensure examples are validated against a real environment.

Makefile

Lines changed: 11 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,4 @@
1-
.PHONY: help clean test test-unit test-integration test-integration-bin build build-fat install upload dev lint format generate-skills
1+
.PHONY: help clean test test-unit test-integration test-integration-bin build build-fat install upload dev lint format generate-skills tag
22

33
help:
44
@echo "cz-cli Makefile commands:"
@@ -15,6 +15,7 @@ help:
1515
@echo " make install - Install package in editable mode"
1616
@echo " make dev - Install with dev dependencies"
1717
@echo " make upload - Upload to PyPI (requires credentials)"
18+
@echo " make tag - Create and push git tag from version.py (e.g. v0.5.2)"
1819
@echo " make all - Run clean, test, build"
1920

2021
clean:
@@ -75,7 +76,8 @@ build-pkg: clean generate-skills
7576
build-fat: generate-skills
7677
@echo "📦 Building standalone binaries (multi-version)..."
7778
bash scripts/build_fat_multi_platform.sh
78-
# After building the zip files, we want to unzip them into a skill zip file
79+
# copy setup.sh to dist for convenience
80+
cp scripts/setup.sh pyinstaller/dist/cz-cli/
7981
@echo "✅ Standalone binaries build complete"
8082

8183
build: build-pkg
@@ -97,6 +99,13 @@ upload-test:
9799
python -m twine upload --repository testpypi dist/*
98100
@echo "✅ Upload to TestPyPI complete"
99101

102+
tag:
103+
$(eval VERSION := v$(shell python -c "exec(open('cz_cli/version.py').read()); print(__version__)"))
104+
@echo "🏷️ Tagging $(VERSION)..."
105+
git tag $(VERSION)
106+
git push origin $(VERSION)
107+
@echo "✅ Tag $(VERSION) pushed"
108+
100109
all: clean format test verify build
101110
@echo "✅ All tasks complete"
102111

cz_cli/auto_update.py

Lines changed: 8 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -48,6 +48,10 @@ def _read_state() -> dict:
4848

4949
def _write_state(state: dict) -> None:
5050
STATE_FILE.parent.mkdir(parents=True, exist_ok=True)
51+
existing = _read_state()
52+
# preserve skip_update across writes
53+
if "skip_update" not in state:
54+
state["skip_update"] = existing.get("skip_update", False)
5155
STATE_FILE.write_text(json.dumps(state))
5256

5357

@@ -125,7 +129,10 @@ def check_and_update(*, debug: bool = False) -> None:
125129
debug=True → loguru debug level (for normal commands, only visible with --debug)
126130
debug=False → stderr directly (for --help, human-readable)
127131
"""
128-
if os.environ.get("CZ_SKIP_UPDATE", "").strip().lower() in {"1", "true", "yes"}:
132+
state = _read_state()
133+
skip_env = os.environ.get("CZ_SKIP_UPDATE", "").strip().lower()
134+
skip_file = str(state.get("skip_update", False)).strip().lower()
135+
if skip_env in {"1", "true", "yes"} or skip_file in {"1", "true", "yes"}:
129136
return
130137
if not _is_standalone_binary():
131138
return

cz_cli/commands/job.py

Lines changed: 188 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,188 @@
1+
"""Job commands — status, download plan/profile, and analyze."""
2+
3+
from __future__ import annotations
4+
5+
import json
6+
import os
7+
from typing import Any
8+
9+
import click
10+
11+
from cz_cli import output
12+
from cz_cli.cli_group import CLIGroup
13+
from cz_cli.guide_builder import examples_epilog
14+
from cz_cli.logger import log_operation
15+
from cz_cli.studio_client import StudioClient, configure_mcp_logging, studio_connection_kwargs
16+
17+
18+
def _new_client(ctx: click.Context) -> StudioClient:
19+
configure_mcp_logging("DEBUG" if ctx.obj.get("debug") else None)
20+
return StudioClient(
21+
profile=ctx.obj.get("profile"),
22+
jdbc_url=ctx.obj.get("jdbc_url"),
23+
**studio_connection_kwargs(ctx.obj),
24+
)
25+
26+
27+
_DOWNLOAD_EXAMPLES = [
28+
{"cmd": "cz-cli job download 2026012808001805432z9g3fx1sok", "desc": "Download job plan & profile to ./job_<id>/"},
29+
{"cmd": "cz-cli job download 2026012808001805432z9g3fx1sok --output-dir /tmp/mydir", "desc": "Download to a custom directory"},
30+
]
31+
32+
_ANALYZE_EXAMPLES = [
33+
{"cmd": "cz-cli job analyze 2026012808001805432z9g3fx1sok", "desc": "Quick performance analysis"},
34+
{"cmd": "cz-cli job analyze 2026012808001805432z9g3fx1sok --mode detailed", "desc": "Detailed analysis"},
35+
{"cmd": "cz-cli job analyze --path /tmp/mydir", "desc": "Analyze from local files (no job_id needed)"},
36+
]
37+
38+
39+
@click.group("job", cls=CLIGroup)
40+
@click.pass_context
41+
def job_cmd(ctx: click.Context) -> None:
42+
"""Job performance: download plan/profile and analyze."""
43+
pass
44+
45+
46+
@job_cmd.command("download", epilog=examples_epilog(_DOWNLOAD_EXAMPLES))
47+
@click.argument("job_id")
48+
@click.option("--workspace", "-w", "workspace_name", default=None, help="Workspace name (uses profile default if omitted).")
49+
@click.option("--output-dir", "output_dir", default=None, help="Output directory (default: ./job_<job_id>).")
50+
@click.pass_context
51+
def job_download(ctx: click.Context, job_id: str, workspace_name: str | None, output_dir: str | None) -> None:
52+
"""Download job plan and profile JSON for a job ID."""
53+
fmt: str = ctx.obj.get("format", "json")
54+
timer = output.Timer()
55+
56+
try:
57+
client = _new_client(ctx)
58+
except Exception as exc:
59+
log_operation("job download", ok=False, error_code="CONNECTION_ERROR")
60+
output.error("CONNECTION_ERROR", str(exc), fmt=fmt)
61+
return
62+
63+
args: dict[str, Any] = {"job_id": job_id}
64+
if workspace_name:
65+
args["workspace_name"] = workspace_name
66+
67+
try:
68+
with timer:
69+
result = client.invoke("fetch_job_plan_profile_data", args)
70+
except Exception as exc:
71+
log_operation("job download", ok=False, error_code="DOWNLOAD_ERROR")
72+
output.error("DOWNLOAD_ERROR", str(exc), fmt=fmt)
73+
return
74+
75+
payload = result.payload
76+
dest = output_dir or f"job_{job_id}"
77+
os.makedirs(dest, exist_ok=True)
78+
79+
# Save job_profile.json
80+
profile_data = payload.get("job_profile_data")
81+
if profile_data:
82+
with open(os.path.join(dest, "job_profile.json"), "w", encoding="utf-8") as f:
83+
json.dump(profile_data, f, ensure_ascii=False, indent=2)
84+
85+
# Save job_plan.json
86+
plan_data = payload.get("job_plan_data")
87+
if plan_data:
88+
with open(os.path.join(dest, "job_plan.json"), "w", encoding="utf-8") as f:
89+
json.dump(plan_data, f, ensure_ascii=False, indent=2)
90+
91+
log_operation("job download", ok=True, time_ms=timer.elapsed_ms)
92+
output.success(
93+
{
94+
"job_id": job_id,
95+
"output_dir": os.path.abspath(dest),
96+
"job_plan_exists": plan_data is not None,
97+
"job_profile_exists": profile_data is not None,
98+
},
99+
time_ms=timer.elapsed_ms,
100+
fmt=fmt,
101+
)
102+
103+
104+
@job_cmd.command("analyze", epilog=examples_epilog(_ANALYZE_EXAMPLES))
105+
@click.argument("job_id", required=False, default=None)
106+
@click.option("--workspace", "-w", "workspace_name", default=None, help="Workspace name (uses profile default if omitted).")
107+
@click.option("--mode", "analysis_mode", type=click.Choice(["quick", "detailed", "expert"]), default="quick", help="Analysis mode.")
108+
@click.option("--path", "local_path", default=None, help="Local folder with job_plan.json & job_profile.json (job_id optional when set).")
109+
@click.option("--state-table/--no-state-table", "enable_state_table", default=True, help="Enable state table optimization.")
110+
@click.option("--incremental/--no-incremental", "enable_incremental", default=False, help="Enable incremental algorithm.")
111+
@click.pass_context
112+
def job_analyze(
113+
ctx: click.Context,
114+
job_id: str | None,
115+
workspace_name: str | None,
116+
analysis_mode: str,
117+
local_path: str | None,
118+
enable_state_table: bool,
119+
enable_incremental: bool,
120+
) -> None:
121+
"""Analyze job performance and provide optimization recommendations."""
122+
fmt: str = ctx.obj.get("format", "json")
123+
124+
if not job_id and not local_path:
125+
output.error("MISSING_ARGUMENT", "JOB_ID is required unless --path is provided.", fmt=fmt)
126+
return
127+
128+
timer = output.Timer()
129+
130+
try:
131+
client = _new_client(ctx)
132+
except Exception as exc:
133+
log_operation("job analyze", ok=False, error_code="CONNECTION_ERROR")
134+
output.error("CONNECTION_ERROR", str(exc), fmt=fmt)
135+
return
136+
137+
args: dict[str, Any] = {
138+
"analysis_mode": analysis_mode,
139+
"enable_state_table": enable_state_table,
140+
"enable_incremental_algorithm": enable_incremental,
141+
}
142+
if job_id:
143+
args["job_id"] = job_id
144+
if workspace_name:
145+
args["workspace_name"] = workspace_name
146+
if local_path:
147+
args["path"] = local_path
148+
149+
try:
150+
with timer:
151+
result = client.invoke("fetch_job_performance_data", args)
152+
except Exception as exc:
153+
log_operation("job analyze", ok=False, error_code="ANALYSIS_ERROR")
154+
output.error("ANALYSIS_ERROR", str(exc), fmt=fmt)
155+
return
156+
157+
log_operation("job analyze", ok=True, time_ms=timer.elapsed_ms)
158+
output.success(result.payload, time_ms=timer.elapsed_ms, fmt=fmt)
159+
160+
161+
@job_cmd.command("status")
162+
@click.argument("job_id")
163+
@click.pass_context
164+
def job_status(ctx: click.Context, job_id: str) -> None:
165+
"""Check status / summary of a SQL job."""
166+
from cz_cli.connection import get_connection
167+
from cz_cli.connection_ctx import connection_kwargs_from_ctx
168+
169+
fmt: str = ctx.obj.get("format", "json")
170+
profile: str | None = ctx.obj.get("profile")
171+
jdbc_url: str | None = ctx.obj.get("jdbc_url")
172+
173+
try:
174+
conn = get_connection(jdbc_url=jdbc_url, profile=profile, **connection_kwargs_from_ctx(ctx))
175+
except Exception as exc:
176+
log_operation("job status", ok=False, error_code="CONNECTION_ERROR")
177+
output.error("CONNECTION_ERROR", str(exc), fmt=fmt)
178+
return
179+
180+
try:
181+
summary = conn.get_job_summary(job_id)
182+
log_operation("job status", ok=True)
183+
output.success(summary, fmt=fmt)
184+
except Exception as exc:
185+
log_operation("job status", ok=False, error_code="JOB_STATUS_ERROR")
186+
output.error("JOB_STATUS_ERROR", str(exc), fmt=fmt)
187+
finally:
188+
conn.close()

cz_cli/commands/profile.py

Lines changed: 27 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -51,7 +51,7 @@ def _save_profiles(data: dict[str, Any]) -> None:
5151
for profile_name, profile_data in data.get("profiles", {}).items():
5252
lines.append(f"[profiles.{profile_name}]")
5353
for key, value in profile_data.items():
54-
if key == "agent":
54+
if key in ("agent", "header"):
5555
continue # written as sub-table below
5656
if isinstance(value, str):
5757
lines.append(f'{key} = "{value}"')
@@ -66,6 +66,12 @@ def _save_profiles(data: dict[str, Any]) -> None:
6666
lines.append(f'{k} = "{v}"')
6767
else:
6868
lines.append(f"{k} = {v}")
69+
# Write [header] sub-table if present
70+
header = profile_data.get("header")
71+
if header:
72+
lines.append(f"[profiles.{profile_name}.header]")
73+
for k, v in header.items():
74+
lines.append(f'"{k}" = "{v}"')
6975
lines.append("")
7076

7177
content = "\n".join(lines)
@@ -208,6 +214,7 @@ def show_profile(ctx: click.Context, name: str, show_secret: bool) -> None:
208214
@click.option("--workspace", help="Workspace name")
209215
@click.option("--schema", help="Default schema")
210216
@click.option("--vcluster", help="Virtual cluster")
217+
@click.option("--header", multiple=True, help="Custom HTTP header KEY=VALUE (repeatable, stored as header.<KEY>).")
211218
@click.option("--skip-verify", is_flag=True, help="Skip connection verification")
212219
@click.pass_context
213220
def create_profile(
@@ -223,6 +230,7 @@ def create_profile(
223230
workspace: str | None,
224231
schema: str | None,
225232
vcluster: str | None,
233+
header: tuple[str, ...],
226234
skip_verify: bool,
227235
) -> None:
228236
"""Create a new profile."""
@@ -347,6 +355,11 @@ def create_profile(
347355
else:
348356
profile_obj["username"] = resolved_username
349357
profile_obj["password"] = resolved_password
358+
# Store custom headers as header.<KEY> = VALUE
359+
for h in header:
360+
if "=" in h:
361+
hk, hv = h.split("=", 1)
362+
profile_obj[f"header.{hk.strip()}"] = hv.strip()
350363
profiles[name] = profile_obj
351364

352365
data["profiles"] = profiles
@@ -391,10 +404,10 @@ def update_profile(ctx: click.Context, name: str, key: str, value: str) -> None:
391404
"schema",
392405
"vcluster",
393406
]
394-
if key not in valid_keys:
407+
if key not in valid_keys and not key.startswith("header."):
395408
log_operation("profile update", ok=False, error_code="INVALID_KEY")
396409
output.error(
397-
"INVALID_KEY", f"Invalid key '{key}'. Valid keys: {', '.join(valid_keys)}", fmt=fmt
410+
"INVALID_KEY", f"Invalid key '{key}'. Valid keys: {', '.join(valid_keys)}, header.<NAME>", fmt=fmt
398411
)
399412
return
400413
if key == "protocol":
@@ -405,7 +418,17 @@ def update_profile(ctx: click.Context, name: str, key: str, value: str) -> None:
405418
return
406419
value = normalized
407420

408-
profiles[name][key] = value
421+
if key.startswith("header."):
422+
hdr_name = key[7:]
423+
hdr_dict = profiles[name].setdefault("header", {})
424+
if value:
425+
hdr_dict[hdr_name] = value
426+
else:
427+
hdr_dict.pop(hdr_name, None)
428+
if not hdr_dict:
429+
profiles[name].pop("header", None)
430+
else:
431+
profiles[name][key] = value
409432
if key == "pat":
410433
profiles[name].pop("username", None)
411434
profiles[name].pop("password", None)

0 commit comments

Comments
 (0)