Add proxy auth support and status/metrics commands

[nik-kale]

• Add authenticated proxy support via environment variables or config
• Send proxy credentials as a Basic Proxy-Authorization header (Digest and NTLM are not supported)
• Add 'cvd status' command for database health checks
• Add 'cvd metrics' command for Prometheus monitoring
• Bump version to 1.3.0

Closes #7 and #9. All existing tests pass, plus new unit tests covering the proxy, status, and metrics paths, including Click-level CLI tests.

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: a84f72c1e1

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

[nik-kale]

Updated based on Codex review feedback:

• Fixed proxy auth: switched from requests.auth objects (which set Authorization instead of Proxy-Authorization) to embedding credentials directly in the proxy URL
• Fixed UnboundLocalError when using --json --check together
• Removed --proxy-auth-type option and NTLM dependency (no longer needed with URL-embedded credentials)
• Updated tests and docs to match

[val-ms]

Thanks for the contribution. The proxy auth and monitoring additions are useful directions, but I think this needs changes before we can merge.

Blocking issues:

• cvd config set is currently broken for normal use. The new optional --proxy-cert and --proxy-cert-key options use click.Path(exists=True) with default="", so Click validates the empty default and exits before the command runs. I reproduced this locally with cvd config set --dbdir /tmp/db, which fails with Invalid value for '--proxy-cert': Path '' does not exist. These should default to None or otherwise avoid validating an absent optional path, and the CLI behavior should be covered by tests.

• The PR says it closes #30, but the update path still requires DNS before any HTTP/proxy download can happen. db_update() still calls _query_dns_txt_entry() and returns 1 when dns_version_tokens is empty, so environments without direct DNS still cannot update through an HTTP proxy. Please either remove the Closes #30 claim or implement and test a real no-DNS fallback.

• Proxy credentials can leak in logs/config output when credentials are embedded in proxy_url. _get_proxy_configuration() logs the full proxy URL in the unauthenticated branch, so http://user:pass@proxy.example.com:8080 exposes the secret. config_show() only masks proxy_pass, not credentials already present in proxy_url. Please sanitize userinfo in all displayed/logged proxy URLs.

Other requested changes:

• The PR body still claims Basic, Digest, NTLM, and certificate-based auth. The current implementation embeds credentials in the proxy URL, which Requests turns into Basic Proxy-Authorization; it does not implement Digest or NTLM. Please narrow the claim or add real support/tests for those auth types.

• cvdupdate_last_check_timestamp is documented as a Unix timestamp but emits milliseconds (time.time() * 1000). Please either emit seconds or rename/re-document it as milliseconds.

• Prometheus label values are interpolated without escaping. Custom database names containing quotes, backslashes, or newlines will produce invalid exposition output. Please escape label values per Prometheus text format rules.

• CHANGES.md still contains placeholder PR links (pull/XX) and says this closes #30. Please update those before merge.

Compatibility note:

This PR also conflicts with PR #88. I checked the current heads with git merge-tree; #81 merges cleanly with #87, but conflicts with #88 in cvdupdate/__main__.py and cvdupdate/cvdupdate.py. The conflict is semantic as well as textual: both PRs introduce or redefine status, both change config set, and #88 restructures config/state handling that #81 builds on. If #88 moves forward, this PR will need to be rebased and reconciled around the CLI design, especially the meaning of status.

Local verification: the PR test suite passes for me (37 passed in 3.05s), and the CLI/config regressions above were reproduced in an isolated temporary venv.

[nik-kale]

Thanks for the detailed review @val-ms. I have addressed each point.

Blocking issues:

1. config set. I removed --proxy-cert and --proxy-cert-key entirely rather than only fixing the empty-default validation. On inspection the requests cert parameter is presented to the destination server during the TLS handshake, not to the proxy during CONNECT, so those options never provided proxy mTLS. Removing them fixes the config set regression and drops a feature that did not do what it claimed. config set is now covered by CliRunner tests.

2. Closes cvdupdate doesn't work without DNS #30. Dropped. A correct no-DNS path is constrained by the bandwidth design in Disable db version check over HTTP when using database.clamav.net #19, and an HTTP-only fallback for custom databases would not help the cvdupdate doesn't work without DNS #30 case, which uses the official databases behind a proxy. I would rather not claim it than ship a partial fix. CHANGES.md now lists Unable to use proxy #7 and Does cvdupdate work with proxy #9 only.

3. Credential leakage. Proxy credentials are now masked wherever they are logged or displayed. _get_proxy_configuration logs a sanitized URL in both branches, and config show masks userinfo embedded in proxy_url in addition to proxy_pass. Both are covered by tests.

Other requested changes:

• Auth claim narrowed to Basic Proxy-Authorization in the PR description and CHANGES.md. Digest and NTLM are not implemented.
• cvdupdate_last_check_timestamp now emits seconds, matching the name and help text.
• Prometheus label values are escaped for backslash, double quote, and newline.

While addressing the above I fixed a few related issues:

• status printed local time labeled UTC; it now reports real UTC.
• status used file age as the health signal, which flagged infrequently changing databases such as main.cvd as stale or critical even on a current mirror. Version state now drives health, age is informational, and an unverifiable version (DNS unavailable) reports as unknown rather than outdated.
• metrics --serve ran a live DNS query on every scrape. Status is now cached with a configurable TTL, default 60 seconds.
• A scheme-less proxy URL now produces a clear warning.

On the #88 overlap: agreed. If #88 lands first I will rebase and reconcile the status command and config set design. Happy to coordinate on ordering.

Test suite passes locally, 51 passed.

[nik-kale]

Rebased onto current main, so this now sits on top of #88. Per your note, I
reconciled around the CLI: the health/currency command is now cvd health
so it doesn't collide with #88's status, which I left alone.

Your May 29 points are addressed: Proxy-Authorization (not Authorization);
credentials redacted from logs and from both config show outputs; only
Basic-via-URL is claimed (no Digest/NTLM/mTLS); metric timestamp is in seconds;
label values are escaped; dropped the #30 claim (now Closes #7, #9); and
removed the pull/XX placeholders.

While rebasing I also tightened a few things the feature needs to actually hold
up: health --json and metrics keep their log output on stderr so the JSON /
Prometheus text stays parseable; malformed / IPv6 / scheme-less proxy URLs no
longer crash config show; the config file is written 0600 and I documented
that credentials are stored in plaintext (env vars recommended); and
metrics --serve uses a threaded server so a slow client can't wedge scrapes.

One behavior choice worth flagging: health --check treats a transient DNS
failure as informational rather than unhealthy when the on-disk databases are
current, so a DNS blip doesn't page a cron job. Easy to flip if you'd rather it
warn.

I kept this focused on the three features. Two adjacent things I noticed I've
split into separate branches rather than pile in here - a pre-existing serve
debug-logging leak, and hardening _index_local_databases against a corrupt
stray file - so this PR stays reviewable; I'll open those as their own small PRs.

No rush given your triage load @val-ms - happy to adjust scope however makes this
easiest to review.

[val-ms]

I found four actionable issues that should be addressed before merge:

1. config show can still leak embedded proxy credentials for scheme-less proxy URLs. _get_proxy_configuration() prefixes http:// before logging, but config show sanitizes the raw stored string. For a stored value such as alice:secret@proxy.example.com:8080, urlparse() treats alice as the scheme, sees no username/password, and _sanitize_proxy_url() returns the secret unchanged. Please normalize scheme-less URLs before sanitizing, or make _sanitize_proxy_url() handle this form directly, and add a regression test for scheme-less embedded userinfo.

2. _save_config() writes the config before applying chmod 0600. A new config file containing proxy_pass can be created under the process umask, often world-readable, before permissions are tightened. Existing config files may also remain too permissive through the write. Please create/truncate the config with owner-only permissions up front, or chmod an existing config before writing the plaintext password.

3. The new tests use CliRunner(mix_stderr=False), but setup.py allows click>=7.0, and current Click 8.4.2 has removed that constructor argument. The focused suite fails under a valid dependency set. Click 8.4 still exposes result.stdout and result.stderr, so the tests can likely be updated to use those properties instead of the removed constructor option, unless the project intentionally pins an older Click.

4. git diff --check origin/main...HEAD fails due extra blank lines at EOF in cvdupdate/metrics.py, tests/test_metrics.py, tests/test_proxy_auth.py, and tests/test_status.py.

Local test notes: with a temp venv under /private/tmp, tests/test_proxy_auth.py tests/test_status.py tests/test_metrics.py passed (49 passed). The CLI subset failed only for the mix_stderr=False tests after excluding localhost-bind tests; the two localhost server tests passed when rerun outside the sandbox.

[nikkal12]

Thanks for the review, @val-ms. All four points are addressed:

1. config show no longer leaks credentials for a scheme-less proxy URL. _sanitize_proxy_url normalizes a scheme-less value before masking, so a stored user:pass@host:port is masked the same as a full URL. Added regression tests for the helper and for config show.
2. _save_config now creates and truncates the config as 0600 and chmods the descriptor before writing, so a plaintext proxy_pass is never briefly world-readable, including when overwriting an existing file. Added a test that pre-creates a 0644 config and asserts 0600 after save.
3. Dropped CliRunner(mix_stderr=False). The stderr-separation tests use a small helper that works on Click before and after 8.2, so the suite passes under current Click. I ran the full suite under 8.4.2 and it is green.
4. Removed the trailing blank lines; git diff --check is now clean.

One related thing I found while on point 3: config set --proxy-pass (the prompt form) relies on Click 8's optional-value flags, so it errors on Click 7 with "option requires an argument". The declared floor in setup.py is click>=7.0. Happy to bump it to click>=8.0 here, or send it as a small separate change, whichever you prefer. I could also add a python_requires at the same time.

The Prometheus polish I mentioned earlier (grouping each metric family and a couple of naming conventions), I'm happy to still send as the separate follow-up.