Admin backend: publish wiki data to a Hugging Face account

[schiste]

Summary

Add the ability, from the admin backend, to publish generated wiki-economics data (parquet outputs / dashboard datasets) to a Hugging Face account, alongside the existing Observable site publish flow.

Context

The admin server (site/admin-server.cjs) already exposes a job-control API for the pipeline (/fetch, /ingest, /compute, /merge, /run, etc. — see docs/admin-server.md) and separately publishes the built Observable site (f45b937 fix: publish Observable site atomically). There's no equivalent path today for pushing a dataset artifact to an external data host like Hugging Face Hub (e.g. as a dataset repo).

Proposed scope

• New admin API route, e.g. POST /publish-huggingface, following the same job-control conventions as the existing endpoints (single job at a time, 409 if one is already running, status/log surfacing via /status).
• Publish target: a Hugging Face dataset repo (not a model), containing the merged parquet output (and/or the generated dashboard JSON) for a given wiki or the full merged set.
• Credentials handled the same way as existing secrets (WIKI_ECON_ADMIN_* pattern) — likely:
  • WIKI_ECON_HF_TOKEN — HF access token (write-scoped), kept in deployment secrets, never committed.
  • WIKI_ECON_HF_REPO_ID — target dataset repo, e.g. org-or-user/wiki-economics.
• Atomic publish semantics matching the existing snapshot/site publish work (b9cf817, f45b937) — avoid partial/corrupt uploads being visible on the HF repo.
• Admin UI: a "Publish to Hugging Face" action/button next to the existing pipeline controls, surfaced only when the feature is configured (token/repo present).

Open questions

• Which artifact(s) get published — raw per-wiki parquet, the merged dataset, generated dashboard JSON, or a combination?
• Publish cadence: manual trigger only (mirroring current job-control model), or wire into the same lifecycle as the monthly snapshot generation (26800a1)?
• Do we want dataset versioning/commit messages on the HF side per publish, or a single always-overwritten revision?

Out of scope (for now)

• Publishing to Hugging Face Spaces or models — this issue is scoped to the dataset-publish use case only.