Skip to content

[Feature request] Auto-generate a model card (README.md) for saved speculator checkpoints #1055

Description

@rasyosef

[Feature request] Auto-generate a model card (README.md) for saved speculator checkpoints

Motivation.

I trained DSpark draft models with Speculators and pushed the best checkpoint to the Hugging Face Hub. The upload works, but the repo has no README.md, so the model page is blank: no frontmatter, no link to the verifier, no usage snippet.

Checkpoints currently contain config.json, model.safetensors, optimizer_state_dict.pt, train_command.txt and val_metrics.json, plus run.yaml at the run root — but no README.md, and no model-card or hub-upload code exists in src/ or scripts/.

Why it matters:

  • A speculator is useless without its verifier. That link lives only in speculator_config inside config.json. Putting it in base_model: frontmatter makes the Hub render the relationship.
  • Discoverability. Without library_name/tags, community speculators don't surface in Hub search or filtering.
  • The published RedHatAI cards set an expectation the library doesn't help users meet. Hand-writing a card is exactly the step people skip.
  • Everything needed is already on disk: architecture from config.json, training config from run.yaml, command/git SHA/package versions from train_command.txt, validation loss from val_metrics.json. This is templating over existing artifacts.

I couldn't find an existing issue for this — happy to close as a duplicate if I missed one.

Proposed Change.

  1. Card generator (src/speculators/model_card.py) that renders README.md from a checkpoint directory. huggingface-hub is already a base dependency, so ModelCard/ModelCardData covers this with no new deps.
  2. Emit during checkpointing in BaseCheckpointer.save_checkpoint, so checkpoint_best is directly uploadable. Gated by --model-card / --no-model-card; never overwrite an existing README.md without an explicit flag.
  3. CLI entry point speculators model-card <checkpoint_dir> for already-trained checkpoints, reused at the end of speculators convert.

Card contents:

  • Frontmatter: library_name: speculators, base_model: (verifier from config.json), tags: (speculative-decoding, vllm, algorithm), pipeline_tag: text-generation.
  • Body: vLLM usage snippet; architecture from config.json (including algorithm-specific fields like DSpark's markov_rank); training summary from run.yaml; reproduction command and environment from train_command.txt; validation metrics table (below); acceptance.csv/perf_results.csv when present.

Validation metrics table. compute_metrics already produces exactly the numbers a reader wants, and val_metrics.json persists them — they just never reach the card. Render them as markdown:

  • A summary row: accept_len_epoch (expected accepted draft length), accept_rate_epoch, full_acc_epoch, loss_epoch.
  • A per-position table from the position_{i}_acc_epoch keys, one row per draft slot, so readers can see acceptance decay across the block — the single most useful signal for choosing num_speculative_tokens at serve time.
  • For DSpark, the confidence-head calibration metrics (confidence_abs_error_epoch, confidence_cumprod_bias_epoch).

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions