Skip to content

fix(cli): reject unparseable numeric flags in benchmark and chat - #161

Open
addyCooks wants to merge 1 commit into
Nano-Collective:mainfrom
addyCooks:fix/134-validate-numeric-flags
Open

fix(cli): reject unparseable numeric flags in benchmark and chat#161
addyCooks wants to merge 1 commit into
Nano-Collective:mainfrom
addyCooks:fix/134-validate-numeric-flags

Conversation

@addyCooks

Copy link
Copy Markdown
Contributor

Description

nanotune train validates its numeric flags before it uses them, but benchmark and chat parsed the identical class of flag with a bare Number.parseInt/parseFloat and no check at all.

parseInt stops at the first character it cannot parse, so --ctx-size 4096x silently became 4096 and the run proceeded under a value the user never typed. Anything that failed outright became NaN and reached llama-server unchecked: String(NaN) is "NaN", so --gpu-layers abc spawned the real argument -ngl NaN, while JSON.stringify(NaN) is null, so a bad --max-tokens/--temperature/--top-p left the field out of the completion body and llama-server quietly used its own default. --timeout was worse still setTimeout(fn, NaN) fires immediately, so a typo aborted every test the instant it started.

Every numeric flag on both commands now goes through one shared parseNumericFlag() in chat-helpers.ts, which uses Number() rather than parseInt/parseFloat so trailing garbage fails instead of truncating, and rejects a fractional value where llama-server wants an integer. A rejected flag is reported as Invalid value for --flag, matching train's existing message, and the command stops before any subprocess or network work: for chat ahead of the project check, for benchmark alongside the sampling flags, ahead of the base-model download and the server spawn.

benchmark's flag wiring moves into resolveBenchmarkFlags() in benchmark-utils.ts, next to the sampling flags it already resolved there, which also drops the duplicate copy of chat's server-option builder and lets the whole path be unit-tested. Flags a --preset overrides are still parsed, so a typo alongside a preset is reported rather than silently discarded.

Closes #134.

Behavior changes

  • --ctx-size 4096.5 is now rejected rather than truncated to 4096.
  • A typo'd numeric flag passed alongside a valid --preset now errors instead of being ignored (the preset still wins on values).

Type of Change

  • Bug fix
  • New feature
  • Breaking change
  • Documentation update

Testing

Automated Tests

  • All existing tests pass (pnpm test:all completes successfully)
  • New tests added for new functionality (if applicable)

Manual Testing

  • Tested nanotune init
  • Tested nanotune data commands (add/import/list/validate)
  • Tested nanotune train
  • Tested nanotune export
  • Tested nanotune benchmark

Checklist

  • Code follows project style guidelines (pnpm format)
  • Self-review completed
  • Documentation updated (if needed)
  • No breaking changes (or clearly documented)

`nanotune train` validates its numeric flags before it uses them, but
`benchmark` and `chat` parsed the identical class of flag with a bare
Number.parseInt/parseFloat and no check at all.

parseInt stops at the first character it cannot parse, so `--ctx-size
4096x` silently became 4096 and the run proceeded under a value the user
never typed. Anything that failed outright became NaN and reached
llama-server unchecked: String(NaN) is "NaN", so `--gpu-layers abc`
spawned the real argument `-ngl NaN`, while JSON.stringify(NaN) is null,
so a bad `--max-tokens`/`--temperature`/`--top-p` left the field out of
the completion body and llama-server quietly used its own default.
`--timeout` was worse still — setTimeout(fn, NaN) fires immediately, so
a typo aborted every test the instant it started.

Every numeric flag on both commands now goes through one shared
parseNumericFlag() in chat-helpers.ts, which uses Number() rather than
parseInt/parseFloat so trailing garbage fails instead of truncating, and
rejects a fractional value where llama-server wants an integer. A
rejected flag is reported as "Invalid value for --flag", matching train's
existing message, and the command stops before any subprocess or network
work: for chat ahead of the project check, for benchmark alongside the
sampling flags, ahead of the base-model download and the server spawn.

benchmark's flag wiring moves into resolveBenchmarkFlags() in
benchmark-utils.ts, next to the sampling flags it already resolved there,
which also drops the duplicate copy of chat's server-option builder and
lets the whole path be unit-tested. Flags a `--preset` overrides are
still parsed, so a typo alongside a preset is reported rather than
silently discarded.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Bug] Non-numeric flags to nanotune benchmark and nanotune chat become NaN and reach llama-server unvalidated

1 participant