Skip to content

SALM with NeMo Automodel integration for Nemotron Nano V3 LLM backbone#15447

Open
pzelasko wants to merge 73 commits intomainfrom
speechlm2-with-nemo-automodel-merge
Open

SALM with NeMo Automodel integration for Nemotron Nano V3 LLM backbone#15447
pzelasko wants to merge 73 commits intomainfrom
speechlm2-with-nemo-automodel-merge

Conversation

@pzelasko
Copy link
Collaborator

Important

The Update branch button must only be pressed in very rare occassions.
An outdated branch is never blocking the merge of a PR.
Please reach out to the automation team before pressing that button.

What does this PR do ?

Add a one line overview of what this PR aims to accomplish.

Collection: [Note which collection this PR will affect]

Changelog

  • Add specific line by line info of high level changes in this PR.

Usage

  • You can potentially add a usage example below
# Add a code snippet demonstrating how to use this 

GitHub Actions CI

The Jenkins CI system has been replaced by GitHub Actions self-hosted runners.

The GitHub Actions CI will run automatically when the "Run CICD" label is added to the PR.
To re-run CI remove and add the label again.
To run CI on an untrusted fork, a NeMo user with write access must first click "Approve and run".

Before your PR is "Ready for review"

Pre checks:

  • Make sure you read and followed Contributor guidelines
  • Did you write any new necessary tests?
  • Did you add or update any necessary documentation?
  • Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
    • Reviewer: Does the PR have correct import guards for all optional libraries?

PR Type:

  • New Feature
  • Bugfix
  • Documentation

If you haven't finished some of the above items you can still open "Draft" PR.

Who can review?

Anyone in the community is free to review the PR once the checks have passed.
Contributor guidelines contains specific people who can review PRs to various areas.

Additional Information

  • Related to # (issue)

pzelasko and others added 30 commits February 4, 2026 14:17
Signed-off-by: Piotr Żelasko <pzelasko@nvidia.com>
Signed-off-by: Piotr Żelasko <pzelasko@nvidia.com>
Signed-off-by: Piotr Żelasko <petezor@gmail.com>
Signed-off-by: Piotr Żelasko <petezor@gmail.com>
Signed-off-by: Piotr Żelasko <petezor@gmail.com>
Signed-off-by: Piotr Żelasko <petezor@gmail.com>
Signed-off-by: Piotr Żelasko <petezor@gmail.com>
Signed-off-by: Piotr Żelasko <petezor@gmail.com>
Signed-off-by: Piotr Żelasko <petezor@gmail.com>
Signed-off-by: Piotr Żelasko <petezor@gmail.com>
Signed-off-by: Piotr Żelasko <petezor@gmail.com>
Signed-off-by: Piotr Żelasko <petezor@gmail.com>
Signed-off-by: Piotr Żelasko <pzelasko@nvidia.com>
Signed-off-by: Piotr Żelasko <petezor@gmail.com>
Signed-off-by: Piotr Żelasko <petezor@gmail.com>
Signed-off-by: Piotr Żelasko <petezor@gmail.com>
Signed-off-by: Piotr Żelasko <petezor@gmail.com>
Signed-off-by: Piotr Żelasko <petezor@gmail.com>
Signed-off-by: Piotr Żelasko <petezor@gmail.com>
…automodel's utility

Signed-off-by: Piotr Żelasko <petezor@gmail.com>
…full LLM

Signed-off-by: Piotr Żelasko <petezor@gmail.com>
Signed-off-by: Piotr Żelasko <petezor@gmail.com>
Signed-off-by: Piotr Żelasko <pzelasko@nvidia.com>
Signed-off-by: Piotr Żelasko <petezor@gmail.com>
Signed-off-by: Piotr Żelasko <petezor@gmail.com>
Signed-off-by: Piotr Żelasko <petezor@gmail.com>
Signed-off-by: Piotr Żelasko <petezor@gmail.com>
Signed-off-by: Piotr Żelasko <petezor@gmail.com>
@pzelasko
Copy link
Collaborator Author

/claude review

roles = self.get_roles()
assert len(turns) > 0, "Empty dialog is not supported."
for turn in turns:
assert "role" in turn, f"A turn must have have a 'role' key. We received {turn=}"
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Typo: duplicated "have"

Suggested change
assert "role" in turn, f"A turn must have have a 'role' key. We received {turn=}"
assert "role" in turn, f"A turn must have a 'role' key. We received {turn=}"

roles = self.get_roles()
assert len(turns) > 0, "Empty dialog is not supported."
for turn in turns:
assert "role" in turn, f"A turn must have have a 'role' key. We received {turn=}"
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same typo: duplicated "have"

Suggested change
assert "role" in turn, f"A turn must have have a 'role' key. We received {turn=}"
assert "role" in turn, f"A turn must have a 'role' key. We received {turn=}"

slot_values = turn.get("slots", {})
if expected_slots:
assert slot_values, (
f"A turn for role {role} must have have a non-empty value under 'slots' key. "
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same typo here too: duplicated "have"

Suggested change
f"A turn for role {role} must have have a non-empty value under 'slots' key. "
f"A turn for role {role} must have a non-empty value under 'slots' key. "

roles = self.get_roles()
assert len(turns) > 0, "Empty dialog is not supported."
for turn in turns:
assert "role" in turn, f"A turn must have have a 'role' key. We received {turn=}"
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Typo: "have have" → "have"

Suggested change
assert "role" in turn, f"A turn must have have a 'role' key. We received {turn=}"
assert "role" in turn, f"A turn must have a 'role' key. We received {turn=}"

roles = self.get_roles()
assert len(turns) > 0, "Empty dialog is not supported."
for turn in turns:
assert "role" in turn, f"A turn must have have a 'role' key. We received {turn=}"
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same "have have" typo here.

Suggested change
assert "role" in turn, f"A turn must have have a 'role' key. We received {turn=}"
assert "role" in turn, f"A turn must have a 'role' key. We received {turn=}"

slot_values = turn.get("slots", {})
if expected_slots:
assert slot_values, (
f"A turn for role {role} must have have a non-empty value under 'slots' key. "
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same "have have" typo.

Suggested change
f"A turn for role {role} must have have a non-empty value under 'slots' key. "
f"A turn for role {role} must have a non-empty value under 'slots' key. "

if self._use_tp:
tp_world_size = self.device_mesh["tp"].size()
if (remainder := (input_embs.shape[1] - 1) % tp_world_size) != 0:
# Truncate some tokens from the end to make the sequence lenght shape divisible by tensor parallelism
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Typo: "lenght" → "length"

Suggested change
# Truncate some tokens from the end to make the sequence lenght shape divisible by tensor parallelism
# Truncate some tokens from the end to make the sequence length shape divisible by tensor parallelism

}


class Qwen3PromptFormatter(PromptFormatter):
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Bug: Qwen3PromptFormatter is not imported in nemo/collections/common/prompts/__init__.py, so it won't be registered in PromptFormatter.resolve(). Users specifying prompt_format: qwen3 would get a resolution failure at runtime. The existing line 29 in __init__.py should be updated to:

from nemo.collections.common.prompts.qwen import QwenPromptFormatter, Qwen3PromptFormatter

Also, there are no tests for Qwen3PromptFormatter (unlike NemotronNanoV3PromptFormatter which has 312 lines of tests).

@claude
Copy link

claude bot commented Mar 19, 2026

Review Summary

Bug

  • Qwen3PromptFormatter not registered: The new class is defined in qwen.py but not imported in prompts/__init__.py, so PromptFormatter.resolve("qwen3") will fail at runtime.

Typos

  • "have have""have" in error messages across nemotron_nano_v3.py, qwen.py, and pre-existing in formatter.py
  • "lenght""length" in salm_automodel.py:213
  • "manfiest""manifest" in the tutorial notebook (2 occurrences)

Test Coverage

  • Qwen3PromptFormatter has zero tests despite ~180 lines of non-trivial logic (random /think//no_think injection, thinking block normalization). NemotronNanoV3PromptFormatter has 312 lines of tests by comparison.

Edresson
Edresson previously approved these changes Mar 20, 2026
Copy link
Collaborator

@Edresson Edresson left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

Signed-off-by: Piotr Żelasko <petezor@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

common core Changes to NeMo Core

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants