Skip to content

Rectified logic for system prompt and audio trimming.#1298

Draft
ankitapasad wants to merge 8 commits into
NVIDIA-NeMo:vmendelev/2512_s2s_evalfrom
ankitapasad:vmendelev/2512_s2s_eval
Draft

Rectified logic for system prompt and audio trimming.#1298
ankitapasad wants to merge 8 commits into
NVIDIA-NeMo:vmendelev/2512_s2s_evalfrom
ankitapasad:vmendelev/2512_s2s_eval

Conversation

@ankitapasad
Copy link
Copy Markdown

  1. System prompt can now be overwritten from config. Previously, the config value was being ignored if a system prompt exists in the data manifest.
  2. For pause handling subsets pass --silence_padding_sec 0.0 to make sure no additional silence is added to the audio, leading to inflated ToR scores. Previously, silence_padding_sec would assume the default value of 5 seconds.
  3. Support for FLEURS subset ASR evaluation.

Signed-off-by: Ankita Pasad <apasad@nvidia.com>
Signed-off-by: Ankita Pasad <apasad@nvidia.com>
Signed-off-by: Ankita Pasad <apasad@nvidia.com>
Signed-off-by: Ankita Pasad <apasad@nvidia.com>
Signed-off-by: Ankita Pasad <apasad@nvidia.com>
if self.v2_config.pad_to_duration_secs is not None:
return float(self.v2_config.pad_to_duration_secs)
if self.v2_config.silence_padding_sec > 0:
if self.v2_config.silence_padding_sec >= 0:
Copy link
Copy Markdown
Member

@melllinia melllinia Mar 10, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Was the change from > 0 to >= 0 intentional? With >= 0, a value of 0 will return duration instead of None, but adding 0 doesn’t change the result.

def main():
parser = argparse.ArgumentParser(description="Prepare FLEURS data for nemo-skills ASR evaluation")
parser.add_argument(
"--manifest_path",
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Normally, we download the official benchmarks from HF, but since this is just an experimental implementation, I think reading from the manifest path is fine for now.

Signed-off-by: Ankita Pasad <apasad@nvidia.com>
Signed-off-by: Ankita Pasad <apasad@nvidia.com>
Signed-off-by: Ankita Pasad <apasad@nvidia.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants