Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

preserve existing run_data and append to it when calibration is run #806

Draft
wants to merge 5 commits into
base: main
Choose a base branch
from
Draft

preserve existing run_data and append to it when calibration is run #806

wants to merge 5 commits into from

Conversation

rogthefrog
Copy link
Contributor

@rogthefrog rogthefrog requested a review from a team as a code owner January 18, 2025 03:27
@rogthefrog rogthefrog temporarily deployed to Scheduled Testing January 18, 2025 03:27 — with GitHub Actions Inactive
@rogthefrog rogthefrog temporarily deployed to Scheduled Testing January 18, 2025 03:27 — with GitHub Actions Inactive
@rogthefrog rogthefrog temporarily deployed to Scheduled Testing January 18, 2025 03:27 — with GitHub Actions Inactive
@github-actions GitHub Actions
Copy link

github-actions bot commented Jan 18, 2025 •
edited
Loading

MLCommons CLA bot All contributors have signed the MLCommons CLA ✍️ ✅

@rogthefrog rogthefrog temporarily deployed to Scheduled Testing January 18, 2025 04:39 — with GitHub Actions Inactive
@rogthefrog rogthefrog temporarily deployed to Scheduled Testing January 18, 2025 04:39 — with GitHub Actions Inactive
@rogthefrog rogthefrog temporarily deployed to Scheduled Testing January 18, 2025 04:39 — with GitHub Actions Inactive
@wpietri
Copy link
Contributor

wpietri commented Jan 18, 2025

I'm glad you're tackling this.

I was thinking that as part of cleaning this up, we would split the calibration into lumps matching a single reasonable run. E.g., that the reference_standards section would be broken up by 1.0-en_us-official, 1.0-fr_fr-practice, and so on. In which case you could attach a run_info block to each lump.

That could also be changed later, but as long as you're in this I thought I'd mention it.

@rogthefrog rogthefrog force-pushed the feat/append-metadata branch from 5a4b66c to ee1621d Compare January 21, 2025 18:56
@rogthefrog rogthefrog temporarily deployed to Scheduled Testing January 21, 2025 18:56 — with GitHub Actions Inactive
@rogthefrog rogthefrog temporarily deployed to Scheduled Testing January 21, 2025 18:56 — with GitHub Actions Inactive
@rogthefrog rogthefrog temporarily deployed to Scheduled Testing January 21, 2025 18:57 — with GitHub Actions Inactive
@rogthefrog rogthefrog force-pushed the feat/append-metadata branch from aa68cb5 to c6fd0e4 Compare January 23, 2025 02:29
@rogthefrog rogthefrog temporarily deployed to Scheduled Testing January 23, 2025 03:02 — with GitHub Actions Inactive
@rogthefrog rogthefrog temporarily deployed to Scheduled Testing January 23, 2025 03:02 — with GitHub Actions Inactive
@rogthefrog rogthefrog temporarily deployed to Scheduled Testing January 23, 2025 03:02 — with GitHub Actions Inactive
@rogthefrog rogthefrog force-pushed the feat/append-metadata branch from c2cb66b to 5526455 Compare January 23, 2025 21:30
@rogthefrog rogthefrog temporarily deployed to Scheduled Testing January 23, 2025 21:30 — with GitHub Actions Inactive
@rogthefrog rogthefrog temporarily deployed to Scheduled Testing January 23, 2025 21:30 — with GitHub Actions Inactive
@rogthefrog rogthefrog temporarily deployed to Scheduled Testing January 23, 2025 21:30 — with GitHub Actions Inactive
@rogthefrog rogthefrog marked this pull request as draft January 23, 2025 21:31
@rogthefrog
refactor: split calibration from displaying the current standard valu…
cdda449 
…es; add support for calibration using specific prompt sets and locales
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@rogthefrog @wpietri