Skip to content

Conversation

@meenakshiramanathan1
Copy link
Contributor

@meenakshiramanathan1 meenakshiramanathan1 commented Aug 6, 2025

Model card: https://huggingface.co/blog/welcome-openai-gpt-oss
Model requires transformer version upgrade of 4.55.0 , post which it fails with RuntimeError: Using MXFP4 quantized models requires a GPU. The model uses MXFP4 quantization, which requires a GPU to run. But since we’re planning to use the model in BF16 or FP32 and don’t actually need quantization, quantization_config has been removed from model config before loading model.

After this change, the model loads without the MXFP4 error, but we encounter a warning: Some weights of GptOssForCausalLM were not initialized from the model checkpoint at openai/gpt-oss-20b and are newly initialized, which suggests that the model weights are not being loaded correctly—likely due to the architecture/config mismatch caused by altering the original configuration. This test is intended to verify whether the model's architecture is compatible with Forge compilation, and to determine how far the compilation process progresses.

@meenakshiramanathan1 meenakshiramanathan1 force-pushed the mramanathan/openai_gpt_oss branch from 27747fb to 297af22 Compare August 6, 2025 13:18
@codecov-commenter
Copy link

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 50.36%. Comparing base (3486037) to head (297af22).
⚠️ Report is 1 commits behind head on main.

Additional details and impacted files
@@           Coverage Diff           @@
##             main    #2749   +/-   ##
=======================================
  Coverage   50.36%   50.36%           
=======================================
  Files         171      171           
  Lines       13042    13042           
=======================================
  Hits         6569     6569           
  Misses       6473     6473           

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Model Bringup] - openai/gpt-oss-20b

3 participants