[NPU] [BUGFIX] Test ascend memory consumption.py fix

[OrangeRedeng]

Motivation

Fix of #17994

Due to the inability to load the model from Huggingface, the test from #15904 PR does not work.

Modifications

Change huggingface path to local path on CI server

Accuracy Tests

shall be covered by ci

Benchmarking and Profiling

shall be covered by ci

Checklist

• Format your code according to the Format code with pre-commit.
• Add unit tests according to the Run and add unit tests.
• Update documentation according to Write documentations.
• Provide accuracy and speed benchmark results according to Test the accuracy and Benchmark the speed.
• Follow the SGLang code style guidance.

Review Process

1. Ping Merge Oncalls to start the PR flow. See the PR Merge Process.
2. Get approvals from CODEOWNERS and other reviewers.
3. Trigger CI tests with comments or contact authorized users to do so.
   • /tag-run-ci-label, /rerun-failed-ci, /tag-and-rerun-ci
4. After green CI and required approvals, ask Merge Oncalls to merge.

[gemini-code-assist]

Summary of Changes

Hello @OrangeRedeng, I'm Gemini Code Assist¹! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request focuses on optimizing memory consumption for Mixture-of-Experts (MoE) models running on Ascend NPUs. It achieves this by refining how MoE layer weights are processed and loaded, introducing lazy initialization for expert weights, and cleaning up outdated environment variable references in documentation. A new comprehensive unit test has been added to validate these memory improvements, ensuring the server's NPU memory footprint remains efficient.

Highlights

• NPU Memory Optimization for MoE Layers: Refactored the processing and format casting of weights for Mixture-of-Experts (MoE) layers on Ascend NPUs. This includes removing a redundant weight caching mechanism and streamlining the application of "npu_format_cast" to reduce memory overhead.
• Lazy Loading of MoE Expert Weights: Implemented lazy initialization for the "routed_experts_weights_of_layer" in the Qwen3 MoE model. This defers the computation and loading of expert weights until they are actually accessed, which can significantly improve the initial memory footprint during model loading.
• Documentation Cleanup: Removed the "ENABLE_MOE_NZ" and "ENABLE_ASCEND_MOE_NZ" environment variables from Ascend NPU example documentation, as they are no longer necessary or relevant for current configurations.
• New Memory Consumption Test: Added a new unit test ("test_ascend_memory_consumption.py") specifically designed to monitor and assert the memory consumption of the SGLang server when running a Qwen3 MoE model on Ascend NPUs. This test ensures that the implemented memory optimizations are effective and within acceptable limits.

🧠 New Feature in Public Preview: You can now enable Memory to help Gemini Code Assist learn from your team's feedback. This makes future code reviews more consistent and personalized to your project's style. Click here to enable Memory in your admin console.

Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point by creating a comment using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands on the current page.

Feature	Command	Description
Code Review	/gemini review	Performs a code review for the current pull request in its current state.
Pull Request Summary	/gemini summary	Provides a summary of the current pull request in its current state.
Comment	@gemini-code-assist	Responds in comments when explicitly tagged, both in pull request comments and review comments.
Help	/gemini help	Displays a list of available commands.

Customization

To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for Github and other Google products, sign up here.

You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension.

Footnotes

1. Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution. ↩

[gemini-code-assist]

Code Review

This pull request introduces several optimizations to reduce memory consumption on Ascend NPU devices, particularly for Mixture-of-Experts (MoE) models. The key changes include refactoring weight processing to avoid unnecessary tensor copies by removing .contiguous() calls, and using lazy initialization for expert weights to speed up model loading. Additionally, a new memory consumption test has been added to verify these improvements. The changes are well-implemented and contribute to better performance and efficiency. I have one minor suggestion for improving the clarity of a comment in the new test file.

I am having trouble creating individual review comments. Click here to see my feedback.

test/registered/ascend/test_ascend_memory_consumption.py (64)

This comment appears to be a copy-paste from above. To improve clarity, it should be updated to reflect that this block of code calculates the memory used by the server after startup.

        ### Calculate memory used by the server

[ping1jing2]

/tag-and-rerun-ci

[ping1jing2]

/rerun-failed-ci

[ping1jing2]

/rerun-failed-ci