Add new export LLM config #11028

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Open

jackzhxng wants to merge 13 commits into gh/jackzhxng/10/base from gh/jackzhxng/10/head

+353 −0

Contributor

jackzhxng commented May 21, 2025 •

edited

Loading

Stack from ghstack (oldest at bottom):

Differential Revision: D75263991


          Add new export LLM config

aa54351

[ghstack-poisoned]

jackzhxng requested a review from lucylq as a code owner

May 21, 2025 00:43

pytorch-bot bot commented May 21, 2025 •

edited

Loading

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/11028

📄 Preview Python docs built from this PR

Note: Links to docs will display an error until the docs builds have been completed.

❗ 1 Active SEVs

There are 1 currently active SEVs. If your PR is affected, please view them below:

Experiencing "429: Too Many Requests" on downloading actions

✅ You can merge normally! (1 Unrelated Failure)

As of commit f769d50 with merge base d5c4ba7 ():

FLAKY - The following job failed but was likely due to flakiness present on trunk:

pull / test-models-linux (phi_4_mini, portable, linux.4xlarge.memory) / linux-job (gh) (matched linux rule in flaky-rules.json)
The process '/usr/bin/git' failed with exit code 128

This comment was automatically generated by Dr. CI and updates every 15 minutes.

jackzhxng mentioned this pull request

Introduce hydra framework with backwards compatibility #11029

Open

facebook-github-bot added the CLA Signed label

jackzhxng added 3 commits

May 22, 2025 13:24


          Update on "Add new export LLM config"

48c8a19

[ghstack-poisoned]


          Update on "Add new export LLM config"

ca9474c

[ghstack-poisoned]


          Update on "Add new export LLM config"

983ff6d

[ghstack-poisoned]

This was referenced May 22, 2025

Convert args to LlmConfig #11081

Open

Use llm_config instead of args in export_llama functions #11084

Closed

jackzhxng added 2 commits

May 22, 2025 17:39


          Update on "Add new export LLM config"

8124a32

[ghstack-poisoned]


          Update on "Add new export LLM config"

c69b158

[ghstack-poisoned]

jackzhxng added release notes: api release notes: llm labels

Contributor Author

jackzhxng commented May 23, 2025

@jackzhxng has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator.

1 similar comment

Contributor Author

jackzhxng commented May 27, 2025

@jackzhxng has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator.

jackzhxng mentioned this pull request

Use llm_config instead of args in export_llama functions #11162

Open

Contributor Author

jackzhxng commented May 27, 2025

@jackzhxng has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator.


          Update on "Add new export LLM config"

5f4c78d

Differential Revision: [D75263991](https://our.internmc.facebook.com/intern/diff/D75263991)

[ghstack-poisoned]

Contributor Author

jackzhxng commented May 27, 2025

@jackzhxng has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator.

This was referenced May 27, 2025

Fix test_llama #11165

Closed

refactor: Replace self.args with LlmConfig in model.py and export_llama_lib.py #11166

Closed

fix: add missing LlmConfig import and parameter in export_llama_lib.py #11167

Closed

refactor: Use LlmConfig for model parameters instead of kwargs #11168

Closed

refactor: remove backward compatibility and simplify Llama2Model configuration #11169

Closed

refactor: simplify Llama2Model constructor to take llm_config directly #11170

Closed

Completely remove args from export_llama_lib #11171

Closed

Contributor Author

jackzhxng commented May 27, 2025

@jackzhxng has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator.

1 similar comment

Contributor Author

jackzhxng commented May 28, 2025

@jackzhxng has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator.


          Update on "Add new export LLM config"

e630be8

Differential Revision: [D75263991](https://our.internmc.facebook.com/intern/diff/D75263991)

[ghstack-poisoned]

Contributor Author

jackzhxng commented May 28, 2025

@jackzhxng has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator.

jackzhxng mentioned this pull request

Reduce memory requirement on export_llama tests with no params #11175

Closed

Contributor Author

jackzhxng commented May 28, 2025

@jackzhxng has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator.


          Update on "Add new export LLM config"

8a4f08b

Differential Revision: [D75263991](https://our.internmc.facebook.com/intern/diff/D75263991)

[ghstack-poisoned]

Contributor

facebook-github-bot commented May 29, 2025

This pull request was exported from Phabricator. Differential Revision: D75263991

facebook-github-bot added the fb-exported label


          Update on "Add new export LLM config"

a2c19be

Differential Revision: [D75263991](https://our.internmc.facebook.com/intern/diff/D75263991)

[ghstack-poisoned]

Contributor

facebook-github-bot commented Jun 2, 2025

This pull request was exported from Phabricator. Differential Revision: D75263991


          Update on "Add new export LLM config"

caaee0e

Differential Revision: [D75263991](https://our.internmc.facebook.com/intern/diff/D75263991)

[ghstack-poisoned]

Contributor

facebook-github-bot commented Jun 3, 2025

This pull request was exported from Phabricator. Differential Revision: D75263991


          Update on "Add new export LLM config"

ed0dd5a

Differential Revision: [D75263991](https://our.internmc.facebook.com/intern/diff/D75263991)

[ghstack-poisoned]

lucylq reviewed

View reviewed changes

examples/models/llama/config/llm_config.py Outdated Show resolved Hide resolved

examples/models/llama/config/llm_config.py Show resolved Hide resolved

examples/models/llama/config/llm_config.py Show resolved Hide resolved

examples/models/llama/config/llm_config.py

    
              @dataclass

              class BackendConfig:

                  xnnpack: XNNPackConfig = field(default_factory=XNNPackConfig)

Contributor

lucylq Jun 3, 2025

Wdyt about making these optional configs, set to None?

I think that could make it easier to tell when a backend is enabled or not, instead of looking through each backend config to check if enabled=False.

Contributor Author

jackzhxng Jun 4, 2025

Mind if I follow up with this change? I would like to keep the changes in this diff as small as possible (ie as close to the original args as possible) to make it easier to pass all the internal CI

Contributor

lucylq Jun 4, 2025

Sure, sounds good.

examples/models/llama/config/llm_config.py Show resolved Hide resolved


          Update on "Add new export LLM config"

f769d50

Differential Revision: [D75263991](https://our.internmc.facebook.com/intern/diff/D75263991)

[ghstack-poisoned]

Contributor

facebook-github-bot commented Jun 4, 2025

This pull request was exported from Phabricator. Differential Revision: D75263991

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

CLA Signed fb-exported release notes: api release notes: llm