Skip to content

[Model] Add Qwen3 and allow switching between thinking and non-thinking mode #75

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 5 commits into from
May 5, 2025

Conversation

CharlieFRuan
Copy link
Contributor

@CharlieFRuan CharlieFRuan commented May 5, 2025

This PR adds Qwen3 models from WebLLM 0.2.79.

We also add a config enable_thinking that can be toggled in the settings page to decide whether the Qwen3 model should reason or not. By default, it is turned off. Turning it on will cause it to reason and will also change its temperature and top_p, following https://huggingface.co/Qwen/Qwen3-0.6B#best-practices

In addition, when we want to summarize/create a topic, we should never reason, as it would take way too long.

In addition, for multi-round chat, we remove the <think>thinking tokens here</think>\n\n from the history assistant's message.

@CharlieFRuan
Copy link
Contributor Author

CharlieFRuan commented May 5, 2025

@Neet-Nestor Hi Nestor! Could you review this PR when you get a chance? Currently, changing the setting page does not seem to be very robust. Specifically, I added an enableThinking checkbox, which when checked, we should see reasoning behavior in Qwen3 models. This only rarely works as expected for now. Thanks!

Edit: should be fixed now

@CharlieFRuan
Copy link
Contributor Author

Never mind, the settings toggling should work now. There was a bug in isDifferentConfig(): config[field] !== config[field] should be this.llmConfig[field] !== config[field], as fixed in the latest commit.

@Neet-Nestor
Copy link
Collaborator

Neet-Nestor commented May 5, 2025

@CharlieFRuan I made several changes:

  1. Put the thinking toggle button in model setting instead of global settings
  2. Add another thinking toggle button in chat actions toolbar
  3. Fix the issue of <think> </think> tags in chat titles
  4. Update store version number in config.ts

I have tested and verified these changes by Qwen-0.6b. In the first screenshot the thinking toggle is on while in the second it's off.

Screenshot 2025-05-05 at 1 08 57 AM Screenshot 2025-05-05 at 1 08 48 AM

@Neet-Nestor
Copy link
Collaborator

I made another change to remove the empty <think> </think> tags in non-thinking mode.

Screenshot 2025-05-05 at 1 17 55 AM

@Neet-Nestor Neet-Nestor merged commit ac629bc into mlc-ai:main May 5, 2025
@CharlieFRuan
Copy link
Contributor Author

@Neet-Nestor Thank you so much! These are great changes!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants