Skip to content

WA for accuracy issues in qwen3-coder_int4 #3947

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft
dtrawins wants to merge 6 commits into
base: main
Choose a base branch
from
+9 −0
Draft

WA for accuracy issues in qwen3-coder_int4 #3947

Changes from all commits
Commits
Show all changes
6 commits
Select commit Hold shift + click to select a range
3a41d63
WA for accuracy issues in qwen3-coder_int4
dtrawins Feb 3, 2026
b084aca
WA for accuracy issues in qwen3-coder_int4
dtrawins Feb 4, 2026
305ecb0
fix
dtrawins Feb 4, 2026
556d925
fix
dtrawins Feb 4, 2026
41e7d41
fix
dtrawins Feb 4, 2026
07e4167
fix
dtrawins Feb 4, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 9 additions & 0 deletions src/server.cpp
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -68,6 +68,7 @@
#include "servables_config_manager_module/servablesconfigmanagermodule.hpp"
#include "stringutils.hpp"
#include "version.hpp"
#include "src/utils/env_guard.hpp"

#if (PYTHON_DISABLE == 0)
#include "python/pythoninterpretermodule.hpp"
Expand Down Expand Up @@ -516,6 +517,14 @@ int Server::startServerFromSettings(ServerSettingsImpl& serverSettings, ModelsSe

// OVMS Start
int Server::start(int argc, char** argv) {
// Set default for MOE_USE_MICRO_GEMM_PREFILL if not set
// This is a workaround for OpenVINO issue where prefill causes accuracy problems in long context in qwen3-coder model
const char* moeEnv = std::getenv("MOE_USE_MICRO_GEMM_PREFILL");
if (moeEnv == nullptr){
std::unique_ptr<EnvGuard> envGuard = std::make_unique<EnvGuard>();
envGuard->set("MOE_USE_MICRO_GEMM_PREFILL", "0");
}

auto paramsOrExit = parseArgs(argc, argv);
// Check for error in parsing
if (std::holds_alternative<std::pair<int, std::string>>(paramsOrExit)) {
Expand Down