Difficulty in Reproducing Reported Qwen3-8B Results.

[SaiprasadKompally]

Hi authors,

First of all, thank you for open-sourcing this work. The repository has been very helpful for understanding and experimenting with latent collaboration.

While trying to reproduce some of the reported Qwen3-8B results, I noticed a significant difference between the results reported in the paper and what I am observing using the released code.

As a simple check, I started with ARC-Easy and ran the baseline setting. I obtained an accuracy of 87.45%, whereas the paper reports 95.6% for Qwen3-8B.

I also experimented with LatentMAS in the sequential setting. As I increased the latent steps from 0 to 10, the performance dropped considerably instead of improving.

On GSM8K, I obtained:

• Baseline: 76%
• LatentMAS Sequential (10 latent steps): 41%

For these experiments, I used the default repository configuration, including the default batch size of 20.

Since the baseline itself is lower than the reported result, and LatentMAS performance decreases substantially with more latent steps, I wanted to check whether I might be missing some important configuration details.

A few questions:

1. What exact settings were used to generate the results reported in Table 1?
2. How many latent steps and what batch size were used for the reported Qwen3-8B experiments? Were there any model-specific configurations?
3. Was latent_space_realign enabled when generating the reported results?

I would appreciate any clarification on the experimental setup. I would like to understand what configuration was used to obtain the reported numbers and whether there are any additional settings required for reproduction.

Thanks..!

[wangsunyan777]

I noticed that Qwen3-8B is easy to fall into repeating under the official temperature setting, and the scores I obtained on AIME 25 is significantly lower than what the paper reports. So it seems to be a common failure.