About Public Available Data

[mattxsong]

Hello,

Thank you for sharing your excellent work with the community. I am currently trying to understand and reproduce parts of your research, and I have a question regarding the use of public datasets such as MIMIC-CXR.

After evaluating both the -pt (pretrained) and -it (instruction-tuned) versions of the model, I noticed significant differences in output formatting and structure. Therefore, could you please clarify whether any preprocessing or cleaning steps were applied to the textual part of the dataset (e.g., reports in MIMIC-CXR) before training or evaluation? In particular, I'm interested in knowing if any modifications were made to align the text format with the expected input of the base model, or to ensure stable performance during inference.

Also, if possible, could you please share the official prompts/templates used during it training and evaluation?

This information would be very helpful for reproducing your results accurately.

Thank you again for your time and contribution!

Best!

[xk-huang]

Same here. My custom eval results are totally different from those in the tech report, and 27b performs worse than 4b.