CT scans consistently produce X-ray report format instead of CT-specific findings

[Chunsik-gut]

Model: google/medgemma-1.5-4b-it and google/medgemma-1.5-27b-it

Description:
When providing CT DICOM images as input, MedGemma 1.5 consistently outputs a X-ray report template instead of CT-specific findings, regardless of body part, prompt engineering, or image preprocessing.

Example output:
FINDINGS: The lungs are clear without focal consolidation. No pleural effusion
or pneumothorax is seen. The cardiomediastinal silhouette is normal.
IMPRESSION: No acute cardiopulmonary process.

This is a standard X-ray report format, not a CT radiology report.

Reproduction conditions:
Modality: CT
BodyPartExamined: CHEST, ABDOMEN, HEAD (all tested)
Preprocessing: Both standard windowing and the 3-channel windowing method from the official high_dimensional_ct_hugging_face.ipynb notebook were tested

What was tried:
Explicit prompt instructions: "This is a CT scan. Do NOT use X-ray report format."
3-channel windowing as described in the official CT notebook [(-1024, 1024), (-135, 215), (0, 80)]
Both 4b and 27b models produce identical output

Expected behavior:
CT-specific findings appropriate for the scanned body part.

[LakshmiKalaKadali]

Hi @Chunsik-gut,

We appreciate your detailed observations. We have run the official notebook and it produces correct CT-specific output. However, based on what you have already tried, official 3-channel windowing [(-1024, 1024), (-135, 215), (0, 80)], explicit prompt instructions, multiple body parts, and both model sizes, we would like to investigate this further on our end.
Could you share your reproducible code with the following details:

1. Exact message construction code
2. Explicit Instruction prompts and query text
3. Number of slices you are passing(MAX_SLICE)
4. Your model loading code.

Thank You

[Chunsik-gut]

@LakshmiKalaKadali
Thank you for your response. Here are the details you requested:

1. Exact message construction code

def analyze_batch(pipe, batch_files, batch_index, total_batches, prompt, system_text):
images, first_meta = [], {}
for i, f in enumerate(batch_files):
f.seek(0)
img, meta = load_image(f)
images.append((f.name, img))
if i == 0:
first_meta = meta
f.seek(0)

model_input = [{"type": "text", "text": f"[묶음 {batch_index}/{total_batches}] {prompt}"}]
if first_meta and batch_index == 1:
    meta_text = " / ".join([f"{k}: {v}" for k, v in first_meta.items()])
    model_input.append({"type": "text", "text": f"메타데이터: {meta_text}"})

for fname, img in images:
    model_input.append({"type": "text", "text": f"[{fname}]"})
    model_input.append({"type": "image", "image": img})

messages = [
    {"role": "system", "content": [{"type": "text", "text": system_text}]},
    {"role": "user",   "content": model_input}
]

output = pipe(text=messages, max_new_tokens=512,
    eos_token_id=pipe.tokenizer.eos_token_id,
    pad_token_id=pipe.tokenizer.pad_token_id)
return parse_response(output[0]["generated_text"])

2. Explicit instruction prompts and query text

System prompt:
당신은 한국 CT 영상의학과 전문의입니다. 반드시 한국어로만 답변하세요.
User prompt (wrapped_prompt):
Please analyze the uploaded medical images. User's question (in Korean): {prompt}
Respond in Korean.

3. Number of slices (MAX_SLICE)

Batch size: 85 slices per batch
Total slices: 700~800 slices split into multiple batches of 85
Tested with as few as 12 slices - same X-ray report format output

4. Model loading code

from transformers import pipeline

MODEL_ID = "google/medgemma-1.5-4b-it"

pipe = pipeline(
"image-text-to-text",
model=MODEL_ID,
device_map="auto",
token=hf_token,
)

Additional notes:

Also tested with google/medgemma-27b-multimodal-it (MedGemma 1) — identical X-ray report format output
Tested with both Korean and English user prompts
Tested with official 3-channel windowing [(-1024, 1024), (-135, 215), (0, 80)]
Issue occurs regardless of body part (CHEST, ABDOMEN, HEAD all tested)
Single slice input also produces X-ray report format