Is this a common/reasonable recipe for full finetuning Qwen3.5-4B?

[celsowm]

I’m about to run a full FT on Qwen/Qwen3.5-4B for a PT-BR legal assistant dataset and wanted a sanity check before I burn a bunch of GPU time.

This is not LoRA, just straight full finetuning.

Setup right now: model: Qwen/Qwen3.5-4B

data: chat dataset with a messages field
domain: Brazilian legal
max length: 1024
split: 95/5 random
epochs: 1
lr: 1e-5
wd: 0.1
warmup: 0.03
scheduler: cosine
batch size: 4
grad accum: 4
precision: bf16 if available, else fp16
grad checkpointing: on
packing: off
optimizer: adamw_torch_fused

What I’m doing is basically:

normalize messages

apply Qwen chat template
drop samples over max length
train with trl.SFTTrainer

Core training code is roughly:

from transformers import AutoModelForCausalLM, AutoTokenizer
from trl import SFTTrainer, SFTConfig
import torch

MODEL_NAME = "Qwen/Qwen3.5-4B"
MAX_LENGTH = 1024

tokenizer = AutoTokenizer.from_pretrained(MODEL_NAME, trust_remote_code=True)
if tokenizer.pad_token is None:
    tokenizer.pad_token = tokenizer.eos_token
tokenizer.padding_side = "right"

model = AutoModelForCausalLM.from_pretrained(
    MODEL_NAME,
    trust_remote_code=True,
    dtype=torch.bfloat16 if torch.cuda.is_bf16_supported() else torch.float16,
    low_cpu_mem_usage=True,
)

for p in model.parameters():
    p.requires_grad = True

model.config.use_cache = False

args = SFTConfig(
    output_dir="output",
    num_train_epochs=1,
    learning_rate=1e-5,
    weight_decay=0.1,
    warmup_ratio=0.03,
    lr_scheduler_type="cosine",
    per_device_train_batch_size=4,
    per_device_eval_batch_size=4,
    gradient_accumulation_steps=4,
    bf16=torch.cuda.is_bf16_supported(),
    fp16=not torch.cuda.is_bf16_supported(),
    tf32=True,
    gradient_checkpointing=True,
    packing=False,
    max_length=MAX_LENGTH,
    eval_strategy="steps",
    eval_steps=100,
    save_strategy="steps",
    save_steps=100,
    report_to="none",
    remove_unused_columns=False,
    eos_token=tokenizer.eos_token,
    pad_token=tokenizer.pad_token,
)

trainer = SFTTrainer(
    model=model,
    args=args,
    train_dataset=train_ds,
    eval_dataset=eval_ds,
    processing_class=tokenizer,
)
trainer.train()

Main thing I’m trying to figure out is: is this a common/reasonable recipe, or am I missing some Qwen-specific gotcha?

Stuff I’m unsure about:

should I be using Qwen/Qwen3.5-4B-Base instead of the post-trained one?

for Qwen chat data, is messages + SFTTrainer enough, or is there some masking/template detail that matters a lot?

would you train on the whole formatted conversation, or only assistant tokens?

do any of these hparams look obviously off for domain adaptation?

any known Qwen3.5 full FT traps?

Not looking for the “best possible” setup, mostly just trying to make sure this is a normal/sane way to do it.

Anyone here already fine-tuned Qwen3.5 and can say whether this looks reasonable?

[demirciberk]

Confirmed facts (from current TRL source/docs):

1. You are not masking the prompt/user/system tokens — this is the biggest gap.
   By default SFTTrainer computes loss over the entire formatted sequence when given a messages/conversational dataset with completion_only_loss not applicable (that flag only fires for prompt-completion datasets). To train on assistant turns only, set:
   python
   args = SFTConfig(
   ...,
   assistant_only_loss=True,
   )

   Per the docs, this requires the chat template to contain {% generation %}/{% endgeneration %} markers — TRL auto-patches the template for known families including Qwen3, so it should work out of the box for a Qwen3.5 tokenizer that follows the Qwen3 template convention. Source: docs/source/sft_trainer.md — "Train on assistant messages only", and the runtime check in trl/trainer/sft_trainer.py (has_generation_markers, get_training_chat_template, is_chat_template_stop_token_trained).
   Whether the exact Qwen3.5-4B template is in TRL's auto-patch list is not something I can confirm without inspecting the tokenizer's chat_template.jinja directly — check tokenizer.chat_template for {% generation %} yourself, or pass chat_template_path explicitly if it's missing.

2. Training on the full sequence (including user/system tokens) is a real, documented option, not just an oversight — completion_only_loss=False is the flag for it — but it's normally worse for instruction/chat SFT because the model also gets gradient signal for predicting the user's/system's tokens, which is not the objective you want. For domain adaptation on chat-formatted data, assistant_only_loss=True is the standard recipe.

3. packing=False + no masking of pad tokens: with padding on the right and DataCollatorForLanguageModeling-style collation, TRL/the tokenizer generally handles label masking of the pad tokens for you (labels for pad_token_id set to -100), but this is worth verifying by inspecting a batch (trainer.get_train_dataloader() and print labels) before a long run — don't assume, print and check.

4. Qwen/Qwen3.5-4B-Base vs. instruct-tuned Qwen/Qwen3.5-4B: For domain full-FT on a fairly large, well-curated chat/instruction dataset, starting from the instruct/post-trained checkpoint is the more common choice — you inherit instruction-following and safety behavior and just shift the domain distribution. Starting from -Base is typically reserved for cases where you're building your own full SFT/chat behavior from scratch or explicitly want to avoid whatever alignment/behavior baked into the post-trained model. This is an established community practice, not something documented in TRL itself, so treat it as a hypothesis to validate against your own eval set (e.g., a few held-out prompts checked qualitatively before committing to a full run).

5. Hyperparameters — nothing obviously broken for domain-adaptation full FT at 4B scale:

   • lr=1e-5, epochs=1, wd=0.1, warmup_ratio=0.03, cosine schedule are all within normal ranges for full-parameter SFT of a small (4B) dense model. Full FT at 4B commonly uses LRs in the 5e-6–2e-5 band, so 1e-5 is reasonable.
   • effective batch size = per_device(4) * grad_accum(4) = 16 (times num GPUs) — fine for 1 epoch domain-adaptation but on the low side if you have very little data; worth confirming your dataset size and epoch count together, since 1 epoch at bs=16 on a small dataset may underfit and a larger legal corpus may be fine.
   • max_length=1024 may truncate long legal documents/contracts — check the length distribution of your PT-BR legal dataset before committing; the drop samples over max length step you mention is a reasonable minimal handling but you should confirm the fraction being dropped isn't significant (e.g., log count_dropped / total).

6. Qwen-specific gotchas (from general Qwen3 family experience, not TRL-documented):

   • Qwen models frequently ship with /reasoning-mode template branches (enable_thinking) in the chat template for hybrid-reasoning variants. If Qwen3.5-4B's template supports a thinking mode, make sure your apply_chat_template call and the auto-patched training template agree on whether reasoning spans are included in the loss mask — check the rendered token IDs for a sample before training.
   • model.config.use_cache = False with gradient_checkpointing=True is correct and necessary — you already have this.
   • Confirm tokenizer.padding_side = "right" matches what the model expects for causal-LM training (right-padding + label masking is standard and correct here).

Recommended additions to your SFTConfig:
python
args = SFTConfig(
...,
assistant_only_loss=True, # mask user/system tokens out of the loss
max_length=MAX_LENGTH,
)

And before the full run, sanity-check masking on one batch:
python
batch = next(iter(trainer.get_train_dataloader()))
print(batch["labels"][0]) # confirm non-assistant / pad tokens are -100

Summary: the recipe is broadly reasonable for full FT at 4B scale, but the likely functional bug (not just a suboptimal choice) is that assistant_only_loss is not enabled, so you're currently training on user+system tokens too. Fix that first, verify the chat template has (or gets auto-patched with) {% generation %} markers for Qwen3.5, and inspect one batch's labels before committing GPU time to the full run. The Base-vs.-Instruct starting point question is a judgment call best resolved empirically on a small subset, not something with a single documented right answer.