Error fine tuning "Phi 4": Phi3ForCausalLM.__init__() got an unexpected keyword argument 'use_flash_attention_2'

[isabelcabezasm]

Technical Feedback

I've tried to fine tune models Phi-4 model.

Finalizing the training, the process fails with this error: "Phi3ForCausalLM.__init__() got an unexpected keyword argument 'use_flash_attention_2'"

Encountered an internal ACFT error. Error Message/Code: Phi3ForCausalLM.__init__() got an unexpected keyword argument 'use_flash_attention_2'. Traceback:   File "swallow_all_exceptions_decorator.py", line 70, in wrapper
    return func(*args, **kwargs)
  File "model_converter.py", line 227, in main
    model_converter(args)
  File "model_converter.py", line 167, in model_converter
    model_converter.convert_model()
  File "model_converter_adapters.py", line 271, in convert_model
    model, tokenizer = self.load_model_and_tokenizer()
  File "model_converter_adapters.py", line 223, in load_model_and_tokenizer
    model = load_model(
  File "model_converter_utils.py", line 99, in load_model
    model = load_and_merge_peft_lora_model(model_path, component_args, ft_config)
  File "model_converter_utils.py", line 198, in load_and_merge_peft_lora_model
    ds3_peft_model = auto_cls.from_pretrained(
  File "auto.py", line 115, in from_pretrained
    base_model = target_class.from_pretrained(base_model_path, revision=base_model_revision, **kwargs)
  File "auto_factory.py", line 600, in from_pretrained
    return model_class.from_pretrained(
  File "modeling_utils.py", line 316, in _wrapper
    return func(*args, **kwargs)
  File "modeling_utils.py", line 4986, in from_pretrained
    model = cls(config, *model_args, **model_kwargs)
. Additional information: Phi3ForCausalLM.__init__() got an unexpected keyword argument 'use_flash_attention_2'.

Training completed on
Sep 17, 2025 1:30 PM
Duration
47m 19.52s
Training file
train_jsonl_2025-09-17_083311_UTC
Base model
Phi-4:v8
Virtual Machine
Standard_NC80adis_H100_v5

Task parameters
Batch size multiplier
1
Epochs
1
Learning rate
0.0003

Desired Outcome

Describe the desired outcome:

Complete the training sucessfully.

Current Workaround

Describe the current workaround.

We didn't find any workaround.

[kinfey]

Can you confirm your transformers version ?