(WIP) DeepseekV3 (and Multi-Head Latent Attention)

[ysjprojects]

follow up from #1945

More alignment with the huggingface implementation, allowing custom values for q_lora_rank, v_dim, etc.
Cleaned up some lines added for debugging purposes.

Also, the errors in #1945 was simply because pythia simply did not have the necessary parameters for MLA.

[Borda]

let's add a test :)

[ali-alshaar7]

Looks good overall. +1 on the test though, and would be nice if we could share more with the base attention implementation instead of duplicating things. Maybe for a followup. Thank you.

[Borda]

@t-vi @lantiga mind have look, pls :)

[ysjprojects]

@t-vi @lantiga @Borda

DeepseekV3 architecture WIP.

Should we move DeepseekV3MoE and MultiHeadLatentAttention out of model.py into its own file since the architecture is quite unique so it's unlikely that a future model would implement it again?