Query related to Training in Urdu and Arabic

[Anas-KhanWP]

I have huge dataset for both Urdu and Arabic, around 1000 hours I am trying to train the model on both.

@SWivid, The only thing that I am concerned about is how will it differentiate between Urdu and Arabic? Both Vocabs are same but different pronunciation. How will the model distinguish between them?

@JarodMica, I am following your tutorial but you also chose to do 1 language only (Japanese) Can you help me out in this?

[SWivid]

@Anas-KhanWP hi, is the wording between them different? i mean, whether the organization of word is different though the characters are same.
if so, you can try bpe

[SWivid]

no big idea, haven't encountered that before

maybe try first accelerate config as mentioned in readme and also try train.py?

[Anas-KhanWP]

I Tried but its still going to use_libuv, maybe the reason is that I am on windows and for multi gpu processing nccl is used and nccl is not supported on windows?

[SWivid]

dunno
maybe try wsl2?

[Anas-KhanWP]

WSL2 didn't show any error, the training started perfectly fine. but still the time taken for it with 2 GPUs is way more then it was with single gpu

[Anas-KhanWP]

@lpscr Why batch_size_per_gpu is 3200 when total memory is 24GB (I explicitly passed the value as 24, just to check) but 1600 when Total Memory is 48GB? See images Below for reference. Or maybe for 2 GPUs, I need to change this value to 38400 * (Total_gpus)?

With 24GB:

With 48GB: