Open
Description
We don't currently check all user specified paths for validity (i.e. if directories / checkpoints/ tokenizers actually exist) at the given paths. As a result (for example), we can initialize the model, load in model checkpoint, FSDP shard, etc and then raise an error that the tokenizer is invalid. Instead, we should a-priori check the tokenizer path for validity and save some dev time here.