CUDA/Pytorch multiprocessing workaround and test fixes#1304

AI-WAIFU wants to merge 9 commits intomainEleutherAI/gpt-neox:mainfrom

all-pass-or-xfailEleutherAI/gpt-neox:all-pass-or-xfailCopy head branch name to clipboard