I keep running into this error when training what I think is a small dataset but might not be (500 annotated texts, mean text size quite long, 3000 characters, 3 labels):
OutOfMemoryError: CUDA out of memory. Tried to allocate 384.00 MiB (GPU 0; 14.61 GiB total capacity; 13.70 GiB already allocated; 72.62 MiB free; 13.78 GiB reserved in total by PyTorch) If reserved memory is >> allocated memory try setting max_split_size_mb to avoid fragmentation. See documentation for Memory Management and PYTORCH_CUDA_ALLOC_CONF
I am using the code from the tutorial (https://pypi.org/project/AugmentedSocialScientist/) using Camembert instead of Bert and my own data, on a server with 2 GPU. The error suggests reducing the batch size (https://stackoverflow.com/questions/59129812/how-to-avoid-cuda-out-of-memory-in-pytorch) but I cannot find a way to do this.
I keep running into this error when training what I think is a small dataset but might not be (500 annotated texts, mean text size quite long, 3000 characters, 3 labels):
I am using the code from the tutorial (https://pypi.org/project/AugmentedSocialScientist/) using Camembert instead of Bert and my own data, on a server with 2 GPU. The error suggests reducing the batch size (https://stackoverflow.com/questions/59129812/how-to-avoid-cuda-out-of-memory-in-pytorch) but I cannot find a way to do this.