Error in network training

[caisy417]

When I try to execute train.py, I met this error.

-------YOUR SETTINGS_________
verbose: True
model: VRES
train_set: train
fine_tune: False
num_epochs: 50
batch_size: 100
learning_rate: 0.001
scale: 3

Contructing dataset...
Training...
Traceback (most recent call last):
File "train.py", line 76, in
main()
File "train.py", line 74, in main
solver.train(train_dataset)
File "C:\Users\csy\Documents\video-super-resolution-master\solver.py", line 233, in train
self._epoch_step(train_dataset, epoch)
File "C:\Users\csy\Documents\video-super-resolution-master\solver.py", line 101, in _epoch_step
running_loss += loss.data[0]
IndexError: invalid index of a 0-dim tensor. Use tensor.item() to convert a 0-dim tensor to a Python number

[thangvubk]

I guess you are using new PyTorch version. As the error says, you may change loss.data[0] to loss.item() to fix the problem.

[caisy417]

Yes, the problem is solved! Thank you!
But other errors appeared...

-------YOUR SETTINGS_________
model: VRES
learning_rate: 0.001
verbose: True
train_set: train
batch_size: 100
num_epochs: 50
fine_tune: False
scale: 3

Contructing dataset...
Training...
C:\Users\csy\Documents\video-super-resolution-master\solver.py:105: UserWarning: torch.nn.utils.clip_grad_norm is now deprecated in favor of torch.nn.utils.clip_grad_norm_.
nn.utils.clip_grad_norm(self.model.parameters(), 0.4)
100% (622 of 622) |###############################################################| Elapsed Time: 0:12:48 ETA: 0:00:00Epoch 0, loss 0.01080
Computing PSNR...
Traceback (most recent call last):
File "train.py", line 76, in
main()
File "train.py", line 74, in main
solver.train(train_dataset)
File "C:\Users\csy\Documents\video-super-resolution-master\solver.py", line 240, in train
train_psnr, train_ssim, _, _ = self._check_PSNR(train_dataset)
File "C:\Users\csy\Documents\video-super-resolution-master\solver.py", line 163, in _check_PSNR
for batch, (input_batch, label_batch) in enumerate(dataloader):
File "C:\Users\csy\Anaconda3\envs\video\lib\site-packages\torch\utils\data\dataloader.py", line 193, in iter
return _DataLoaderIter(self)
File "C:\Users\csy\Anaconda3\envs\video\lib\site-packages\torch\utils\data\dataloader.py", line 469, in init
w.start()
File "C:\Users\csy\Anaconda3\envs\video\lib\multiprocessing\process.py", line 105, in start
self._popen = self._Popen(self)
File "C:\Users\csy\Anaconda3\envs\video\lib\multiprocessing\context.py", line 212, in _Popen
return _default_context.get_context().Process._Popen(process_obj)
File "C:\Users\csy\Anaconda3\envs\video\lib\multiprocessing\context.py", line 313, in _Popen
return Popen(process_obj)
File "C:\Users\csy\Anaconda3\envs\video\lib\multiprocessing\popen_spawn_win32.py", line 66, in init
reduction.dump(process_obj, to_child)
File "C:\Users\csy\Anaconda3\envs\video\lib\multiprocessing\reduction.py", line 59, in dump
ForkingPickler(file, protocol).dump(obj)
OSError: [Errno 22] Invalid argument

(video) C:\Users\csy\Documents\video-super-resolution-master>Traceback (most recent call last):
File "", line 1, in
File "C:\Users\csy\Anaconda3\envs\video\lib\multiprocessing\spawn.py", line 106, in spawn_main
exitcode = _main(fd)
File "C:\Users\csy\Anaconda3\envs\video\lib\multiprocessing\spawn.py", line 116, in _main
self = pickle.load(from_parent)
EOFError: Ran out of input

[thangvubk]

I will check it again and let u know

[LINSEVENQI]

I meet the problem also.Have you solved this problem? Can you tell me?

[LINSEVENQI]

Traceback (most recent call last):
File "D:/LQwork/video-super-resolution-master-oral/video-super-resolution-master/train.py", line 78, in
main()
File "D:/LQwork/video-super-resolution-master-oral/video-super-resolution-master/train.py", line 76, in main
solver.train(train_dataset)
File "D:\LQwork\video-super-resolution-master-oral\video-super-resolution-master\solver.py", line 242, in train
self._epoch_step(train_dataset, epoch)
File "D:\LQwork\video-super-resolution-master-oral\video-super-resolution-master\solver.py", line 89, in _epoch_step
for i, (input_batch, label_batch) in enumerate(dataloader):
File "D:\Anaconda\envs\pytorch\lib\site-packages\torch\utils\data\dataloader.py", line 193, in iter
return _DataLoaderIter(self)
File "D:\Anaconda\envs\pytorch\lib\site-packages\torch\utils\data\dataloader.py", line 469, in init
w.start()
File "D:\Anaconda\envs\pytorch\lib\multiprocessing\process.py", line 112, in start
self._popen = self._Popen(self)
File "D:\Anaconda\envs\pytorch\lib\multiprocessing\context.py", line 223, in _Popen
return _default_context.get_context().Process._Popen(process_obj)
File "D:\Anaconda\envs\pytorch\lib\multiprocessing\context.py", line 322, in _Popen
return Popen(process_obj)
File "D:\Anaconda\envs\pytorch\lib\multiprocessing\popen_spawn_win32.py", line 89, in init
reduction.dump(process_obj, to_child)
File "D:\Anaconda\envs\pytorch\lib\multiprocessing\reduction.py", line 60, in dump
ForkingPickler(file, protocol).dump(obj)
OSError: [Errno 22] Invalid argument
Traceback (most recent call last):
File "", line 1, in
File "D:\Anaconda\envs\pytorch\lib\multiprocessing\spawn.py", line 105, in spawn_main
exitcode = _main(fd)
File "D:\Anaconda\envs\pytorch\lib\multiprocessing\spawn.py", line 115, in _main
self = reduction.pickle.load(from_parent)
_pickle.UnpicklingError: pickle data was truncated

Process finished with exit code 1

[thangvubk]

I have run the code sucessfully. Are you using window?

[caisy417]

I have run the code sucessfully. Are you using window?

yes

[thangvubk]

I dont have window server to test the code. You may try to tun the code on Linux or figure out to handle the problem based on error mesage.

[caisy417]

I dont have window server to test the code. You may try to tun the code on Linux or figure out to handle the problem based on error mesage.

Thank you. I will try it out.

[LINSEVENQI]

Yes, the problem is solved! Thank you!
But other errors appeared...

-------YOUR SETTINGS_________
model: VRES
learning_rate: 0.001
verbose: True
train_set: train
batch_size: 100
num_epochs: 50
fine_tune: False
scale: 3

Contructing dataset...
Training...
C:\Users\csy\Documents\video-super-resolution-master\solver.py:105: UserWarning: torch.nn.utils.clip_grad_norm is now deprecated in favor of torch.nn.utils.clip_grad_norm_.
nn.utils.clip_grad_norm(self.model.parameters(), 0.4)
100% (622 of 622) |###############################################################| Elapsed Time: 0:12:48 ETA: 0:00:00Epoch 0, loss 0.01080
Computing PSNR...
Traceback (most recent call last):
File "train.py", line 76, in
main()
File "train.py", line 74, in main
solver.train(train_dataset)
File "C:\Users\csy\Documents\video-super-resolution-master\solver.py", line 240, in train
train_psnr, train_ssim, _, _ = self._check_PSNR(train_dataset)
File "C:\Users\csy\Documents\video-super-resolution-master\solver.py", line 163, in _check_PSNR
for batch, (input_batch, label_batch) in enumerate(dataloader):
File "C:\Users\csy\Anaconda3\envs\video\lib\site-packages\torch\utils\data\dataloader.py", line 193, in iter
return _DataLoaderIter(self)
File "C:\Users\csy\Anaconda3\envs\video\lib\site-packages\torch\utils\data\dataloader.py", line 469, in init
w.start()
File "C:\Users\csy\Anaconda3\envs\video\lib\multiprocessing\process.py", line 105, in start
self._popen = self._Popen(self)
File "C:\Users\csy\Anaconda3\envs\video\lib\multiprocessing\context.py", line 212, in _Popen
return _default_context.get_context().Process._Popen(process_obj)
File "C:\Users\csy\Anaconda3\envs\video\lib\multiprocessing\context.py", line 313, in _Popen
return Popen(process_obj)
File "C:\Users\csy\Anaconda3\envs\video\lib\multiprocessing\popen_spawn_win32.py", line 66, in init
reduction.dump(process_obj, to_child)
File "C:\Users\csy\Anaconda3\envs\video\lib\multiprocessing\reduction.py", line 59, in dump
ForkingPickler(file, protocol).dump(obj)
OSError: [Errno 22] Invalid argument

(video) C:\Users\csy\Documents\video-super-resolution-master>Traceback (most recent call last):
File "", line 1, in
File "C:\Users\csy\Anaconda3\envs\video\lib\multiprocessing\spawn.py", line 106, in spawn_main
exitcode = _main(fd)
File "C:\Users\csy\Anaconda3\envs\video\lib\multiprocessing\spawn.py", line 116, in _main
self = pickle.load(from_parent)
EOFError: Ran out of input
Can you tell me if you solve this problem? I have not found a solution.Thanks!

[LINSEVENQI]

I dont have window server to test the code. You may try to tun the code on Linux or figure out to handle the problem based on error mesage.

when I set the num_workers=0,this problem is salved ! However, I still have a question, how to use your code to save the best training model, because I see the prompt "best_model_state" is not used, and how to set "fine_tune" and "verbose" to True,I tried a few times but still showing False, Thank you very much!

[thangvubk]

I wll revise it soon.

[LINSEVENQI]

I wll revise it soon.

capture best model

    best_val_psnr = -1
    best_model_state = self.model.state_dict()

capture running PSNR on train and val dataset

    train_psnr, train_ssim, _, _ = self._check_PSNR(train_dataset)
    self.hist_train_psnr.append(train_psnr)

When I read these code I have some questions,I don't understand how to capture best model and I don't know where use the val dataset.I am very embarrassed to bother you many times, but I really want to know the answers to these questions.Thank you very much!!!!!!!!!!!!!!!!!

[thangvubk]

Saving model with best val PSNR is supported from 16dfac2

[thangvubk]

when I set the num_workers=0,this problem is salved ! However, I still have a question, how to use your code to save the best training model, because I see the prompt "best_model_state" is not used, and how to set "fine_tune" and "verbose" to True,I tried a few times but still showing False, Thank you very much!

fine_tune and verbose are boolean arguments. If they are omitted, their values are false. If they are presented (just simply --verbose), their values are true.