--dev

[HalaSalameh]

what is the use of --dev?
what does early stopping mean?

[himkt]

As you mentioned, the help comment says dev_file is used to do early stopping.
However, current implementation does not seem to stop training loop by watching metrics on dev data.

dev_file is used to decide whether deep-crf creates a snapshot or not for each epoch, I think.

deep-crf/deepcrf/main.py

Lines 484 to 494 in ff24ecc

	if prev_dev_f < dev_f:
	logging.info(' [update best model on dev set!]')
	dev_list = [prev_dev_f, dev_f]
	dev_str = ' ' + ' => '.join(map(str, dev_list))
	logging.info(dev_str)
	prev_dev_f = dev_f
	# Save model
	model_filename = save_name + '_epoch' + str(epoch)
	serializers.save_hdf5(model_filename + '.model', net)
	serializers.save_hdf5(model_filename + '.state', opt)

[HalaSalameh]

As you mentioned, the help comment says dev_file is used to do early stopping.
However, current implementation does not seem to stop training loop by watching metrics on dev data.

dev_file is used to decide whether deep-crf creates a snapshot or not for each epoch, I think.

deep-crf/deepcrf/main.py

Lines 484 to 494 in ff24ecc

	if prev_dev_f < dev_f:
	logging.info(' [update best model on dev set!]')
	dev_list = [prev_dev_f, dev_f]
	dev_str = ' ' + ' => '.join(map(str, dev_list))
	logging.info(dev_str)
	prev_dev_f = dev_f
	# Save model
	model_filename = save_name + '_epoch' + str(epoch)
	serializers.save_hdf5(model_filename + '.model', net)
	serializers.save_hdf5(model_filename + '.state', opt)

the code throws an error if there is no dev_file argument and i dont know what file i should give it so i just give it the same file as the input dataset