[BUG] data was not collected with "context_type=local"

[Weizhang88]

Summary
Output data was not collected from the temporary folder.
I was using dpgen to do some autotest work with my local computer. The context_type=local in the machine.json file. After successfully calculation, the oupt files were not collected back to the original input file contained dir.
context_type=lazylocal is working.

"batch_type": "shell", "context_type": "local", "local_root": "./", "remote_root": "xyz"

DPGEN Version and Platform
0.11.1 local computer

Job submission and computing cluster configuration

Expected Behavior

Actual Behavior

Steps to Reproduce

Further Information, Files, and Links

[njzjz]

Hi @Weizhang88, what task did you run? What file should be collected?

[Weizhang88]

Hi @Weizhang88, what task did you run? What file should be collected?

Sorry for the late reply. I just used dpgen run para.json machine.json to collect data sets for training on my own local computer.

Recently, I upgraded dpgen from version 0.10.6 to version 0.11.1 and made changes to the machine.json file to accommodate the new version. When I specified "context_type": "local" like before, a temporary directory was created where the calculations were processed. However, after the calculations were completed, the temporary directory was deleted and the output files (e.g. output from cp2k) were not collected backward to the original directory within the iter.000000 directory (as an example).

'context_type=lazylocal' is working and does not create a temporary directory. dp or lmp were executed in the original directory. However, when I stopped the dpgen run with ctrl + c using this setting, the dp, lmp, or cp2k processes were still running in the terminal shell. I had to use the command kill $(pgrep cp2k) to terminate them. This behavior was not observed with "context_type": "local", as the dp or lmp processes were terminated simultaneously with the dpgen run using ctrl + c.

Therefore, I prefer to use "context_type": "local" as it allows for easier control of the process.

[njzjz]

Do you use the latest dpdispatcher? Could you provide the structure of directories?

I checked LocalContext and LazyLocalContext. The codes to execute commands are the same, so there should be no difference.