Transferring a file named xfer.py does not work (job goes on hold)

[brianaydemir]

The precise cause still needs to be determined as of this writing, but it is suspicious that this is occurring for a file whose name is the same as the executable.

[brianaydemir]

Transfer jobs also go on hold for files named logging.py, shutil.py, and (likely) any other file that matches a Python standard library module used by xfer.py.

[brianaydemir]

When we push a file, the job sandbox on the remote host is populated with the file being pushed. The executable being a Python script, it will attempt to load modules from the sandbox. This ends poorly if the file being pushed happens to have a name that looks like a Python module.

We need to ensure that the file that ends up in the job sandbox has an anodyne name, e.g., flattened_name.

If think we can use a pre-script on the DAG node to create a symlink from <working directory>/<flattened_name> to the file being pushed, rely on HTCondor to transfer the symlink as the regular file it points to, and then rename the file appropriately on the remote host. (The post-script would need to remove this symlink.)

For symmetry, we should probably do something similar for pulls.