Description
Describe the bug
Glideinwms factory 3.10.1 and greater (possibly before that too) set an IDTOKENS_FILE variable
which appears to be pointing to the local factory location of the idtoken and not the destination to which
it ought to be transferred. This IDTOKENS_FILE variable is propagated to the job although it does not appear
to actually be used for anything there. It appears to be a consequence of
these lines in a job.condor file as produced by a config of glideinwms 3.10.1/ condor 10.0.1
[root@hepcintfactory01 entry_CMSHTPC_T3_US_TACC_FRONTERA]# cat /var/lib/gwms-factory/work-dir/entry_CMSHTPC_T3_US_Bridges2/job.condor
File: job.condor
environment = "IDTOKENS_FILE=$ENV(IDTOKENS_FILE:) JOB_TOKENS='/var/lib/gwms-factory/server-credentials/entry_CMSHTPC_T3_US_Bridges2/tokens.tgz'"
+SciTokensFile = "$ENV(SCITOKENS_FILE:)"
Universe = grid
Grid_Resource = condor psc-bridges2-ce1.svc.opensciencegrid.org psc-bridges2-ce1.svc.opensciencegrid.org:9619
Executable = glidein_startup.sh
copy_to_spool = True
Arguments = $ENV(GLIDEIN_ARGUMENTS)
transfer_Input_files = $ENV(IDTOKENS_FILE:),/var/lib/gwms-factory/server-credentials/entry_CMSHTPC_T3_US_Bridges2/tokens.tgz
encrypt_Input_files = $ENV(IDTOKENS_FILE:),/var/lib/gwms-factory/server-credentials/entry_CMSHTPC_T3_US_Bridges2/tokens.tgz
Transfer_Executable = True
transfer_Output_files =
WhenToTransferOutput = ON_EXIT
stream_output = False
stream_error = False
+TransferOutput = ""
x509userproxy = $ENV(X509_USER_PROXY:)
+maxMemory = 250000
+maxWallTime = 2880
+GlideinFactory = "$ENV(FACTORY_NAME)"
+GlideinName = "$ENV(GLIDEIN_NAME)"
+GlideinEntryName = "$ENV(GLIDEIN_ENTRY_NAME)"
+GlideinEntrySubmitFile = "$ENV(GLIDEIN_ENTRY_SUBMIT_FILE)"
+GlideinClient = "$ENV(GLIDEIN_CLIENT)"
+GlideinFrontendName = "$ENV(GLIDEIN_FRONTEND_NAME)"
+GlideinCredentialIdentifier = "$ENV(GLIDEIN_CREDENTIAL_ID)"
+GlideinSecurityClass = "$ENV(GLIDEIN_SEC_CLASS)"
+GlideinWebBase = "$ENV(GLIDEIN_WEB_URL)"
+GlideinLogNr = "$ENV(GLIDEIN_LOGNR)"
+GlideinWorkDir = "$ENV(GLIDEIN_STARTUP_DIR)"
+GlideinSlotsLayout = "$ENV(GLIDEIN_SLOTS_LAYOUT)"
+GlideinMaxWalltime = $ENV(GLIDEIN_MAX_WALLTIME)
+fename = "$ENV(GLIDEIN_USER)"
periodic_remove = (isUndefined(GlideinSkipIdleRemoval)==True || GlideinSkipIdleRemoval==False) && (JobStatus==1 && isInteger($ENV(GLIDEIN_IDLE_LIFETIME)) && $ENV(GLIDEIN_IDLE_LIFETIME)>0 && (time() - QDate)>$ENV(GLIDEIN_IDLE_LIFETIME)) || (JobStatus==2 && ((time() - EnteredCurrentStatus) > (GlideinMaxWalltime + 126060)))
Notification = Never
+Owner = undefined
Log = /var/log/gwms-factory/client/user_$ENV(GLIDEIN_USER)/glidein_gfactory_instance_hepcintfactory01/entry_CMSHTPC_T3_US_Bridges2/condor_activity_$ENV(GLIDEIN_LOGNR)$ENV(GLIDEIN_CLIENT).log
Output = /var/log/gwms-factory/client/user
Error = /var/log/gwms-factory/client/user_$ENV(GLIDEIN_USER)/glidein_gfactory_instance_hepcintfactory01/entry_CMSHTPC_T3_US_Bridges2/job.$(Cluster).$(Process).err
It appears that the local location of the IDTOKENS_FILE variable is intentional and as given above is used to give the condor_submit command info on where to find the transfer_Input_files. So it is no problem that it has this value.
What would be nice is to also have a variable set within the glidein which shows what the path to said IDTOKENS file
is once it has been transferred to the remote location.
To Reproduce
Steps to reproduce the behavior:
Expected behavior
A clear and concise description of what you expected to happen.
Screenshots
If applicable, add screenshots and/or console outputs to help explain your problem.
Info (please complete the following information):
Stakeholders and components can be a comma separated list or on multiple lines.
If you add a new stakeholder or component, not on the sample list, add it on a line by its own.
- GlideinWMS version: 3.10.1
- Python version:3
- OS version: SL7
- HTCondor version: 10.0.1
- Priority: Priority level of this bug [critical, high, medium, low] medium
- Stakeholders: Concerned stakeholder(s) CMS, HEPCloud
- Components: The affected component due to this bug [frontend, factory, glidein, documentation, CI, testing, release, factory monitoring, frontend monitoring, ...]. Factory, glidein (particularly setup_x509.sh)
Additional context
Add any other context or supporting files about the problem here.