Skip to content

IDTOKENS_FILE and other glidein variables set to paths on the factory instead of on the remote execution host #281

Open
@StevenCTimm

Description

@StevenCTimm

Describe the bug
Glideinwms factory 3.10.1 and greater (possibly before that too) set an IDTOKENS_FILE variable
which appears to be pointing to the local factory location of the idtoken and not the destination to which
it ought to be transferred. This IDTOKENS_FILE variable is propagated to the job although it does not appear
to actually be used for anything there. It appears to be a consequence of
these lines in a job.condor file as produced by a config of glideinwms 3.10.1/ condor 10.0.1

[root@hepcintfactory01 entry_CMSHTPC_T3_US_TACC_FRONTERA]# cat /var/lib/gwms-factory/work-dir/entry_CMSHTPC_T3_US_Bridges2/job.condor

File: job.condor

environment = "IDTOKENS_FILE=$ENV(IDTOKENS_FILE:) JOB_TOKENS='/var/lib/gwms-factory/server-credentials/entry_CMSHTPC_T3_US_Bridges2/tokens.tgz'"
+SciTokensFile = "$ENV(SCITOKENS_FILE:)"
Universe = grid
Grid_Resource = condor psc-bridges2-ce1.svc.opensciencegrid.org psc-bridges2-ce1.svc.opensciencegrid.org:9619
Executable = glidein_startup.sh
copy_to_spool = True
Arguments = $ENV(GLIDEIN_ARGUMENTS)
transfer_Input_files = $ENV(IDTOKENS_FILE:),/var/lib/gwms-factory/server-credentials/entry_CMSHTPC_T3_US_Bridges2/tokens.tgz
encrypt_Input_files = $ENV(IDTOKENS_FILE:),/var/lib/gwms-factory/server-credentials/entry_CMSHTPC_T3_US_Bridges2/tokens.tgz
Transfer_Executable = True
transfer_Output_files =
WhenToTransferOutput = ON_EXIT
stream_output = False
stream_error = False
+TransferOutput = ""
x509userproxy = $ENV(X509_USER_PROXY:)
+maxMemory = 250000
+maxWallTime = 2880
+GlideinFactory = "$ENV(FACTORY_NAME)"
+GlideinName = "$ENV(GLIDEIN_NAME)"
+GlideinEntryName = "$ENV(GLIDEIN_ENTRY_NAME)"
+GlideinEntrySubmitFile = "$ENV(GLIDEIN_ENTRY_SUBMIT_FILE)"
+GlideinClient = "$ENV(GLIDEIN_CLIENT)"
+GlideinFrontendName = "$ENV(GLIDEIN_FRONTEND_NAME)"
+GlideinCredentialIdentifier = "$ENV(GLIDEIN_CREDENTIAL_ID)"
+GlideinSecurityClass = "$ENV(GLIDEIN_SEC_CLASS)"
+GlideinWebBase = "$ENV(GLIDEIN_WEB_URL)"
+GlideinLogNr = "$ENV(GLIDEIN_LOGNR)"
+GlideinWorkDir = "$ENV(GLIDEIN_STARTUP_DIR)"
+GlideinSlotsLayout = "$ENV(GLIDEIN_SLOTS_LAYOUT)"
+GlideinMaxWalltime = $ENV(GLIDEIN_MAX_WALLTIME)
+fename = "$ENV(GLIDEIN_USER)"
periodic_remove = (isUndefined(GlideinSkipIdleRemoval)==True || GlideinSkipIdleRemoval==False) && (JobStatus==1 && isInteger($ENV(GLIDEIN_IDLE_LIFETIME)) && $ENV(GLIDEIN_IDLE_LIFETIME)>0 && (time() - QDate)>$ENV(GLIDEIN_IDLE_LIFETIME)) || (JobStatus==2 && ((time() - EnteredCurrentStatus) > (GlideinMaxWalltime + 126060)))
Notification = Never
+Owner = undefined
Log = /var/log/gwms-factory/client/user_$ENV(GLIDEIN_USER)/glidein_gfactory_instance_hepcintfactory01/entry_CMSHTPC_T3_US_Bridges2/condor_activity_$ENV(GLIDEIN_LOGNR)$ENV(GLIDEIN_CLIENT).log
Output = /var/log/gwms-factory/client/user
$ENV(GLIDEIN_USER)/glidein_gfactory_instance_hepcintfactory01/entry_CMSHTPC_T3_US_Bridges2/job.$(Cluster).$(Process).out
Error = /var/log/gwms-factory/client/user_$ENV(GLIDEIN_USER)/glidein_gfactory_instance_hepcintfactory01/entry_CMSHTPC_T3_US_Bridges2/job.$(Cluster).$(Process).err

It appears that the local location of the IDTOKENS_FILE variable is intentional and as given above is used to give the condor_submit command info on where to find the transfer_Input_files. So it is no problem that it has this value.
What would be nice is to also have a variable set within the glidein which shows what the path to said IDTOKENS file
is once it has been transferred to the remote location.

To Reproduce
Steps to reproduce the behavior:

Expected behavior
A clear and concise description of what you expected to happen.

Screenshots
If applicable, add screenshots and/or console outputs to help explain your problem.

Info (please complete the following information):
Stakeholders and components can be a comma separated list or on multiple lines.
If you add a new stakeholder or component, not on the sample list, add it on a line by its own.

  • GlideinWMS version: 3.10.1
  • Python version:3
  • OS version: SL7
  • HTCondor version: 10.0.1
  • Priority: Priority level of this bug [critical, high, medium, low] medium
  • Stakeholders: Concerned stakeholder(s) CMS, HEPCloud
  • Components: The affected component due to this bug [frontend, factory, glidein, documentation, CI, testing, release, factory monitoring, frontend monitoring, ...]. Factory, glidein (particularly setup_x509.sh)

Additional context
Add any other context or supporting files about the problem here.

Metadata

Metadata

Assignees

No one assigned

    Labels

    BUGFor BUGSCritical**Needs urgent/immediate addressing**ci-testingfor affected componentcmsCMS stakeholderdocumentationfor affected componentfactoryfor affected componentfactory-monfor affected componentfrontendfor affected componentfrontend-monfor affected componentglideinfor affected componenthepcloudHEPCloud stakeholderreleasefor affected component

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions