Description
Describe the issue:
I was advised on SO to post here, not sure its a bug, I may just not know something...
Im running dask locally and spinning up fargate cluster in AWS.
Its assuming a role correctly and starting up the cluster.
After six hours its permissions fail, I have set the maximum session duration on the role to 12 hours but it seems to be getting a defailt 6 hour token anyway and not asking for a 12 hour token.
Is there a way to instruct it to get a 12 hour token or do I need to refresh the session token somehow?
Minimal Complete Verifiable Example:
Cant post a full Minimal Complete Verifiable Example, would need something that goes over 6 hrs.
But our cluster is setup like:
cluster = FargateCluster(
cloudwatch_logs_group=f"{project}_log_group",
cluster_name_template=f"{project}-{uuid}",
environment={"DEFAULT_GSF_VERSION": gsf_version},
image=f"{aws_account_id}.dkr.ecr.eu-west-2.amazonaws.com/{project}:latest",
n_workers=9,
scheduler_cpu=4096,
scheduler_mem=30720,
scheduler_timeout="240 minutes",
security_groups=[aws_security_group_id],
task_role_arn=f"arn:aws:iam::{aws_account_id}:role/{project}ECSTaskRole",
worker_cpu=4096,
worker_mem=30720,
worker_extra_args=algorithm_extra_cluster_args,
fargate_use_private_ip=use_private_ip,
vpc=vpc,
subnets=subnets,
skip_cleanup=True,
platform_version="LATEST",
shutdown_on_close=True,
mount_points=mount_points,
volumes=volumes,
)
cluster.adapt(minimum=9, maximum=80)
logger.info(f"dashboard_link is: {cluster.dashboard_link}")
Anything else we need to know?:
I have set the maximum session duration on my role to 43200(12 hours), seems it still only gets a 6 hour token.
I tried adding duration_seconds=43200 to AWS config files hoping to override the default 6 hour token, still lost perms after 6 hours
Environment:
- Dask version:022.12.0
- Python version:3.8.3
- Operating System:linux
- Install method (conda, pip, source):poetry