Version compatibility issues in samples/ml/ml_jobs/e2e_task_graph

[KaranamLokesh]

In the requirements.txt file, It is instructed to use snowflake-ml-python>=1.9.0,
requirements.txt

sf-samples/samples/ml/ml_jobs/e2e_task_graph/requirements.txt

Line 1 in 68c5507

snowflake-ml-python>=1.9.0

but here

sf-samples/samples/ml/ml_jobs/e2e_task_graph/src/pipeline_dag.py

Line 344 in 68c5507

packages=["snowflake-snowpark-python", "snowflake-ml-python<1.9.0", "xgboost"], # NOTE: Temporarily pinning to <1.9.0 due to compatibility issues

it is using snowflake-ml-python<1.9.0

By using this setting, getting the following error

cleanup_task = DAGTask("cleanup_task", definition=cleanup, is_finalizer=True)
TypeError: __init__() got an unexpected keyword argument 'is_finalizer'

[KaranamLokesh]

There was another issue with snowflake certificate chain traversal.
According to Ron, In late April 2025, changes surfaced a bug in the Python connector’s certificate-chain traversal against S3 endpoints, causing many to see 254007 on sfc-*-customer-stage.s3.amazonaws.com; upgrading snowflake-connector-python to ≥ 3.15.0 fixed it by stopping chain traversal once a trusted anchor is reached. Pinning certifi to 2025.1.31 also helped where upgrading wasn’t immediately possible.

The work around for me was to remove is_finalizer argument and pin the CA bundle package to certifi==2025.1.31.