Feat/snowflake s3 path spec lineage

[alokr-dhub]

No description provided.

[github-actions]

PR Title Check Failed

Your PR title must follow the format: <type>[optional scope]: <description>

Examples:

• feat(ingestion): add Snowflake v2 source
• fix: resolve crash on empty dashboard

See the Contributing Guide for allowed types and format details.

[github-actions]

Remaining comments which cannot be posted as a review comment to avoid GitHub Rate Limit

actionlint

---

🚫 [actionlint] _{reported by reviewdog 🐶}
shellcheck reported issue in this script: SC2001:style:3:7: See if you can use ${variable//search/replace} instead [shellcheck]

datahub/.github/workflows/publish-prefect-plugin-pypi-release.yml

Line 21 in 6b1c773

run: |

---

⚠️ [actionlint] _{reported by reviewdog 🐶}
shellcheck reported issue in this script: SC2034:warning:2:1: SHORT_SHA appears unused. Verify use (or export if used externally) [shellcheck]

datahub/.github/workflows/publish-prefect-plugin-pypi-release.yml

Line 21 in 6b1c773

run: |

---

📝 [actionlint] _{reported by reviewdog 🐶}
shellcheck reported issue in this script: SC2086:info:3:12: Double quote to prevent globbing and word splitting [shellcheck]

datahub/.github/workflows/publish-prefect-plugin-pypi-release.yml

Line 21 in 6b1c773

run: |

---

📝 [actionlint] _{reported by reviewdog 🐶}
shellcheck reported issue in this script: SC2086:info:4:20: Double quote to prevent globbing and word splitting [shellcheck]

datahub/.github/workflows/publish-prefect-plugin-pypi-release.yml

Line 21 in 6b1c773

run: |

[github-actions]

🚫 [actionlint] _{reported by reviewdog 🐶}
shellcheck reported issue in this script: SC2001:style:3:7: See if you can use ${variable//search/replace} instead [shellcheck]

[github-actions]

⚠️ [actionlint] _{reported by reviewdog 🐶}
shellcheck reported issue in this script: SC2034:warning:2:1: SHORT_SHA appears unused. Verify use (or export if used externally) [shellcheck]

[github-actions]

📝 [actionlint] _{reported by reviewdog 🐶}
shellcheck reported issue in this script: SC2086:info:3:12: Double quote to prevent globbing and word splitting [shellcheck]

[github-actions]

📝 [actionlint] _{reported by reviewdog 🐶}
shellcheck reported issue in this script: SC2086:info:4:20: Double quote to prevent globbing and word splitting [shellcheck]

[github-actions]

📝 [actionlint] _{reported by reviewdog 🐶}
shellcheck reported issue in this script: SC2086:info:2:43: Double quote to prevent globbing and word splitting [shellcheck]

[github-actions]

📝 [actionlint] _{reported by reviewdog 🐶}
shellcheck reported issue in this script: SC2086:info:4:20: Double quote to prevent globbing and word splitting [shellcheck]

[github-actions]

🚫 [actionlint] _{reported by reviewdog 🐶}
shellcheck reported issue in this script: SC2001:style:3:7: See if you can use ${variable//search/replace} instead [shellcheck]

[github-actions]

⚠️ [actionlint] _{reported by reviewdog 🐶}
shellcheck reported issue in this script: SC2034:warning:2:1: SHORT_SHA appears unused. Verify use (or export if used externally) [shellcheck]

[github-actions]

📝 [actionlint] _{reported by reviewdog 🐶}
shellcheck reported issue in this script: SC2086:info:3:12: Double quote to prevent globbing and word splitting [shellcheck]

[github-actions]

📝 [actionlint] _{reported by reviewdog 🐶}
shellcheck reported issue in this script: SC2086:info:4:20: Double quote to prevent globbing and word splitting [shellcheck]

[codecov]

❌ 1 Tests Failed:

Tests completed	Failed	Passed	Skipped
21648	1	21647	94

View the top 1 failed test(s) by shortest run time

tests.unit.test_packages::test_check_str_enum_usage

Stack Traces | 0.468s run time

pytestconfig = <_pytest.config.Config object at 0x7ff2fae3dca0>

    def test_check_str_enum_usage(pytestconfig: pytest.Config) -> None:
        root = pytestconfig.rootpath
    
>       ensure_no_enum_mixin([root / "src", root / "tests"])

tests/unit/test_packages.py:28: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

dirs = [PosixPath('.../datahub/metadata-ingestion/src'), PosixPath('.../datahub/metadata-ingestion/tests')]

    def ensure_no_enum_mixin(dirs: List[pathlib.Path]) -> None:
        # See the docs on the StrEnum implementation for why this is necessary.
    
        bad_lines = {
            "(str, Enum)",
            "(str, enum.Enum)",
            # We don't have any int enums right now, but this will catch them if we add some.
            "(int, Enum)",
            "(int, enum.Enum)",
        }
    
        ignored_files = {
            "datahub/utilities/str_enum.py",
            "datahub/testing/check_str_enum.py",
        }
    
        for dir in dirs:
            for file in dir.rglob("*.py"):
                if any(str(file).endswith(ignored_file) for ignored_file in ignored_files):
                    continue
    
                with file.open() as f:
                    for line in f:
                        if any(bad_line in line for bad_line in bad_lines):
>                           raise ValueError(
                                f"Disallowed enum mixin found in {file}: `{line.rstrip()}`. "
                                "This enum mixin's behavior changed in Python 3.11, so it will work inconsistently across versions."
                                "Use datahub.utilities.str_enum.StrEnum instead."
                            )
E                           ValueError: Disallowed enum mixin found in .../datahub/metadata-ingestion/src/datahub/ingestion/source/data_lake_common/config.py: `class S3PathMode(str, Enum):`. This enum mixin's behavior changed in Python 3.11, so it will work inconsistently across versions.Use datahub.utilities.str_enum.StrEnum instead.

.../datahub/testing/check_str_enum.py:29: ValueError

To view more test analytics, go to the Test Analytics Dashboard
_{📋 Got 3 mins? Take this short survey to help us improve Test Analytics.}

[github-actions]

🚫 [actionlint] _{reported by reviewdog 🐶}
shellcheck reported issue in this script: SC2001:style:3:7: See if you can use ${variable//search/replace} instead [shellcheck]

[github-actions]

⚠️ [actionlint] _{reported by reviewdog 🐶}
shellcheck reported issue in this script: SC2034:warning:2:1: SHORT_SHA appears unused. Verify use (or export if used externally) [shellcheck]

[github-actions]

📝 [actionlint] _{reported by reviewdog 🐶}
shellcheck reported issue in this script: SC2086:info:3:12: Double quote to prevent globbing and word splitting [shellcheck]

[github-actions]

📝 [actionlint] _{reported by reviewdog 🐶}
shellcheck reported issue in this script: SC2086:info:4:20: Double quote to prevent globbing and word splitting [shellcheck]

[alokr-dhub]

Closing — re-opening on a fresh branch.