fix(ingestion/schema-resolver): add enabler for sql parsing for more than 3 parts and implement in dremio

[acrylJonny]

No description provided.

[github-actions]

Linear: ING-1303

[codecov]

Codecov Report

❌ Patch coverage is 74.00000% with 13 lines in your changes missing coverage. Please review.
✅ All tests successful. No failed tests found.

Files with missing lines	Patch %	Lines
...c/datahub/ingestion/source/dremio/dremio_source.py	66.66%	12 Missing ⚠️
...adata-ingestion/src/datahub/sql_parsing/_models.py	92.85%	1 Missing ⚠️

📢 Thoughts on this report? Let us know!

[sgomezvillamor]

databse, db_schema and table are self-explained

⚠️ however this new parts field would require some docs about the purpose

💡 additionally, we transform some comments into actual code to emphasize the identity constraint

eg

@property
def identity(self) -> tuple:
    return (self.database, self.db_schema, self.table) # parts excluded on purpose

[sgomezvillamor]

why the hasattr? in which cases sqlglot.exp.Table have/miss this parts field?

similar with the gettattr below, quite confusing

maybe AI agent being excessively/unnecessarily conservative?

[sgomezvillamor]

where this constraint comes from? us, sqlglot, dremio itself?

[sgomezvillamor]

would it make sense to have a method in _TableName providing this or part of this logic?

# _TableName
@property
def name_with_parts(self): # I'm sure you can find a better name
    return ".".join(filter(None, list(self.parts)))

# here
table_name = DREMIO_DATABASE_NAME + "." + table.name_with_parts

And I also think part of this could also be pushed to the _TableName

                table_name_parts = [
                    DREMIO_DATABASE_NAME,
                    table.database,
                    table.db_schema,
                    table.table,
                ]
            else:
                table_name_parts = [
                    table.database or DREMIO_DATABASE_NAME,
                    table.db_schema,
                    table.table,
                ]

[sgomezvillamor]

Bart also found this

BLOCKERS - Must Fix Before Merge:*

1. qualified() method loses the parts field (line 74-78 in _models.py)
   • When creating a new _TableName instance with defaults applied, the parts field isn't preserved
   • This breaks multi-part table handling - the entire feature falls apart here
   • Fix: Add parts=self.parts when returning the new instance

[sgomezvillamor]

Left some comments

Main concern is: given we have new fields in _TableName let's try to keep the logic around that encapsulated in the _TableName class.

[sgomezvillamor]

please, add docs around this field

[sgomezvillamor]

approving to unblock