Skip to content

KeyError 'table.head' when running validator.head() using IBM DB2 datasource #5583

@Adenmin

Description

@Adenmin

Hello there again!

I continue working on integration GE and IBM Db2
and I already fixed issue #5579 by upgrading GE to 0.15.15
but I faced new issue...

Environment:
Operating System: Linux Ubuntu 22.04
Great Expectations Version: 0.15.15
ibm-db Version: 3.1.2
ibm-db-sa Version: 0.3.8
SQLAlchemy Version: 1.4.39

Config

datasources:
  db2_test_datasource:
    class_name: Datasource
    data_connectors:
      default_runtime_data_connector_name:
        class_name: RuntimeDataConnector
        module_name: great_expectations.datasource.data_connector
        batch_identifiers:
          - default_identifier_name
      default_inferred_data_connector_name:
        class_name: InferredAssetSqlDataConnector
        include_schema_name: true
        module_name: great_expectations.datasource.data_connector
    module_name: great_expectations.datasource
    execution_engine:
      module_name: great_expectations.execution_engine
      class_name: SqlAlchemyExecutionEngine
      connection_string: ibm_db_sa://admin:admin@127.0.0.1:50000/TEST

run it

context = ge.data_context.DataContext()

batch_request = RuntimeBatchRequest(
    datasource_name="db2_test_datasource",
    data_connector_name="default_runtime_data_connector_name",
    data_asset_name="default_name",  # this can be anything that identifies this data
    runtime_parameters={"query": "SELECT * FROM TEST_TABLE LIMIT 10"},
    batch_identifiers={"default_identifier_name": "default_identifier"},
)
context.create_expectation_suite(
    expectation_suite_name="test_suite", overwrite_existing=True
)
validator = context.get_validator(
    batch_request=batch_request, expectation_suite_name="test_suite"
)
print(validator.head())

and I got

KeyError: ('table.head', 'batch_id=
... 
raise ge_exceptions.MetricResolutionError(\ngreat_expectations.exceptions.exceptions.MetricResolutionError: (ibm_db_dbi.ProgrammingError) ibm_db_dbi::ProgrammingError: SQLNumResultCols failed: [IBM][CLI Driver][DB2/LINUXX8664] SQL0204N  "ADMIN.GE_TEMP_C844EBCD" is an undefined name.  SQLSTATE=42704 SQLCODE=-204\n[SQL: SELECT * \nFROM ge_temp_c844ebcd FETCH FIRST 5 ROWS ONLY]\n(Background on this error at: https://sqlalche.me/e/14/f405)\n', 'exception_message': '(ibm_db_dbi.ProgrammingError) ibm_db_dbi::ProgrammingError: SQLNumResultCols failed: [IBM][CLI Driver][DB2/LINUXX8664] SQL0204N  "ADMIN.GE_TEMP_C844EBCD" is an undefined name.  SQLSTATE=42704 SQLCODE=-204\n[SQL: SELECT * \nFROM ge_temp_c844ebcd FETCH FIRST 5 ROWS ONLY]\n(Background on this error at: https://sqlalche.me/e/14/f405)', 'raised_exception': True}}}}

Looks like something wrong with temp tables.
I saw some similar issues about validator.head()
#5424
#5478
but I'm not sure that they have common root cause.

Metadata

Metadata

Assignees

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions