Db2 implementation technical debt

[nj1973]

While working on adding Sybase support I had some confusion with a cast to varchar(3000) that I didn't implement. It turned out to be due to this line of code in the Db2 modifications:

professional-services-data-validator/third_party/ibis/ibis_db2/datatypes.py

Line 36 in 49ec473

ibis_type_to_sqla[dt.String] = sa.sql.sqltypes.String(length=3000)

This code is changing a global structure that is leaking into other supported engines.

It looks like we do something similar in the Oracle code too (although those lines were not manifesting in the Sybase testing I was doing).

It was further complicated by me seeing different results depending on which Python version I was using.

Either way - we should revisit this technique and consider an engine specific change rather than a global change.

[nj1973]

There is actually a second problem with this code.

By casting many columns to varchar(3000) we persuade Db2 that the length of a concatenation of multiple column is a multiple of 3000. Which quickly gives us an overflow error:

sqlalchemy.exc.ProgrammingError: (ibm_db_dbi.ProgrammingError) ibm_db_dbi::ProgrammingError: Binding Error: [IBM][CLI Driver][DB2/LINUXX8664] SQL0171N  The statement was not processed because the data type, length or value of the argument for the parameter in position "1" of routine "SYSIBM.HEX" is incorrect. Parameter name: "".  SQLSTATE=42815 SQLCODE=-171

As an experiment I reduced the 3000 to 300 and the test passed. That is not a solution, just confirmation that a system wide cast to varchar(3000) is a bad idea.