Skip to content

limitations.md

Latest commit

 

History

History
54 lines (36 loc) · 3.6 KB

limitations.md

File metadata and controls

54 lines (36 loc) · 3.6 KB

Data Validation Tool Restrictions and Limitations

General

  • Floating-point data types, e.g. Float and Double, are inexact by nature. Validations that involve conversion of floating-point data to string, e.g. --hash and --concat, can be problematic.

BigQuery

  • BigQuery does not have a 32 bit floating-point data type. Validations of systems containing 32 bit floats (e.g. Oracle BINARY_FLOAT) will likely be problematic when compared to FLOAT64.

Db2 LUW

  • Db2 LUW BLOB datatype is excluded from row validation due to incompatibility with concatenation/hashing functions.
  • Db2 LUW XML data type is not compatible with the LENGTH function and is excluded from any column validations other than --count.
  • Db2 LUW custom query validations require the creation of temporary views in the database to resolve additional data type information. This requires CREATE VIEW privileges for the connected user. Without it some data types will not be resolved correctly, for example CHARACTER will be interpreted as VARCHAR.

Db2 z/OS

  • Db2 z/OS XML data type is not compatible with aggregation functions or LENGTH function and is excluded from all column validations.
  • Db2 z/OS custom query validations require the creation of temporary views in the database to resolve additional data type information. This requires CREATE VIEW privileges for the connected user. Without it some data types will not be resolved correctly, for example CHARACTER will be interpreted as VARCHAR.
  • Db2 Z/OS ROWID data type is not supported.

Oracle

  • Requires the oracledb package to be installed as an extra dependency.

Snowflake

  • Requires the snowflake-connector-python and snowflake-sqlalchemy packages to be installed as extra dependencies.

SQL Server

  • Requires the pyodbc package to be installed as an extra dependency plus an OS level ODBC driver manager and client.
  • SQL Server does not have a function to "right trim" all whitespace, only spaces, therefore any validations relying on removal of trailing white space may encounter issues.
  • The text and ntext data types are incompatible with the len() therefore the datalength() function ius used in it's place which will give different results for multibyte characters.
  • The image data type is not currently supported, these columns are skipped when validated. See (issue-1578)[#1578] for details.

Sybase ASE

  • Requires the sqlalchemy_sybase package to be installed as an extra dependency plus an OS level ODBC driver manager and client.
  • Sybase ASE does not have a SHA-256 hash function, therefore row validations much use --comparison-fields or --concat only.
  • Sybase ASE does not have a function to "right trim" all whitespace, only spaces, therefore any validations relying on removal of trailing white space may encounter issues.
  • The text data type is not currently supported because many Sybase ASE SQL functions do not support the data type.
  • Sybase ASE does not include window functions. At the current time this prevents Sybase ASE from being the source table in a generate-table-partitions command. A simple workaround is to flip the source and target connections.
  • We have only tested against Sybase ASE. DVT Sybase support does not include other Sybase flavours.

Teradata