contrib/drivers: comprehensive database driver test coverage improvement plan

[lingcoder]

Background

Current State

GoFrame's contrib/drivers provides 11 database driver implementations.
Each driver adapts gdb.Model's 161 public methods to its specific SQL dialect.
MySQL driver has the most mature test suite, but there's a significant gap between drivers.

Data as of: 2026-02-12

Driver	Test Files	Test Functions	Test Lines	Assessment
MySQL	23	348	~18,700	Broad coverage, depth gaps in some areas
MariaDB	2	39	~900	Minimal, MySQL-compatible
PgSQL	11	110	~5,500	+5 PRs adding ~14 files
GaussDB	13	123	~5,800	+4 PRs adding ~13 files
SQLite	3	157	~4,200	High function count but few files
MSSQL	5	139	~4,000	Missing transaction/where/hook coverage
Oracle	3	60	~1,800	Significant gaps
DM	6	39	~1,500	Significant gaps
ClickHouse	5	31	~1,200	Limited by column-store nature
OceanBase	0	0	0	No tests
TiDB	0	0	0	No tests

Problem

1. Database driver coverage imbalance: MySQL has 23 test files, while most database drivers have 2~6, making it hard to guarantee driver adapter quality across databases
2. No shared baseline: Each database driver's tests were written independently, no systematic standard to measure coverage against
3. MySQL itself has depth gaps: Despite being the most complete, Lock operations (4 variants), OmitEmpty/OmitNil (6 methods), Context timeout/cancellation, concurrency safety, and advanced data types (JSON/Binary/Decimal) lack dedicated testing in the MySQL driver
4. Duplicated effort: Similar database operation tests are rewritten per driver rather than systematically ported

Core Idea

MySQL First → Standardized Baseline → Per-Driver Alignment + Driver-Specific Features

Use MySQL database driver as the single source of truth. First make MySQL driver tests comprehensive, then systematically port them to all other database drivers using standard SQL, and finally add driver-specific feature tests for each database's unique capabilities.

Three-Layer Test Model

Layer	Scope	Content
Layer 3	Driver-Specific Features	MySQL: PARTITION, ON DUPLICATE KEY. PgSQL: RETURNING, JSONB, ARRAY. DM: MERGE INTO. ClickHouse: MergeTree, Array functions
Layer 2	Standard SQL Tests (MySQL → port to all)	CRUD, Transaction, Where, Join, Union, SoftTime, Hook, Ctx, ScanList, With, Builder, Subquery, Struct, DO, Raw, Lock, Batch, Cache, OmitNil, Concurrent ...
Layer 1	Driver Basics (each driver already has)	init_test, open_test, basic_test, type_mapping

Porting Rules for Layer 2

• Change package mysql_test → package xxx_test
• Placeholders: GoFrame ORM uses ? uniformly across all drivers — no manual conversion needed (Core layer automatically translates to driver-specific formats: PgSQL $N, Oracle :N, etc.)
• Identifier quoting: Adapt quoting style per database (MySQL `table` → PgSQL "table" → MSSQL [table])
• SQL syntax differences: LIMIT/OFFSET variants, JOIN syntax, function names, data type mappings
• Unsupported features: t.Skip("xxx not supported by <driver>"), preserving test structure
• SQL should use standard syntax as much as possible to maximize reuse
• Each driver's createTable/createInitTable/dropTable helpers handle dialect differences internally
• Test SQL files (testdata/) are copied per driver rather than shared, since each driver is an independent Go module under contrib/drivers/

Step 1: Strengthen MySQL Test Coverage

MySQL API method coverage breadth is ~99% (348 test functions touching nearly all 161 Model public methods), but the following areas need deeper, dedicated testing:

Priority	Area	Current	Gap
P0	Pagination APIs (Chunk/AllAndCount/ScanAndCount)	❌ No tests	Full coverage needed for production-critical features
P0	Error handling & edge cases	⚠️ Minimal	Nil/empty data, SQL injection prevention, boundary values
P1	Lock (Lock/LockUpdate/LockShared/LockUpdateSkipLocked)	❌ No dedicated tests	Full coverage needed
P1	OmitEmpty/OmitNil (6 methods)	Only 2 covered	All combinations
P1	Transaction isolation & deadlock	⚠️ Basic only	Isolation levels, deadlock retry, deep nesting
P1	Advanced types (JSON/Binary/Decimal/Geometry)	⚠️ Minimal	Deep coverage for production data types
P1	Context	3 basic tests	Timeout, cancellation, propagation chain
P1	Concurrency	Almost none	goroutine race, connection pool
P1	Batch operations	Basic Insert only	Batch Delete, large batch edge cases
P1	Hook	4 basic tests	Hook chaining, ordering
P1	Complex queries (Subquery/Join/Having)	⚠️ Basic	Correlated subquery, 5+ table joins, advanced scenarios

Multiple PRs will be submitted to fill these gaps before porting to other drivers.

New test files needed (P0 - Critical Missing Coverage)

Test File	Coverage	Est. Functions
mysql_z_unit_feature_pagination_test.go	Chunk batch iteration, AllAndCount/ScanAndCount for efficient pagination, Page/Limit edge cases	~20
mysql_z_unit_feature_error_handling_test.go	Nil/empty data handling, SQL injection prevention, invalid parameters, boundary values, resource exhaustion	~28

New test files needed (P1 - Depth Gaps)

Test File	Coverage	Est. Functions
mysql_z_unit_feature_lock_test.go	Lock / LockUpdate / LockShared / LockUpdateSkipLocked, combined with transactions	~8
mysql_z_unit_feature_omit_test.go	OmitEmpty / OmitNil / OmitEmptyData / OmitNilData / OmitEmptyWhere / OmitNilWhere, all 6 methods × multiple scenarios	~12
mysql_z_unit_feature_cache_test.go	Cache / PageCache with TTL, invalidation, concurrent access	~8
mysql_z_unit_feature_batch_test.go	Batch Insert / Update / Delete, large batch edge cases, empty batch	~6
mysql_z_unit_feature_metadata_test.go	TableFields / HasField / QuoteWord	~6
mysql_z_unit_feature_concurrent_test.go	Connection pool race, concurrent read/write, goroutine safety	~10

Existing files to enhance (P1 - Depth Gaps)

File	Enhancement	Est. New Functions
_transaction_test.go	Isolation levels (READ_COMMITTED/SERIALIZABLE), deadlock detection & retry, 5+ nested transactions	+25
_feature_raw_type_test.go	JSON path queries (JSON_EXTRACT/JSON_SET), Binary 1MB+ integrity, Decimal precision (65,30), Datetime timezone & boundaries, ENUM/SET validation, Geometry types (Point/Polygon)	+28
_feature_model_subquery_test.go	Correlated subquery, FROM subquery, multi-level nesting	+6
_feature_model_join_test.go	5+ table joins, self-join, outer join NULL handling	+6
_model_where_test.go	WherePrefix* / WhereOrPrefix* series completion, Having with aggregates	+14
_feature_ctx_test.go	Context timeout, cancellation, propagation chain	+5
_feature_hook_test.go	Hook chaining, multiple hooks ordering, hook abort	+4
_feature_master_slave_test.go	Concurrent read/write routing	+3

MySQL-specific feature tests (Layer 3)

Test File	Coverage
mysql_z_unit_feature_json_test.go	JSON column CRUD, JSON_EXTRACT, JSON path queries
mysql_z_unit_feature_duplicate_test.go	ON DUPLICATE KEY UPDATE full scenarios
mysql_z_unit_feature_partition_test.go	PARTITION BY RANGE / HASH / LIST

Step 2: Align All Drivers to Standard Layer

Once MySQL test coverage is solid, systematically port Layer 2 tests to each driver.

How it works

• Group related test files into one PR (e.g., Transaction/Where/Hook/Ctx as one PR, SoftTime/With/ScanList as another)
• Each PR focuses on a cohesive set of features for easier review
• Exact number of PRs per driver depends on the final MySQL test file count after Step 1
• Already proven with PgSQL (5 PRs) and GaussDB (4 PRs) — the same grouping pattern will be applied to all drivers
• Each driver's PRs will reference this issue via ref #<this-issue>
• Standard SQL maximizes cross-driver reuse, reducing total effort

Per-driver status and portability

Driver	Current Test Files	Portability	Key Adaptation Notes
MariaDB	2	Easy	MySQL-compatible, nearly zero SQL changes needed
PgSQL	11	Easy	Double-quote identifiers ("table"), RETURNING clause, array types
GaussDB	13	Easy	PgSQL-compatible syntax and behavior
SQLite	3	Medium	No Lock, no stored procedures, limited ALTER TABLE — t.Skip where unsupported
MSSQL	5	Medium	No SAVEPOINT, TOP instead of LIMIT, bracket identifiers [table], OUTPUT clause
Oracle	3	Hard	ROWNUM pagination, no LIMIT, MERGE syntax, significant type differences
DM	6	Medium	Oracle-family dialect, ROWNUM-style pagination, similar constraints
ClickHouse	5	Hard	Column-store engine, no Transaction/Update/Delete — majority of tests will t.Skip

Drivers not yet started

OceanBase and TiDB currently have 0 test files. They will be bootstrapped after the above drivers are aligned, starting with init_test.go and basic CRUD tests.

Step 3: Driver-Specific Feature Tests

After Layer 2 alignment, add dedicated tests for each database's unique capabilities that go beyond standard SQL. These ensure the driver adapter correctly handles dialect-specific features.

Driver	Unique Features to Test
MySQL	ON DUPLICATE KEY UPDATE, PARTITION BY, JSON column operations, FULLTEXT search
MariaDB	INSERT ... RETURNING (10.5+), Sequences, Window functions
PgSQL	RETURNING, JSONB operators, ARRAY types, LISTEN/NOTIFY, ON CONFLICT
GaussDB	OpenGauss-specific syntax beyond PgSQL compatibility
SQLite	In-memory database, WAL mode, ATTACH DATABASE
MSSQL	OUTPUT clause, TOP syntax, Common Table Expressions
Oracle	RETURNING INTO, ROWNUM, MERGE, PL/SQL integration
DM	MERGE INTO, ROWNUM, DM-specific type mappings
ClickHouse	MergeTree engine, materialized views, Array functions, FINAL keyword

These will be submitted as separate PRs per driver after the standard layer is aligned.

Progress

• MySQL test coverage strengthening (Step 1)
• MariaDB alignment
• PgSQL alignment
• GaussDB alignment
• SQLite alignment
• MSSQL alignment
• Oracle alignment
• DM alignment
• ClickHouse alignment
• OceanBase / TiDB bootstrap
• Driver-specific feature tests (Step 3)