generate-table-partitions make error

[Threeboys33]

Describe the bug

The YAML file generated using the generate-table-partitions method contains an SQL statement with the :param_1 parameter. When subsequently running config run, the following error occurs:
parse error: Unfinished string at EOF at line 1, column 248180736

What version of DVT are you using?
DVT: 8.5.2, python 3.11

What type of connections are you using for source and target?
Ex: Mysql 8.x for source, PG 9.x for target

Steps to reproduce the behavior

1. Add 'data-validation generate-table-partitions -sc mysql -tc dws
   -pk "l_orderkey,l_partkey,l_suppkey,l_linenumber"
   --tables-list "test_db.lineitem=public.lineitem"
   --hash "l_shipinstruct"
   -cdir partions
   --partition-num 10
   --format json'
2. Execute 'data-validation -v -ll DEBUG configs run -c partions/test_db.lineitem/0000.yaml | jq '.''
3. Scroll down to '03/19/2026 07:27:29 PM-INFO: -- ** Source Query ** --
   03/19/2026 07:27:29 PM-INFO: SELECT t0.hash__all, t0.l_orderkey, t0.l_partkey, t0.l_suppkey, t0.l_linenumber
   FROM (SELECT t1.l_orderkey AS l_orderkey, t1.l_partkey AS l_partkey, t1.l_suppkey AS l_suppkey, t1.l_linenumber AS l_linenumber, t1.l_quantity AS l_quantity, t1.l_extendedprice AS l_extendedprice, t1.l_discount AS l_discount, t1.l_tax AS l_tax, t1.l_returnflag AS l_returnflag, t1.l_linestatus AS l_linestatus, t1.l_shipdate AS l_shipdate, t1.l_commitdate AS l_commitdate, t1.l_receiptdate AS l_receiptdate, t1.l_shipinstruct AS l_shipinstruct, t1.l_shipmode AS l_shipmode, t1.l_comment AS l_comment, t1.cast__l_shipinstruct AS cast__l_shipinstruct, t1.ifnull__cast__l_shipinstruct AS ifnull__cast__l_shipinstruct, t1.rstrip__ifnull__cast__l_shipinstruct AS rstrip__ifnull__cast__l_shipinstruct, t1.concat__all AS concat__all, sha2(t1.concat__all, '256') AS hash__all
   FROM (SELECT t2.l_orderkey AS l_orderkey, t2.l_partkey AS l_partkey, t2.l_suppkey AS l_suppkey, t2.l_linenumber AS l_linenumber, t2.l_quantity AS l_quantity, t2.l_extendedprice AS l_extendedprice, t2.l_discount AS l_discount, t2.l_tax AS l_tax, t2.l_returnflag AS l_returnflag, t2.l_linestatus AS l_linestatus, t2.l_shipdate AS l_shipdate, t2.l_commitdate AS l_commitdate, t2.l_receiptdate AS l_receiptdate, t2.l_shipinstruct AS l_shipinstruct, t2.l_shipmode AS l_shipmode, t2.l_comment AS l_comment, t2.cast__l_shipinstruct AS cast__l_shipinstruct, t2.ifnull__cast__l_shipinstruct AS ifnull__cast__l_shipinstruct, t2.rstrip__ifnull__cast__l_shipinstruct AS rstrip__ifnull__cast__l_shipinstruct, concat_ws(:param_1, t2.rstrip__ifnull__cast__l_shipinstruct) AS concat__all
   '
4. 5. See error '03/19/2026 07:28:03 PM-INFO: {"run_id": "8103a851-a491-458f-b6fa-77379a8591e1", "start_time": "2026-03-19T11:27:29.642633+00:00", "end_time": "2026-03-19T11:27:43.132156+00:00", "total_source_rows": 1199799, "total_target_rows": 1199799, "total_rows_validated": 1199799, "total_rows_success_validation_status": 1199799, "total_rows_fail_validation_status": 0, "failed_rows_present_in_source_not_in_target": 0, "failed_rows_present_in_target_not_in_source": 0, "failed_rows_present_in_both_source_and_target": 0}
      03/19/2026 07:28:03 PM-DEBUG: Generate report elapsed: 22.75s
      parse error: Unfinished string at EOF at line 1, column 248180736'
      Expected behavior
      A clear and concise description of what you expected to happen.

Screenshots
If applicable, add screenshots to help explain your problem.

How are you executing DVT?
Ex: CLI commands on local computer or VM, within a DAG, Cloud Run job, etc

Additional context
Add any other context about the problem here.

[sundar-mudupalli-work]

Hi,

Generate-table-partitions is used when you have a large table that will not fit into memory. You can split the table it into chunks that fit into memory and can be validated. Json format is not recommended in such situations - because parsing large volumes of text is difficult. Usage of the Postgres or Bigquery result handler is recommended.

Coming to your specific issue, the use of DEBUG log level conflicts with the json format. DEBUG messages are plain text.

If you still see an issue after removing the DEBUG log level, then capture the output into a file and try to find out the exact line where jq is having difficulty. Please attach your table schema, the output line that creates an issue for jq and the summary of the run - usually the first json object.

Sundar Mudupalli