enh: refactor sqilte database to speed up while loading large dataset

[shun2wang]

Improve the current database behavior with minimal workload. Now implement a chunked schema to store our tables. This avoids the hacky behavior of scaling SQLite's maximum column and significantly improves I/O performance for vary large datasets.I'm not sure if this is friendly to multiple datasets in the future, but it's not an issue in terms of I/O.

Befor:

[JorisGoosen]

@shun2wang dont work too much on this!

#5783 will require substantial changes to the database and Im getting near the end of it!
Unless you really think it won't conflict, but Ill leave the rebase to you then when the time comes.

[shun2wang]

@shun2wang dont work too much on this!

#5783 will require substantial changes to the database and Im getting near the end of it! Unless you really think it won't conflict, but Ill leave the rebase to you then when the time comes.

If I look into #5783 its purpose is to load multiple datasets simultaneously, right? My PR aims to split and store a single large dataset, mainly to address the performance impact of manually enabling the maximum column limit in SQLite. I believe the goals are not conflicting.

I can rebase it on #5783，but maybe atfer #5783 be merged?

[JorisGoosen]

Splitting the tables might help, but I also think @RensDofferhoff is gearing up to replace sqlite entirely with something quicker.