How to migrate data from old versions to new one?

Problem:

Since v0.20-beta, TickTockDB uses a new binary format and it is not backward compatible. If you upgrade TickTockDB from an old version to v0.20 or upper, TickTockDB will require you to manually migrate your old data to the new binary format. Otherwise, TickTockDB will exit immediately, as shown in the following.

ylin30@rpi4:~/ticktock $  TickTockDB v0.20.0,  Maintained by
 Yongtao You (yongtao.you@gmail.com) and Yi Lin (ylin30@gmail.com).
 This program comes with ABSOLUTELY NO WARRANTY. It is free software,
 and you are welcome to redistribute it under certain conditions.
 For details, see <https://www.gnu.org/licenses/>.
Initialization failed: ticktockdb.version config missing; TickTockDB version mismatch!

[1]+  Exit 9                  ./bin/tt -c conf/tt.conf --tsdb.timestamp.resolution millisecond --http.server.port 6182,6183 --http.listener.count 8,8
ylin30@rpi4:~/ticktock $

How to migrate data:

We prepare a script <ticktock>/scripts/migrate_data.sh for this purpose. Please read its usage message:

ylin30@rpi4:~/ticktock $ ./scripts/migrate_data.sh -h
Usage  : ./scripts/migrate_data.sh <host> <tcp port for put> <inspect binary> <data file dir>
Note: Inspect binary must be version compatible to data files.
E.g., if the data files are generated with TT v0.12.1, then you'd better use the inspect binary in v0.12.1.
You can simply copy the inspect binary into a backup dir before you update your TT binary dir.
Example: ./scripts/migrate_data.sh localhost 6181 /tmp/inspect.0.12.1 /home/usr1/backup/data
ylin30@rpi4:~/ticktock $

There are a few preparation steps before you run migrate_data.sh.

0. Gracefully stop Ticktock (i.e., ./admin/stop.sh);
1. Move the old data folder to a secure place (e.g., /home/usr1/backup/data);
2. Keep the old version of Inspect which can interpret the old data files as <inspect binary>, e.g., /tmp/inspect.0.12.1. You can download some old versions here;
3. Install nc (e.g., sudo apt install netcat in ubuntu) if not installed yet;
4. Make sure the new TT (v0.20 or upper) is running and listening at <host> and <tcp port for OpenTSDB put, default 6181>
5. Then you can run migrate_data.sh, e.g.:

./scripts/migrate_data.sh localhost 6181 /tmp/inspect.0.12.1 /home/usr1/backup/data