Fix: Improve TikTok message and replied-message parsers with fallback table detection and enhanced #1416
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
…ack for missing TIMMessageKVORM, unified contacts logic
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description
This PR improves the robustness of the TikTok artifact parsers (tikTok.py and tikTokReplied.py) by adding flexible table-detection logic, fallback mechanisms, and more reliable contact extraction. These changes resolve execution failures and incomplete output when iLEAPP processes TikTok datasets where internal database schemas differ—such as missing TIMMessageKVORM, renamed message tables, or variations in contact-storage tables.
Errors Encountered
When running iLEAPP against TikTok datasets that use alternative or older database structures, the following issues were encountered:
Error 1 — Missing TIMMessageKVORM table
Some TikTok versions do not include the TIMMessageKVORM table at all.
This caused the original tikTokReplied.py parser to fail or return no replied-message results.
Example behavior:
Query against TIMMessageKVORM produced an operational error.
iLEAPP halted execution for the TikTok module.
Error 2 — Variations in message table names
Certain devices store messages in alternative tables, such as TIMMessageNewPropertyORM.
The original parser assumed a single table name, resulting in incomplete or empty output.
Error 3 — Contacts not detected consistently
TikTok stores contact and user metadata in several possible locations.
The previous implementation depended on a specific table name and could fail or return an empty contact list when this table was absent.
Dataset to Reproduce
Source: 2020 CTF – iOS
(Internal TikTok database structure in this dataset uses non-standard table names and missing cache tables.)
The Fix
This PR introduces multiple improvements to ensure compatibility across TikTok versions:
Added fallback detection for TIMMessageKVORM and alternative message tables such as TIMMessageNewPropertyORM.
Implemented wildcard table detection (AwemeContacts%) to reliably enumerate contacts regardless of schema variation.
Improved SQL query handling and error boundaries to prevent module termination when certain TikTok cache tables are missing.
Added safer message and contact extraction logic, ensuring returned artifacts remain complete even when database layouts differ across devices or versions.
Verification
Tested locally on the 2020 CTF – iOS dataset.
Before fix:
TikTok modules failed to execute when required tables were missing.
Replied messages and contacts could not be extracted.
After fix:
Parsers executed successfully.
Replied messages extracted using fallback tables.
Contacts populated using expanded table-detection logic.
