From df891761a45666b378ab7637151b4982714f4f8f Mon Sep 17 00:00:00 2001 From: Andrew Skowronski Date: Fri, 19 Dec 2025 09:02:57 -0500 Subject: [PATCH 1/3] Established individual topic for each UnityDataTool command --- UnityDataTool/Commands/analyze.md | 112 +++++++++++ UnityDataTool/Commands/archive.md | 73 ++++++++ UnityDataTool/Commands/dump.md | 107 +++++++++++ UnityDataTool/Commands/find-refs.md | 97 ++++++++++ UnityDataTool/README.md | 281 ++++++++++++---------------- 5 files changed, 509 insertions(+), 161 deletions(-) create mode 100644 UnityDataTool/Commands/analyze.md create mode 100644 UnityDataTool/Commands/archive.md create mode 100644 UnityDataTool/Commands/dump.md create mode 100644 UnityDataTool/Commands/find-refs.md diff --git a/UnityDataTool/Commands/analyze.md b/UnityDataTool/Commands/analyze.md new file mode 100644 index 0000000..b48aacd --- /dev/null +++ b/UnityDataTool/Commands/analyze.md @@ -0,0 +1,112 @@ +# analyze Command + +The `analyze` command extracts information from Unity Archives (e.g. AssetBundles) and SerializedFiles and dumps the results into a SQLite database. + +## Quick Reference + +``` +UnityDataTool analyze [options] +``` + +| Option | Description | Default | +|--------|-------------|---------| +| `` | Path to folder containing files to analyze | *(required)* | +| `-o, --output-file ` | Output database filename | `database.db` | +| `-p, --search-pattern ` | File search pattern (`*` and `?` supported) | `*` | +| `-s, --skip-references` | Skip CRC and reference extraction (faster, smaller DB) | `false` | +| `-v, --verbose` | Show more information during analysis | `false` | +| `--no-recurse` | Do not recurse into sub-directories | `false` | + +## Examples + +Analyze all files in a directory: +```bash +UnityDataTool analyze /path/to/asset/bundles +``` + +Analyze only `.bundle` files and save to a custom database: +```bash +UnityDataTool analyze /path/to/asset/bundles -o my_database.db -p "*.bundle" +``` + +Fast analysis (skip reference tracking): +```bash +UnityDataTool analyze /path/to/bundles -s +``` + +--- + +## What Can Be Analyzed + +The analyze command works with the following types of directories: + +| Input Type | Description | +|------------|-------------| +| **AssetBundle build output** | The output path of an AssetBundle build | +| **Addressables folder** | `StreamingAssets/aa` folder from a Player build | +| **Entities content** | `StreamingAssets/ContentArchives` folder for [Entities](https://docs.unity3d.com/Packages/com.unity.entities@1.4/manual/content-management-intro.html) projects | +| **Player Data folder** | The `Data` folder of a Unity Player build | +| **Compressed Player builds** | The `data.unity3d` file will be analyzed like AssetBundles | + +> **Note**: Some platforms require extracting content from platform-specific containers first (e.g., `.apk` files on Android). + +--- + +## Output Database + +The analysis creates a SQLite database that can be explored using tools like [DB Browser for SQLite](https://sqlitebrowser.org/) or the command line `sqlite3` tool. + +**Refer to the [Analyzer documentation](../../Analyzer/README.md) for complete database schema reference and usage examples.** + +See also: [Analyze Examples](../../Documentation/analyze-examples.md) + +--- + +## Troubleshooting + +### File Loading Warnings + +``` +Failed to load 'C:\....\MyData.db'. File may be corrupted or was serialized with a newer version of Unity. +``` + +These warnings occur when the tool encounters non-Unity files in the analyzed directory. They are usually harmless—the analyze process continues and produces a valid database. + +**Solutions:** +- Use `-p "*.bundle"` to filter by file extension +- Use `--no-recurse` to limit directory depth +- Use `-v` (verbose) to see which files are ignored + +The tool automatically ignores common non-Unity file types (`.txt`, `.json`, `.manifest`, etc.). + +### TypeTree Errors + +``` +Error processing file: C:\...\TestProject_Data\level0 +System.ArgumentException: Invalid object id. +``` + +This error occurs when SerializedFiles are built without TypeTrees. The command will skip these files and continue. + +**Solution:** Enable **ForceAlwaysWriteTypeTrees** in your Unity build settings. See [Unity Content Format](../../Documentation/unity-content-format.md) for details. + +### SQL Constraint Errors + +``` +SQLite Error 19: 'UNIQUE constraint failed: objects.id' +``` +or +``` +SQLite Error 19: 'UNIQUE constraint failed: serialized_files.id'. +``` + +These errors occur when the same serialized file name appears in multiple sources: + +| Cause | Solution | +|-------|----------| +| Multiple builds in same directory | Analyze each build separately | +| Scenes with same filename (different paths) | Rename scenes to be unique | +| AssetBundle variants | Analyze variants separately | + +See [Comparing Builds](../../Documentation/comparing-builds.md) for strategies to compare different versions of builds. + diff --git a/UnityDataTool/Commands/archive.md b/UnityDataTool/Commands/archive.md new file mode 100644 index 0000000..0057ad8 --- /dev/null +++ b/UnityDataTool/Commands/archive.md @@ -0,0 +1,73 @@ +# archive Command + +The `archive` command provides utilities for working with Unity Archives (AssetBundles and web platform `.data` files). + +## Sub-Commands + +| Sub-Command | Description | +|-------------|-------------| +| [`list`](#list) | List contents of an archive | +| [`extract`](#extract) | Extract contents of an archive | + +--- + +## list + +Lists the SerializedFiles contained within an archive. + +### Quick Reference + +``` +UnityDataTool archive list +``` + +### Example + +```bash +UnityDataTool archive list scenes.bundle +``` + +--- + +## extract + +Extracts the contents of an archive to disk. This is similar to Unity's `WebExtract` tool. + +### Quick Reference + +``` +UnityDataTool archive extract [options] +``` + +| Option | Description | Default | +|--------|-------------|---------| +| `` | Path to the archive file | *(required)* | +| `-o, --output-path ` | Output directory | `archive` | + +### Example + +```bash +UnityDataTool archive extract scenes.bundle -o contents +``` + +**Output files:** +``` +contents/BuildPlayer-SampleScene.sharedAssets +contents/BuildPlayer-SampleScene +contents/BuildPlayer-Scene2.sharedAssets +contents/BuildPlayer-Scene2 +``` + +> **Note:** The extracted files are binary SerializedFiles, not text. Use the [`dump`](dump.md) command to convert them to readable text format. + +--- + +## Comparison: extract vs dump + +| Command | Output | Use Case | +|---------|--------|----------| +| `archive extract` | Binary SerializedFiles | When you need the raw files | +| `dump` | Human-readable text | When you need to inspect content | + +The `dump` command can directly process archives without extracting first. + diff --git a/UnityDataTool/Commands/dump.md b/UnityDataTool/Commands/dump.md new file mode 100644 index 0000000..b721675 --- /dev/null +++ b/UnityDataTool/Commands/dump.md @@ -0,0 +1,107 @@ +# dump Command + +The `dump` command converts Unity SerializedFiles into human-readable text format. This is useful for inspecting the internal structure and properties of Unity assets. + +## Quick Reference + +``` +UnityDataTool dump [options] +``` + +| Option | Description | Default | +|--------|-------------|---------| +| `` | Path to file to dump | *(required)* | +| `-o, --output-path ` | Output folder | Current folder | +| `-f, --output-format ` | Output format | `text` | +| `-s, --skip-large-arrays` | Skip dumping large arrays | `false` | +| `-i, --objectid ` | Only dump object with this ID | All objects | + +## Examples + +Dump all objects in a file to the current directory: +```bash +UnityDataTool dump /path/to/file +``` + +Dump to a specific output folder: +```bash +UnityDataTool dump /path/to/file -o /path/to/output +``` + +Dump a single object by ID: +```bash +UnityDataTool dump /path/to/file -i 1234567890 +``` + +Skip large arrays for cleaner output: +```bash +UnityDataTool dump /path/to/file -s +``` + +--- + +## Archive Support + +When you pass an Archive file (like an AssetBundle), the command dumps all SerializedFiles inside. + +**Example:** For an AssetBundle `scenes.bundle` containing two scenes: + +```bash +UnityDataTool dump scenes.bundle +``` + +**Output files:** +``` +BuildPlayer-SampleScene.sharedAssets.txt +BuildPlayer-SampleScene.txt +BuildPlayer-Scene2.sharedAssets.txt +BuildPlayer-Scene2.txt +``` + +--- + +## Output Format + +The output is similar to Unity's `binary2text` tool. Each file begins with external references: + +``` +External References +path(1): "Library/unity default resources" GUID: 0000000000000000e000000000000000 Type: 0 +path(2): "Resources/unity_builtin_extra" GUID: 0000000000000000f000000000000000 Type: 0 +path(3): "archive:/CAB-35fce856128a6714740898681ea54bbe/..." GUID: 00000000000000000000000000000000 Type: 0 +``` + +Followed by object entries: + +``` +ID: -8138362113332287275 (ClassID: 135) SphereCollider + m_GameObject PPtr + m_FileID int 0 + m_PathID SInt64 -1473921323670530447 + m_Material PPtr + m_FileID int 0 + m_PathID SInt64 0 + m_IsTrigger bool False + m_Enabled bool True + m_Radius float 0.5 + m_Center Vector3f + x float 0 + y float 0 + z float 0 +``` + +**Refer to the [TextDumper documentation](../../TextDumper/README.md) for detailed output format explanation.** + +--- + +## Understanding PPtrs + +PPtrs (Property Pointers) are Unity's mechanism for referencing objects: + +| Field | Description | +|-------|-------------| +| `m_FileID` | Index into External References list (0 = same file) | +| `m_PathID` | Object's Local File Identifier (LFID) in that file | + +Use external references to resolve cross-file references. + diff --git a/UnityDataTool/Commands/find-refs.md b/UnityDataTool/Commands/find-refs.md new file mode 100644 index 0000000..9f4e8fb --- /dev/null +++ b/UnityDataTool/Commands/find-refs.md @@ -0,0 +1,97 @@ +# find-refs Command + +> ⚠️ **Experimental:** This command may not work as expected in all cases. + +The `find-refs` command traces reference chains leading to specific objects. Use it to understand why an asset was included (and potentially duplicated) in a build. + +## Quick Reference + +``` +UnityDataTool find-refs [options] +``` + +| Option | Description | Default | +|--------|-------------|---------| +| `` | Path to database from `analyze` command | *(required)* | +| `-i, --object-id ` | ID of object to analyze (from `id` column) | — | +| `-n, --object-name ` | Name of objects to analyze | — | +| `-t, --object-type ` | Type filter when using `-n` | — | +| `-o, --output-file ` | Output filename | — | +| `-a, --find-all` | Find all chains instead of stopping at first | `false` | + +> **Note:** Either `--object-id` or `--object-name` must be provided. + +## Prerequisites + +This command requires a database created by the [`analyze`](analyze.md) command **without** the `--skip-references` option. + +--- + +## Examples + +Find references to an object by name and type: +```bash +UnityDataTool find-refs my_database.db -n "MyTexture" -t "Texture2D" -o refs.txt +``` + +Find references to a specific object by ID: +```bash +UnityDataTool find-refs my_database.db -i 12345 -o refs.txt +``` + +Find all duplicate references (useful for finding why an asset is duplicated): +```bash +UnityDataTool find-refs my_database.db -n "SharedMaterial" -t "Material" -a -o all_refs.txt +``` + +--- + +## Use Cases + +| Scenario | Approach | +|----------|----------| +| **Why is this asset included?** | Use `-n` with the asset name | +| **Why is this asset duplicated?** | Use `-n` to find all instances (same name, different IDs) | +| **Trace specific object** | Use `-i` with the object ID from the database | +| **Find all reference chains** | Add `-a` flag (may take longer) | + +--- + +## Output Format + +The output shows the reference chain from root assets to your target object: + +``` +Reference chains to + ID: 1234 + Type: Transform + AssetBundle: asset_bundle_name + SerializedFile: CAB-353837edf22eb1c4d651c39d27a233b7 + +Found reference in: +MyPrefab.prefab +(AssetBundle = MyAssetBundle; SerializedFile = CAB-353837edf22eb1c4d651c39d27a233b7) + GameObject (id=1348) MyPrefab + ↓ m_Component.Array[0].component + RectTransform (id=721) [Component of MyPrefab (id=1348)] + ↓ m_Children.Array[9] + RectTransform (id=1285) [Component of MyButton (id=1284)] + ↓ m_GameObject + GameObject (id=1284) MyButton + ... + Transform (id=1234) [Component of MyButtonEffectLayer (1) (id=938)] + +Analyzed 266 object(s). +Found 1 reference chain(s). +``` + +### Reading the Output + +| Element | Description | +|---------|-------------| +| `↓ property_name` | The property containing the reference | +| `[Component of X (id=Y)]` | Shows the GameObject for Components | +| `[Script = X]` | Shows the script name for MonoBehaviours | + +**Refer to the [ReferenceFinder documentation](../../ReferenceFinder/README.md) for more details.** + diff --git a/UnityDataTool/README.md b/UnityDataTool/README.md index 1a6cb14..db9243e 100644 --- a/UnityDataTool/README.md +++ b/UnityDataTool/README.md @@ -1,227 +1,186 @@ # UnityDataTool -The UnityDataTool is a command line tool providing a set of commands related to Unity data files. It -is the main purpose of this repository. +A command-line tool for analyzing and inspecting Unity build output—AssetBundles, Player builds, Addressables, and more. -## Running UnityDataTool +## Commands -First you need to build, as described [here](../README.md#how-to-build). +| Command | Description | Documentation | +|---------|-------------|---------------| +| [`analyze`](#analyze) | Extract data from Unity files into a SQLite database | [Full docs →](Commands/analyze.md) | +| [`dump`](#dump) | Convert SerializedFiles to human-readable text | [Full docs →](Commands/dump.md) | +| [`archive`](#archive) | List or extract contents of Unity Archives | [Full docs →](Commands/archive.md) | +| [`find-refs`](#find-refs) | Trace reference chains to objects *(experimental)* | [Full docs →](Commands/find-refs.md) | -Once built, the UnityDataTools executable can be found inside `UnityDataTool/bin/[target specific folders]`. +--- -For example, on Windows the tool may be found a `[UnityDataTools repo path]\UnityDataTool\bin\Release\net9.0\UnityDataTool.exe` +## Quick Start ->[!TIP] ->Consider adding the directory that contains the UnityDataTool executable to the "Path" environmental variable on your computer. That makes it easy to invoke UnityDataTool from the command line and from other scripts, without specifying the full path every time. +```bash +# Show all commands +UnityDataTool --help -### Instructions for running on Mac +# Analyze AssetBundles into SQLite database +UnityDataTool analyze /path/to/bundles -o database.db -Note that on Mac, you need to publish the UnityDataTool project to get an executable file. You -can do it from your IDE or execute this command in the UnityDataTool folder (not from the root -folder): +# Dump a file to text format +UnityDataTool dump /path/to/file.bundle -o /output/path -(On intel-based macs) -`dotnet publish UnityDataTool -c Release -r osx-x64 -p:PublishSingleFile=true -p:UseAppHost=true` +# Extract archive contents +UnityDataTool archive extract file.bundle -o contents/ -(On apple silicon macs) -`dotnet publish UnityDataTool -c Release -r osx-arm64 -p:PublishSingleFile=true -p:UseAppHost=true` - -Also on Mac, you may be a warning that "UnityFileSystemApi.dylib" cannot be opened because the -developer cannot be verified. In that case click "Cancel", then open the System Preferences -> Security & -Privacy window. You should be able to allow the file from there. - - -# Usage - -The tool is invoked from the command line like this: `UnityDataTool [command] [command options]` - -For a list of available commands run it like this: `UnityDataTool --help` - -For help on a specific command use `--help` along with the command name, for example: `UnityDataTool analyze --help` - -Use `UnityDataTool --version` to print the current version of the tool. - -# Commands - -## analyze/analyse +# Find reference chains to an object +UnityDataTool find-refs database.db -n "ObjectName" -t "Texture2D" +``` -This command extracts information from Unity Archives (e.g. AssetBundles) and SerializedFiles and dumps the results into a SQLite database. +Use `--help` with any command for details: `UnityDataTool analyze --help` -The command takes the path of the folder containing the files to analyze as argument. +Use `--version` to print the tool version. -It also provides the following options: -* -o, --output-file \: filename of the database that will be created, the - default is database.db. Any existing file by that name will be replaced by the data from running this command. -* -s, --skip-references: skip CRC and reference (PPtrs) extraction. Faster processing and smaller - database, but inaccurate duplicate asset detection and no references table. -* -p, --search-pattern \: search pattern used to determine which files are AssetBundles. The default is \*. The * and ? characters are supported. Regular expressions are not supported. -* -v, --verbose: show more information during the analysis process, for example list any files that are ignored. -* --no-recurse: do not recurse into sub-directories. +--- -Example: `UnityDataTool analyze /path/to/asset/bundles -o my_database.db -p "*.bundle"` +## analyze -**Refer to this [documentation](../Analyzer/README.md#How-to-use-the-database) for more information -about the output database structure.** +Extract information from Unity Archives and SerializedFiles into a SQLite database. +```bash +UnityDataTool analyze [options] +``` -### Example Input to the Analyze command +| Option | Description | Default | +|--------|-------------|---------| +| `-o, --output-file` | Output database filename | `database.db` | +| `-p, --search-pattern` | File filter pattern (`*.bundle`) | `*` | +| `-s, --skip-references` | Skip CRC/reference extraction (faster) | — | +| `-v, --verbose` | Show detailed progress | — | +| `--no-recurse` | Don't recurse into subdirectories | — | -Example of directories that could be analyzed: +**Example:** +```bash +UnityDataTool analyze /path/to/bundles -o my_database.db -p "*.bundle" +``` -* The output path of an AssetBundle build. -* A folder inside the StreamingAssets folder of a Unity Player build. For example: - * The "StreamingAssets/aa" folder, containing AssetBundles from an Addressables build. - * The "StreamingAssets/ContentArchives" folder containing sub-scene content if your project uses [Entities](https://docs.unity3d.com/Packages/com.unity.entities@1.4/manual/content-management-intro.html). -* The "Data" folder of a Unity Player build. - * By default, any AssetBundles or ContentArchives in the StreamingAssets folder would also be included. Use the "--no-recurse" option to avoid that. - * Compressed Player Builds are supported. The data.unity3d file will be analyzed the same way AssetBundles are. - * The structure and content of a Player varies based on the platform. In some cases you may have to first extract the content out of a platform-specific container file prior to Analysis (for example .apk files on Android). +📖 [Full documentation](Commands/analyze.md) — Troubleshooting, database schema, example inputs -### Filtering Other File Types +--- -Analyze works by trying to process all files in the provided path, assuming they are all Unity Archives or SerializedFiles. Because there is no standard file extension for those files it is tricky to reliably distinguish these file types from the other files that may also be found in the build output. +## dump -This can lead to error messages in the UnityDataTool output like this: +Convert SerializedFiles to human-readable text format. +```bash +UnityDataTool dump [options] ``` -Failed to load 'C:\....\MyData.db'. File may be corrupted or was serialized with a newer version of Unity. -``` - -Typically these are not serious errors. The analyze process will continue, and can still produce a perfectly valid database file. However if there are many messages like this it can obscure more important or unexpected failures. -To reduce the number of these warnings, UnityDataTool automatically ignores common filenames and file paths that are found in Player, AssetBundle, Addressable or Entities builds. For example ".txt, .json, .manifest". When the `--verbose` option is passed each ignored file will be listed. +| Option | Description | Default | +|--------|-------------|---------| +| `-o, --output-path` | Output folder | Current folder | +| `-f, --output-format` | Output format | `text` | +| `-s, --skip-large-arrays` | Skip large array contents | — | +| `-i, --objectid` | Only dump this object ID | All objects | -If you use an extension or other naming convention for your AssetBundles, for example ".bundle", then you can avoid those warnings using the `-p .bundle` option to ignore other files. +**Example:** +```bash +UnityDataTool dump /path/to/file -o /output/path +``` -For Player builds there is no single -p option that can catch all SerializedFiles (unless it is a compressed build with a single `data.unity3d` file). +📖 [Full documentation](Commands/dump.md) — Output format details, archive support -The `--no-recurse` option can reduce the volume of these warnings. +--- +## archive -### Errors when TypeTrees are missing +Work with Unity Archives (AssetBundles, `.data` files). -If a SerializedFile is built without TypeTrees, then the Analyze command will not be able to extract information about the contained objects. It will print an error similar to this example, then skip to the next file: +### list +```bash +UnityDataTool archive list ``` -Error processing file: C:\Src\TestProject\Build\Player\TestProject_Data\level0 -System.ArgumentException: Invalid object id. -``` - -See [this topic](../Documentation/unity-content-format.md) for more information about TypeTrees. -### SQL Errors +### extract -The following SQL errors may occur when running the analyze command: - -``` -SQLite Error 19: 'UNIQUE constraint failed: objects.id' +```bash +UnityDataTool archive extract [options] ``` -or +| Option | Description | Default | +|--------|-------------|---------| +| `-o, --output-path` | Output directory | `archive` | -``` -SQLite Error 19: 'UNIQUE constraint failed: serialized_files.id'. +**Example:** +```bash +UnityDataTool archive extract scenes.bundle -o contents/ ``` -The likely cause of these errors is the same serialized file name appearing in more than Player or AssetBundle file that is processed by Analyze. +📖 [Full documentation](Commands/archive.md) — Sub-command details, comparison with dump -This may occur: +--- -* If you analyze files from more than one version of the same build (e.g. if you run it on a directory that contains two different builds of the same project in separate sub-directories). -* If two scenes with the same filename (but different paths) are included in a build. -* In a build that used AssetBundle variants. +## find-refs -The conflicting name makes it impossible to uniquely identify the serialized file and its object in the database, and makes it ambiguous how to interpret dependencies from one file to another. +> ⚠️ Experimental -The [comparing builds](../Documentation/comparing-builds.md) topic gives some ideas about how to run Analyze more than once if you want to compare two different versions of the same build. +Find reference chains leading to specific objects. Requires a database from `analyze` (without `--skip-references`). -## dump +```bash +UnityDataTool find-refs [options] +``` -This command dumps the contents of a SerializedFile into a file of the selected format. It currently -only supports the 'text' format, which is similar to the binary2text output format. +| Option | Description | +|--------|-------------| +| `-i, --object-id` | Object ID to trace | +| `-n, --object-name` | Object name to trace | +| `-t, --object-type` | Type filter (with `-n`) | +| `-o, --output-file` | Output filename | +| `-a, --find-all` | Find all chains (slower) | -The command takes the path of the file to dump as argument. It also provides the following options: -* -o, --output-path Output folder, default is the current folder. -* -f, --output-format : output format, default is 'text'. -* -s, --skip-large-arrays: the contents of basic data type arrays with a large number of elements - won't be dumped. -* -i, --objectid : only dump the object with this local file id. If not specified or 0, all objects will be dumped. +**Example:** +```bash +UnityDataTool find-refs my_database.db -n "MyTexture" -t "Texture2D" -o refs.txt +``` -Example: `UnityDataTool dump /path/to/file -o /path/to/output` +📖 [Full documentation](Commands/find-refs.md) — Use cases, output format -If you pass an Archive file to this command, it will dump the contents of all the SerializedFiles inside. +--- -As an example, suppose you have a AssetBundle "scenes.bundle" that contains "SampleScene.unity" and "Scene2.unity". +## Installation -Running this command: -``` -UnityDataTool dump scenes.bundle -``` +### Building -writes out these text dumps to the current directory: +First, build the solution as described in the [main README](../README.md#how-to-build). +The executable will be at: ``` -BuildPlayer-SampleScene.sharedAssets.txt -BuildPlayer-SampleScene.txt -BuildPlayer-Scene2.sharedAssets.txt -BuildPlayer-Scene2.txt +UnityDataTool/bin/Release/net9.0/UnityDataTool.exe ``` -**Refer to this [documentation](../TextDumper/README.md#How-to-interpret-the-output-files) for more -information about the contents of the output file.** +> **Tip:** Add the directory containing `UnityDataTool.exe` to your `PATH` environment variable for easy access. -## archive - -The archive command offers a set of sub-commands related to Unity archives (AssetBundles and web platform .data files). - -**list** This sub-command lists the contents of an archive. It takes the archive path as argument. +### Mac Instructions -**extract** This sub-command extracts the contents of an archive. This is similar to the WebExtract tool that is part of the Unity installation. -It takes the archive path as argument and also accepts the following option: -* -o, --output-path \: Output directory of the extracted archive (default: archive) +On Mac, publish the project to get an executable: -As an example, suppose you have a AssetBundle "scenes.bundle" that contains "SampleScene.unity" and "Scene2.unity". - -Running this command: -``` -UnityDataTool archive extract scenes.bundle -o contents +**Intel Mac:** +```bash +dotnet publish UnityDataTool -c Release -r osx-x64 -p:PublishSingleFile=true -p:UseAppHost=true ``` -write out these 4 files: - -``` -contents/BuildPlayer-SampleScene.sharedAssets -contents/BuildPlayer-SampleScene -contents/BuildPlayer-Scene2.sharedAssets -contents/BuildPlayer-Scene2 +**Apple Silicon Mac:** +```bash +dotnet publish UnityDataTool -c Release -r osx-arm64 -p:PublishSingleFile=true -p:UseAppHost=true ``` -Note: When using this command the files are not transformed into text dumps, e.g. in this case they are the exact same binary SerializedFiles that are inside the AssetBundle. See also the `dump` command. +If you see a warning about `UnityFileSystemApi.dylib` not being verified, go to **System Preferences → Security & Privacy** and allow the file. -## find-refs +--- + +## Related Documentation -> Note: this is an experimental command, it may not work as expected. - -This command finds reference chains leading to specific objects. It requires a database that was -created by the 'analyze' command without the --skip-references option. It takes an object id or -name as input and will find reference chains originating from a root asset to the specified object -(s). A root asset is an asset that was explicitly added to an AssetBundle at build time. It can be -particularly useful to determine why an asset was included (and potentially more than once) in a -build. - -The command takes the path of the database as argument. It also provides the following options: -* -i, --object-id \: the id of the object to analyze ('id' column in the database). -* -n, --object-name \: name of the objects to analyze (it can be useful to find the origin - of duplicates as they will have different ids but the same name). -* -t, --object-type \: type of the objects to analyze, used to filter objects when using - the -n option. -* -o, --output-file \: name of the output file. -* -a, --find-all: this will force a search for all reference chains originating from the same root. - object instead of stopping at the first one. It may take a lot more time. Note that - either --object-id or --object-name must be provided. - -Example: `UnityDataTool find-refs my_database.db -n "MyObjectName" -t "Texture2D" -o -references.txt` - -**Refer to this [documentation](../ReferenceFinder/README.md#How-to-interpret-the-output-file) for -more information about the contents of the output file.** +| Topic | Description | +|-------|-------------| +| [Analyzer Database Reference](../Analyzer/README.md) | SQLite schema, views, and extending the analyzer | +| [TextDumper Output Format](../TextDumper/README.md) | Understanding dump output | +| [ReferenceFinder Details](../ReferenceFinder/README.md) | Reference chain output format | +| [Analyze Examples](../Documentation/analyze-examples.md) | Practical database queries | +| [Comparing Builds](../Documentation/comparing-builds.md) | Strategies for build comparison | +| [Unity Content Format](../Documentation/unity-content-format.md) | TypeTrees and file formats | From 76c9489464ca49589dfd0d83d04a808c590a48f6 Mon Sep 17 00:00:00 2001 From: Andrew Skowronski Date: Fri, 19 Dec 2025 09:27:30 -0500 Subject: [PATCH 2/3] Some manual edits Remove duplicated reference section in main readme that would be hard to maintain A few edits based on review. --- UnityDataTool/Commands/analyze.md | 12 +-- UnityDataTool/Commands/archive.md | 4 +- UnityDataTool/Commands/dump.md | 6 +- UnityDataTool/README.md | 118 ++---------------------------- 4 files changed, 20 insertions(+), 120 deletions(-) diff --git a/UnityDataTool/Commands/analyze.md b/UnityDataTool/Commands/analyze.md index b48aacd..67a4dea 100644 --- a/UnityDataTool/Commands/analyze.md +++ b/UnityDataTool/Commands/analyze.md @@ -24,7 +24,7 @@ Analyze all files in a directory: UnityDataTool analyze /path/to/asset/bundles ``` -Analyze only `.bundle` files and save to a custom database: +Analyze only `.bundle` files and specify a custom database name: ```bash UnityDataTool analyze /path/to/asset/bundles -o my_database.db -p "*.bundle" ``` @@ -34,6 +34,8 @@ Fast analysis (skip reference tracking): UnityDataTool analyze /path/to/bundles -s ``` +See also [Analyze Examples](../../Documentation/analyze-examples.md). + --- ## What Can Be Analyzed @@ -43,10 +45,12 @@ The analyze command works with the following types of directories: | Input Type | Description | |------------|-------------| | **AssetBundle build output** | The output path of an AssetBundle build | -| **Addressables folder** | `StreamingAssets/aa` folder from a Player build | +| **Addressables folder** | `StreamingAssets/aa` folder from a Player build, including BuildLayout files | | **Entities content** | `StreamingAssets/ContentArchives` folder for [Entities](https://docs.unity3d.com/Packages/com.unity.entities@1.4/manual/content-management-intro.html) projects | | **Player Data folder** | The `Data` folder of a Unity Player build | | **Compressed Player builds** | The `data.unity3d` file will be analyzed like AssetBundles | +| **BuildReport files** | The build report is typically found at a path like `Library/LastBuild.buildreport`and is a binary serialized file | +| **AssetDatabase Artifacts** | The tool will work to some extent with serialized files created in the AssetDatabase artifact storage, inside the Library folder | > **Note**: Some platforms require extracting content from platform-specific containers first (e.g., `.apk` files on Android). @@ -56,9 +60,7 @@ The analyze command works with the following types of directories: The analysis creates a SQLite database that can be explored using tools like [DB Browser for SQLite](https://sqlitebrowser.org/) or the command line `sqlite3` tool. -**Refer to the [Analyzer documentation](../../Analyzer/README.md) for complete database schema reference and usage examples.** - -See also: [Analyze Examples](../../Documentation/analyze-examples.md) +**Refer to the [Analyzer documentation](../../Analyzer/README.md) for the database schema reference and information about extending this command.** --- diff --git a/UnityDataTool/Commands/archive.md b/UnityDataTool/Commands/archive.md index 0057ad8..80e59f5 100644 --- a/UnityDataTool/Commands/archive.md +++ b/UnityDataTool/Commands/archive.md @@ -66,8 +66,8 @@ contents/BuildPlayer-Scene2 | Command | Output | Use Case | |---------|--------|----------| -| `archive extract` | Binary SerializedFiles | When you need the raw files | -| `dump` | Human-readable text | When you need to inspect content | +| `archive extract` | Binary SerializedFiles, .resS anything else inside the archive content | When you need all the raw files inside an archive | +| `dump` | text | When you want to inspect object content | The `dump` command can directly process archives without extracting first. diff --git a/UnityDataTool/Commands/dump.md b/UnityDataTool/Commands/dump.md index b721675..398775b 100644 --- a/UnityDataTool/Commands/dump.md +++ b/UnityDataTool/Commands/dump.md @@ -103,5 +103,9 @@ PPtrs (Property Pointers) are Unity's mechanism for referencing objects: | `m_FileID` | Index into External References list (0 = same file) | | `m_PathID` | Object's Local File Identifier (LFID) in that file | -Use external references to resolve cross-file references. +A null reference will have value m_FileID = 0, m_PathID = 0 + +The external reference table is used to resolve cross-file references. It always starts at index 1. m_FileID 0 is used for references within the current file. + + diff --git a/UnityDataTool/README.md b/UnityDataTool/README.md index db9243e..b47c70a 100644 --- a/UnityDataTool/README.md +++ b/UnityDataTool/README.md @@ -4,12 +4,12 @@ A command-line tool for analyzing and inspecting Unity build output—AssetBundl ## Commands -| Command | Description | Documentation | -|---------|-------------|---------------| -| [`analyze`](#analyze) | Extract data from Unity files into a SQLite database | [Full docs →](Commands/analyze.md) | -| [`dump`](#dump) | Convert SerializedFiles to human-readable text | [Full docs →](Commands/dump.md) | -| [`archive`](#archive) | List or extract contents of Unity Archives | [Full docs →](Commands/archive.md) | -| [`find-refs`](#find-refs) | Trace reference chains to objects *(experimental)* | [Full docs →](Commands/find-refs.md) | +| Command | Description | +|---------|-------------| +| [`analyze`](Commands/analyze.md) | Extract data from Unity files into a SQLite database | +| [`dump`](Commands/dump.md) | Convert SerializedFiles to human-readable text | +| [`archive`](Commands/archive.md) | List or extract contents of Unity Archives | +| [`find-refs`](Commands/find-refs.md) | Trace reference chains to objects *(experimental)* | --- @@ -36,112 +36,6 @@ Use `--help` with any command for details: `UnityDataTool analyze --help` Use `--version` to print the tool version. ---- - -## analyze - -Extract information from Unity Archives and SerializedFiles into a SQLite database. - -```bash -UnityDataTool analyze [options] -``` - -| Option | Description | Default | -|--------|-------------|---------| -| `-o, --output-file` | Output database filename | `database.db` | -| `-p, --search-pattern` | File filter pattern (`*.bundle`) | `*` | -| `-s, --skip-references` | Skip CRC/reference extraction (faster) | — | -| `-v, --verbose` | Show detailed progress | — | -| `--no-recurse` | Don't recurse into subdirectories | — | - -**Example:** -```bash -UnityDataTool analyze /path/to/bundles -o my_database.db -p "*.bundle" -``` - -📖 [Full documentation](Commands/analyze.md) — Troubleshooting, database schema, example inputs - ---- - -## dump - -Convert SerializedFiles to human-readable text format. - -```bash -UnityDataTool dump [options] -``` - -| Option | Description | Default | -|--------|-------------|---------| -| `-o, --output-path` | Output folder | Current folder | -| `-f, --output-format` | Output format | `text` | -| `-s, --skip-large-arrays` | Skip large array contents | — | -| `-i, --objectid` | Only dump this object ID | All objects | - -**Example:** -```bash -UnityDataTool dump /path/to/file -o /output/path -``` - -📖 [Full documentation](Commands/dump.md) — Output format details, archive support - ---- - -## archive - -Work with Unity Archives (AssetBundles, `.data` files). - -### list - -```bash -UnityDataTool archive list -``` - -### extract - -```bash -UnityDataTool archive extract [options] -``` - -| Option | Description | Default | -|--------|-------------|---------| -| `-o, --output-path` | Output directory | `archive` | - -**Example:** -```bash -UnityDataTool archive extract scenes.bundle -o contents/ -``` - -📖 [Full documentation](Commands/archive.md) — Sub-command details, comparison with dump - ---- - -## find-refs - -> ⚠️ Experimental - -Find reference chains leading to specific objects. Requires a database from `analyze` (without `--skip-references`). - -```bash -UnityDataTool find-refs [options] -``` - -| Option | Description | -|--------|-------------| -| `-i, --object-id` | Object ID to trace | -| `-n, --object-name` | Object name to trace | -| `-t, --object-type` | Type filter (with `-n`) | -| `-o, --output-file` | Output filename | -| `-a, --find-all` | Find all chains (slower) | - -**Example:** -```bash -UnityDataTool find-refs my_database.db -n "MyTexture" -t "Texture2D" -o refs.txt -``` - -📖 [Full documentation](Commands/find-refs.md) — Use cases, output format - ---- ## Installation From 6f060c8285d46dcd839c80b44cd410ecef098a4c Mon Sep 17 00:00:00 2001 From: Andrew Skowronski Date: Fri, 19 Dec 2025 09:40:48 -0500 Subject: [PATCH 3/3] Add section about the impact of "--skip-references" --- UnityDataTool/Commands/analyze.md | 22 ++++++++++++++++++++++ 1 file changed, 22 insertions(+) diff --git a/UnityDataTool/Commands/analyze.md b/UnityDataTool/Commands/analyze.md index 67a4dea..15a6c31 100644 --- a/UnityDataTool/Commands/analyze.md +++ b/UnityDataTool/Commands/analyze.md @@ -112,3 +112,25 @@ These errors occur when the same serialized file name appears in multiple source See [Comparing Builds](../../Documentation/comparing-builds.md) for strategies to compare different versions of builds. +### Slow Analyze times, large output database + +Consider using the `--skip-references` argument. + +A real life analyze of a big Addressables build shows how large a difference this can make: + +* 208 seconds and producted a 500MB database (not specifying --skip-reference) +* 9 seconds and produced a 68 MB file (with --skip-reference) + +The references are not needed for core asset inventory and size information. + +Note: When specifying `--skip-reference` some functionality is lost: + +* the `find-refs` command will not work +* `view_material_shader_refs` and `view_material_texture_refs` will be empty +* Queries that look at the relationship between objects will not work. For example the refs table is required to link between a `MonoBehaviour` and its `MonoScript`. +* The `objects.crc32` column will be NULL/0 for all objects. This means: + * No detection of identical objects by content hash (See [Comparing Builds](../../Documentation/comparing-builds.md)) + * The `view_potential_duplicates` view relies partially on CRC32 to distinguish true duplicates + +Future work: The refs table contains a lot of repeated strings and could be made smaller and more efficient. It might also be prudent to control the CRC32 calculation using an independent flag. +