Skip to content

Commit fd85a23

Browse files
SkowronskiAndrewSkowronskiAndrew
authored
Documentation revamp - Established individual topic for each UnityDataTool command (#43)
Split up the README file for UnityDataTool so that its easier to see the list of commands Establish more structure to the documentation comment.
1 parent 5d86f8f commit fd85a23

File tree

5 files changed

+465
-195
lines changed

5 files changed

+465
-195
lines changed

‎UnityDataTool/Commands/analyze.md‎

Lines changed: 136 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,136 @@
1+
# analyze Command
2+
3+
The `analyze` command extracts information from Unity Archives (e.g. AssetBundles) and SerializedFiles and dumps the results into a SQLite database.
4+
5+
## Quick Reference
6+
7+
```
8+
UnityDataTool analyze <path> [options]
9+
```
10+
11+
| Option | Description | Default |
12+
|--------|-------------|---------|
13+
| `<path>` | Path to folder containing files to analyze | *(required)* |
14+
| `-o, --output-file <file>` | Output database filename | `database.db` |
15+
| `-p, --search-pattern <pattern>` | File search pattern (`*` and `?` supported) | `*` |
16+
| `-s, --skip-references` | Skip CRC and reference extraction (faster, smaller DB) | `false` |
17+
| `-v, --verbose` | Show more information during analysis | `false` |
18+
| `--no-recurse` | Do not recurse into sub-directories | `false` |
19+
20+
## Examples
21+
22+
Analyze all files in a directory:
23+
```bash
24+
UnityDataTool analyze /path/to/asset/bundles
25+
```
26+
27+
Analyze only `.bundle` files and specify a custom database name:
28+
```bash
29+
UnityDataTool analyze /path/to/asset/bundles -o my_database.db -p "*.bundle"
30+
```
31+
32+
Fast analysis (skip reference tracking):
33+
```bash
34+
UnityDataTool analyze /path/to/bundles -s
35+
```
36+
37+
See also [Analyze Examples](../../Documentation/analyze-examples.md).
38+
39+
---
40+
41+
## What Can Be Analyzed
42+
43+
The analyze command works with the following types of directories:
44+
45+
| Input Type | Description |
46+
|------------|-------------|
47+
| **AssetBundle build output** | The output path of an AssetBundle build |
48+
| **Addressables folder** | `StreamingAssets/aa` folder from a Player build, including BuildLayout files |
49+
| **Entities content** | `StreamingAssets/ContentArchives` folder for [Entities](https://docs.unity3d.com/Packages/[email protected]/manual/content-management-intro.html) projects |
50+
| **Player Data folder** | The `Data` folder of a Unity Player build |
51+
| **Compressed Player builds** | The `data.unity3d` file will be analyzed like AssetBundles |
52+
| **BuildReport files** | The build report is typically found at a path like `Library/LastBuild.buildreport`and is a binary serialized file |
53+
| **AssetDatabase Artifacts** | The tool will work to some extent with serialized files created in the AssetDatabase artifact storage, inside the Library folder |
54+
55+
> **Note**: Some platforms require extracting content from platform-specific containers first (e.g., `.apk` files on Android).
56+
57+
---
58+
59+
## Output Database
60+
61+
The analysis creates a SQLite database that can be explored using tools like [DB Browser for SQLite](https://sqlitebrowser.org/) or the command line `sqlite3` tool.
62+
63+
**Refer to the [Analyzer documentation](../../Analyzer/README.md) for the database schema reference and information about extending this command.**
64+
65+
---
66+
67+
## Troubleshooting
68+
69+
### File Loading Warnings
70+
71+
```
72+
Failed to load 'C:\....\MyData.db'. File may be corrupted or was serialized with a newer version of Unity.
73+
```
74+
75+
These warnings occur when the tool encounters non-Unity files in the analyzed directory. They are usually harmless—the analyze process continues and produces a valid database.
76+
77+
**Solutions:**
78+
- Use `-p "*.bundle"` to filter by file extension
79+
- Use `--no-recurse` to limit directory depth
80+
- Use `-v` (verbose) to see which files are ignored
81+
82+
The tool automatically ignores common non-Unity file types (`.txt`, `.json`, `.manifest`, etc.).
83+
84+
### TypeTree Errors
85+
86+
```
87+
Error processing file: C:\...\TestProject_Data\level0
88+
System.ArgumentException: Invalid object id.
89+
```
90+
91+
This error occurs when SerializedFiles are built without TypeTrees. The command will skip these files and continue.
92+
93+
**Solution:** Enable **ForceAlwaysWriteTypeTrees** in your Unity build settings. See [Unity Content Format](../../Documentation/unity-content-format.md) for details.
94+
95+
### SQL Constraint Errors
96+
97+
```
98+
SQLite Error 19: 'UNIQUE constraint failed: objects.id'
99+
```
100+
or
101+
```
102+
SQLite Error 19: 'UNIQUE constraint failed: serialized_files.id'.
103+
```
104+
105+
These errors occur when the same serialized file name appears in multiple sources:
106+
107+
| Cause | Solution |
108+
|-------|----------|
109+
| Multiple builds in same directory | Analyze each build separately |
110+
| Scenes with same filename (different paths) | Rename scenes to be unique |
111+
| AssetBundle variants | Analyze variants separately |
112+
113+
See [Comparing Builds](../../Documentation/comparing-builds.md) for strategies to compare different versions of builds.
114+
115+
### Slow Analyze times, large output database
116+
117+
Consider using the `--skip-references` argument.
118+
119+
A real life analyze of a big Addressables build shows how large a difference this can make:
120+
121+
* 208 seconds and producted a 500MB database (not specifying --skip-reference)
122+
* 9 seconds and produced a 68 MB file (with --skip-reference)
123+
124+
The references are not needed for core asset inventory and size information.
125+
126+
Note: When specifying `--skip-reference` some functionality is lost:
127+
128+
* the `find-refs` command will not work
129+
* `view_material_shader_refs` and `view_material_texture_refs` will be empty
130+
* Queries that look at the relationship between objects will not work. For example the refs table is required to link between a `MonoBehaviour` and its `MonoScript`.
131+
* The `objects.crc32` column will be NULL/0 for all objects. This means:
132+
* No detection of identical objects by content hash (See [Comparing Builds](../../Documentation/comparing-builds.md))
133+
* The `view_potential_duplicates` view relies partially on CRC32 to distinguish true duplicates
134+
135+
Future work: The refs table contains a lot of repeated strings and could be made smaller and more efficient. It might also be prudent to control the CRC32 calculation using an independent flag.
136+

‎UnityDataTool/Commands/archive.md‎

Lines changed: 73 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,73 @@
1+
# archive Command
2+
3+
The `archive` command provides utilities for working with Unity Archives (AssetBundles and web platform `.data` files).
4+
5+
## Sub-Commands
6+
7+
| Sub-Command | Description |
8+
|-------------|-------------|
9+
| [`list`](#list) | List contents of an archive |
10+
| [`extract`](#extract) | Extract contents of an archive |
11+
12+
---
13+
14+
## list
15+
16+
Lists the SerializedFiles contained within an archive.
17+
18+
### Quick Reference
19+
20+
```
21+
UnityDataTool archive list <archive-path>
22+
```
23+
24+
### Example
25+
26+
```bash
27+
UnityDataTool archive list scenes.bundle
28+
```
29+
30+
---
31+
32+
## extract
33+
34+
Extracts the contents of an archive to disk. This is similar to Unity's `WebExtract` tool.
35+
36+
### Quick Reference
37+
38+
```
39+
UnityDataTool archive extract <archive-path> [options]
40+
```
41+
42+
| Option | Description | Default |
43+
|--------|-------------|---------|
44+
| `<archive-path>` | Path to the archive file | *(required)* |
45+
| `-o, --output-path <path>` | Output directory | `archive` |
46+
47+
### Example
48+
49+
```bash
50+
UnityDataTool archive extract scenes.bundle -o contents
51+
```
52+
53+
**Output files:**
54+
```
55+
contents/BuildPlayer-SampleScene.sharedAssets
56+
contents/BuildPlayer-SampleScene
57+
contents/BuildPlayer-Scene2.sharedAssets
58+
contents/BuildPlayer-Scene2
59+
```
60+
61+
> **Note:** The extracted files are binary SerializedFiles, not text. Use the [`dump`](dump.md) command to convert them to readable text format.
62+
63+
---
64+
65+
## Comparison: extract vs dump
66+
67+
| Command | Output | Use Case |
68+
|---------|--------|----------|
69+
| `archive extract` | Binary SerializedFiles, .resS anything else inside the archive content | When you need all the raw files inside an archive |
70+
| `dump` | text | When you want to inspect object content |
71+
72+
The `dump` command can directly process archives without extracting first.
73+

‎UnityDataTool/Commands/dump.md‎

Lines changed: 111 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,111 @@
1+
# dump Command
2+
3+
The `dump` command converts Unity SerializedFiles into human-readable text format. This is useful for inspecting the internal structure and properties of Unity assets.
4+
5+
## Quick Reference
6+
7+
```
8+
UnityDataTool dump <path> [options]
9+
```
10+
11+
| Option | Description | Default |
12+
|--------|-------------|---------|
13+
| `<path>` | Path to file to dump | *(required)* |
14+
| `-o, --output-path <path>` | Output folder | Current folder |
15+
| `-f, --output-format <format>` | Output format | `text` |
16+
| `-s, --skip-large-arrays` | Skip dumping large arrays | `false` |
17+
| `-i, --objectid <id>` | Only dump object with this ID | All objects |
18+
19+
## Examples
20+
21+
Dump all objects in a file to the current directory:
22+
```bash
23+
UnityDataTool dump /path/to/file
24+
```
25+
26+
Dump to a specific output folder:
27+
```bash
28+
UnityDataTool dump /path/to/file -o /path/to/output
29+
```
30+
31+
Dump a single object by ID:
32+
```bash
33+
UnityDataTool dump /path/to/file -i 1234567890
34+
```
35+
36+
Skip large arrays for cleaner output:
37+
```bash
38+
UnityDataTool dump /path/to/file -s
39+
```
40+
41+
---
42+
43+
## Archive Support
44+
45+
When you pass an Archive file (like an AssetBundle), the command dumps all SerializedFiles inside.
46+
47+
**Example:** For an AssetBundle `scenes.bundle` containing two scenes:
48+
49+
```bash
50+
UnityDataTool dump scenes.bundle
51+
```
52+
53+
**Output files:**
54+
```
55+
BuildPlayer-SampleScene.sharedAssets.txt
56+
BuildPlayer-SampleScene.txt
57+
BuildPlayer-Scene2.sharedAssets.txt
58+
BuildPlayer-Scene2.txt
59+
```
60+
61+
---
62+
63+
## Output Format
64+
65+
The output is similar to Unity's `binary2text` tool. Each file begins with external references:
66+
67+
```
68+
External References
69+
path(1): "Library/unity default resources" GUID: 0000000000000000e000000000000000 Type: 0
70+
path(2): "Resources/unity_builtin_extra" GUID: 0000000000000000f000000000000000 Type: 0
71+
path(3): "archive:/CAB-35fce856128a6714740898681ea54bbe/..." GUID: 00000000000000000000000000000000 Type: 0
72+
```
73+
74+
Followed by object entries:
75+
76+
```
77+
ID: -8138362113332287275 (ClassID: 135) SphereCollider
78+
m_GameObject PPtr<GameObject>
79+
m_FileID int 0
80+
m_PathID SInt64 -1473921323670530447
81+
m_Material PPtr<PhysicMaterial>
82+
m_FileID int 0
83+
m_PathID SInt64 0
84+
m_IsTrigger bool False
85+
m_Enabled bool True
86+
m_Radius float 0.5
87+
m_Center Vector3f
88+
x float 0
89+
y float 0
90+
z float 0
91+
```
92+
93+
**Refer to the [TextDumper documentation](../../TextDumper/README.md) for detailed output format explanation.**
94+
95+
---
96+
97+
## Understanding PPtrs
98+
99+
PPtrs (Property Pointers) are Unity's mechanism for referencing objects:
100+
101+
| Field | Description |
102+
|-------|-------------|
103+
| `m_FileID` | Index into External References list (0 = same file) |
104+
| `m_PathID` | Object's Local File Identifier (LFID) in that file |
105+
106+
A null reference will have value m_FileID = 0, m_PathID = 0
107+
108+
The external reference table is used to resolve cross-file references. It always starts at index 1. m_FileID 0 is used for references within the current file.
109+
110+
111+

0 commit comments

Comments
 (0)