Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 8 additions & 2 deletions AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -78,6 +78,10 @@ UnityDataTool dump /path/to/file.bundle -o /output/path
# Extract archive contents
UnityDataTool archive extract file.bundle -o contents/

# Quick inspect SerializedFile metadata
UnityDataTool serialized-file objectlist level0
UnityDataTool sf externalrefs sharedassets0.assets --format json

# Find reference chains to an object
UnityDataTool find-refs database.db -n "ObjectName" -t "Texture2D"
```
Expand Down Expand Up @@ -139,13 +143,15 @@ UnityDataTool (CLI executable)

**Entry Points**:
- `UnityDataTool/Program.cs` - CLI using System.CommandLine
- `UnityDataTool/Commands/` - Command handlers (Analyze.cs, Dump.cs, Archive.cs, FindReferences.cs)
- `Documentation/` - Command documentation (command-analyze.md, command-dump.md, command-archive.md, command-find-refs.md)
- `UnityDataTool/SerializedFileCommands.cs` - SerializedFile inspection handlers
- `UnityDataTool/Archive.cs` - Archive manipulation handlers
- `Documentation/` - Command documentation (command-analyze.md, command-dump.md, command-archive.md, command-serialized-file.md, command-find-refs.md)

**Core Libraries**:
- `UnityFileSystem/UnityFileSystem.cs` - Init(), MountArchive(), OpenSerializedFile()
- `UnityFileSystem/DllWrapper.cs` - P/Invoke bindings to native library
- `UnityFileSystem/SerializedFile.cs` - Represents binary data files
- `UnityFileSystem/TypeIdRegistry.cs` - Built-in TypeId to type name mappings
- `UnityFileSystem/RandomAccessReader.cs` - TypeTree property navigation

**Analyzer**:
Expand Down
7 changes: 6 additions & 1 deletion Analyzer/SQLite/Handlers/PackedAssetsHandler.cs
Original file line number Diff line number Diff line change
Expand Up @@ -64,7 +64,7 @@ public void Init(SqliteConnection db)
public void Process(Context ctx, long objectId, RandomAccessReader reader, out string name, out long streamDataSize)
{
var packedAssets = PackedAssets.Read(reader);

m_InsertPackedAssetsCommand.Transaction = ctx.Transaction;
m_InsertPackedAssetsCommand.Parameters["@id"].Value = objectId;
m_InsertPackedAssetsCommand.Parameters["@path"].Value = packedAssets.Path;
Expand Down Expand Up @@ -96,6 +96,11 @@ public void Process(Context ctx, long objectId, RandomAccessReader reader, out s
m_InsertContentsCommand.Transaction = ctx.Transaction;
m_InsertContentsCommand.Parameters["@packed_assets_id"].Value = objectId;
m_InsertContentsCommand.Parameters["@object_id"].Value = content.ObjectID;

// TODO: Ideally we would also populate the type table if the content.Type is
// not already in that table, and if we have a string value for it in TypeIdRegistry. That would
// make it possible to view object types as strings, for the most common types, when importing a BuildReport
// without the associated built content.
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

#55 tracks this "todo"

m_InsertContentsCommand.Parameters["@type"].Value = content.Type;
m_InsertContentsCommand.Parameters["@size"].Value = (long)content.Size;
m_InsertContentsCommand.Parameters["@offset"].Value = (long)content.Offset;
Expand Down
2 changes: 1 addition & 1 deletion Analyzer/SQLite/Parsers/SerializedFileParser.cs
Original file line number Diff line number Diff line change
Expand Up @@ -64,7 +64,7 @@ bool ShouldIgnoreFile(string file)
private static readonly HashSet<string> IgnoredExtensions = new()
{
".txt", ".resS", ".resource", ".json", ".dll", ".pdb", ".exe", ".manifest", ".entities", ".entityheader",
".ini", ".config", ".hash"
".ini", ".config", ".hash", ".md"
};

bool ProcessFile(string file, string rootDirectory)
Expand Down
2 changes: 1 addition & 1 deletion Analyzer/SerializedObjects/BuildReport.cs
Original file line number Diff line number Diff line change
Expand Up @@ -120,7 +120,7 @@ public static string GetBuildTypeString(int buildType)
{
1 => "Player",
2 => "AssetBundle",
3 => "Player, AssetBundle",
3 => "ContentDirectory",
_ => buildType.ToString()
};
}
Expand Down
181 changes: 181 additions & 0 deletions Documentation/command-serialized-file.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,181 @@
# serialized-file Command

The `serialized-file` command (alias: `sf`) provides utilities for quickly inspecting SerializedFile metadata without performing a full analysis.

## Sub-Commands

| Sub-Command | Description |
|-------------|-------------|
| [`externalrefs`](#externalrefs) | List external file references |
| [`objectlist`](#objectlist) | List all objects in the file |

---

## externalrefs

Lists the external file references (dependencies) in a SerializedFile. This shows which other files the SerializedFile depends on.

### Quick Reference

```
UnityDataTool serialized-file externalrefs <filename> [options]
UnityDataTool sf externalrefs <filename> [options]
```

| Option | Description | Default |
|--------|-------------|---------|
| `<filename>` | Path to the SerializedFile | *(required)* |
| `-f, --format <format>` | Output format: `Text` or `Json` | `Text` |

### Example - Text Output

```bash
UnityDataTool serialized-file externalrefs level0
```

**Output:**
```
Index: 1, Path: globalgamemanagers.assets
Index: 2, Path: sharedassets0.assets
Index: 3, Path: Library/unity default resources
```

### Example - JSON Output

```bash
UnityDataTool sf externalrefs sharedassets0.assets --format json
```

**Output:**
```json
[
{
"index": 1,
"path": "globalgamemanagers.assets",
"guid": "00000000000000000000000000000000",
"type": "NonAssetType"
},
{
"index": 2,
"path": "Library/unity default resources",
"guid": "0000000000000000e000000000000000",
"type": "NonAssetType"
}
]
```

---

## objectlist

Lists all objects contained in a SerializedFile, showing their IDs, types, offsets, and sizes.

### Quick Reference

```
UnityDataTool serialized-file objectlist <filename> [options]
UnityDataTool sf objectlist <filename> [options]
```

| Option | Description | Default |
|--------|-------------|---------|
| `<filename>` | Path to the SerializedFile | *(required)* |
| `-f, --format <format>` | Output format: `Text` or `Json` | `Text` |

### Example - Text Output

```bash
UnityDataTool sf objectlist sharedassets0.assets
```

**Output:**
```
Id Type Offset Size
------------------------------------------------------------------------------------------
1 PreloadData 83872 49
2 Material 83936 268
3 Shader 84208 6964
4 Cubemap 91184 240
5 MonoBehaviour 91424 60
6 MonoBehaviour 91488 72
```

### Example - JSON Output

```bash
UnityDataTool serialized-file objectlist level0 --format json
```

**Output:**
```json
[
{
"id": 1,
"typeId": 1,
"typeName": "GameObject",
"offset": 4864,
"size": 132
},
{
"id": 2,
"typeId": 4,
"typeName": "Transform",
"offset": 5008,
"size": 104
}
]
```

---

## Use Cases

### Quick File Inspection

Use `serialized-file` when you need quick information about a SerializedFile without generating a full SQLite database:

```bash
# Check what objects are in a file
UnityDataTool sf objectlist sharedassets0.assets

# Check file dependencies
UnityDataTool sf externalrefs level0
```

### Scripting and Automation

The JSON output format is ideal for scripts and automated processing:

```bash
# Extract object count
UnityDataTool sf objectlist level0 -f json | jq 'length'

# Find specific object types
UnityDataTool sf objectlist sharedassets0.assets -f json | jq '.[] | select(.typeName == "Material")'
```

---

## SerializedFile vs Archive

When working with AssetBundles (or a compressed Player build) you need to extract the contents first (with `archive extract`), then run the `serialized-file` command on individual files in the extracted output.

**Example workflow:**
```bash
# 1. List contents of an archive
UnityDataTool archive list scenes.bundle

# 2. Extract the archive
UnityDataTool archive extract scenes.bundle -o extracted/

# 3. Inspect individual SerializedFiles
UnityDataTool sf objectlist extracted/CAB-5d40f7cad7c871cf2ad2af19ac542994
```

---

## Notes

- This command only supports extracting information from the SerializedFile header of individual files. It does not extract detailed type-specific properties. Use `analyze` for full analysis of one or more SerializedFiles.
- The command uses the same native library (UnityFileSystemApi) as other UnityDataTool commands, ensuring consistent file reading across all Unity versions.

6 changes: 3 additions & 3 deletions Documentation/unity-content-format.md
Original file line number Diff line number Diff line change
Expand Up @@ -108,8 +108,8 @@ However in cases where you want to understand what contributes to the size your

Often the source of content can be easily inferred, based on your own knowledge of your project, and the names of objects. For example the name of a Shader should be unique, and typically has a filename that closely matches the Shader name.

You can also use the [BuildReport](https://docs.unity3d.com/Documentation/ScriptReference/Build.Reporting.BuildReport.html) for Player and AssetBundle builds (excluding Addressables). The [Build Report Inspector](https://github.com/Unity-Technologies/BuildReportInspector) is a tool to aid in analyzing that data.
You can include a Unity BuildReport file when running `UnityDataTools analyze`. This will import the PackedAsset information, tracking the source asset information for each object in the build output. See [Build Reports](./build-reports.md) for more information, including alternative ways to view the build report.

For AssetBundles built by [BuildPipeline.BuildAssetBundles()](https://docs.unity3d.com/ScriptReference/BuildPipeline.BuildAssetBundles.html), there is also source information available in the .manifest files for each bundle.
`UnityDataTools analyze` can also import Addressables build layout files, which include source asset information. See [Addressable Build Reports](./addressables-build-reports.md).

Addressables builds do not produce a BuildReport or .manifest files, but it offers similar build information in the user interface.
For AssetBundles built by [BuildPipeline.BuildAssetBundles()](https://docs.unity3d.com/ScriptReference/BuildPipeline.BuildAssetBundles.html) Unity creates a .manifest file for each AssetBundle that has source information. This is a text-base format.
5 changes: 5 additions & 0 deletions Documentation/unitydatatool.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,7 @@ A command-line tool for analyzing and inspecting Unity build output—AssetBundl
| [`analyze`](command-analyze.md) | Extract data from Unity files into a SQLite database |
| [`dump`](command-dump.md) | Convert SerializedFiles to human-readable text |
| [`archive`](command-archive.md) | List or extract contents of Unity Archives |
| [`serialized-file`](command-serialized-file.md) | Quick inspection of SerializedFile metadata |
| [`find-refs`](command-find-refs.md) | Trace reference chains to objects *(experimental)* |

---
Expand All @@ -28,6 +29,10 @@ UnityDataTool dump /path/to/file.bundle -o /output/path
# Extract archive contents
UnityDataTool archive extract file.bundle -o contents/

# Quick inspect SerializedFile
UnityDataTool serialized-file objectlist level0
UnityDataTool sf externalrefs sharedassets0.assets --format json

# Find reference chains to an object
UnityDataTool find-refs database.db -n "ObjectName" -t "Texture2D"
```
Expand Down
3 changes: 3 additions & 0 deletions TestCommon/Data/PlayerNoTypeTree/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
This is a partial copy of the same build as PlayerWithTypeTrees, but with typetrees turned off.

Without typetrees the information that can be retrieved is quite limited.
Binary file added TestCommon/Data/PlayerNoTypeTree/level0
Binary file not shown.
Binary file added TestCommon/Data/PlayerNoTypeTree/level1
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
36 changes: 36 additions & 0 deletions TestCommon/Data/PlayerWithTypeTrees/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,36 @@
# Test data description

This is the content output of a Player build, made with Unity 6000.0.65f1.
The diagnostic switch to enable TypeTrees was enabled when the build was performed.

The project is very simple and intended to be used for precise tests of expected content.

## Content

The build includes two scene files:
* SceneWithReferences.unity (level0) uses MonoBehaviours to references BasicScriptableObject.asset and Asset2.asset
* SceneWithReferences2.unity (level1) uses MonoBehaviours to reference BasicScriptableObject.asset and ScriptableObjectWithSerializeReference.asset

Based on that sharing arrangement:
* sharedassets0.assets contains BasicScriptableObject.asset and Asset2.asset
* sharedassets1.assets contains ScriptableObjectWithSerializeReference.asset

There are also additional content
* globalgamemanager with the preference objects
* globalgamemanager.assets with assets referenced from the globalgamemanager file
* globalgamemanagers.assets.resS containing the splash screen referenced from globalgamemanager.assets

Note: The binaries, json files and other output that were also output from the player build are not checked in, because they are not needed by UnityDataTool.

## BuildReport

The LastBuild.buildreport file (created in the Library folder) has also been copied in.

## Scripting types

The MonoBehaviour used in level0 and level1 to reference the ScriptableObject is of type MonoBehaviourWithReference.

* BasicScriptableObject.asset and Asset2.asset are instances of the BasicScriptableObject class.
* ScriptableObjectWithSerializeReference.asset is an instance of the MyNamespace.ScriptableObjectWithSerializeReference class.


Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file added TestCommon/Data/PlayerWithTypeTrees/level0
Binary file not shown.
Binary file added TestCommon/Data/PlayerWithTypeTrees/level1
Binary file not shown.
Binary file not shown.

Large diffs are not rendered by default.

Binary file not shown.
Loading