Support single-file packaging format spec `.3dtiles` or `.3tz`

[jo-chemla]

Is your feature request related to a problem? Please describe.

There are multiple packaging formats for 3dtiles tilesets, which store the tiles + hierarchy content within a single dataset file. This makes it easier to copy/transfer tilesets (faster operations since there is a single large file rather than thousands of small files), usually reduces disk size (no minimum file allocation) etc. The two main existing formats are:

• .3tz which is equivalent to a ~zip of the tileset directory
• and .3dtiles which is an ~sqlite db with key unix path and content blob

Link to the Cesium proposal thread for a 3D Tiles packaging format specification. It seems that the .3dtiles is the one that cesium team is aiming for broader support. After a quick research, it does not seem that support for any of these packaging formats is yet implemented within an existing library 3dtiles (cesiumjs, UE plugin etc).

Describe the solution you'd like

It would be great if the 3DTilesRendererJS library would support one of these packaging formats at some point.

Describe alternatives you've considered

Both formats can be converted from/to via 3d-tiles-tools convert, so one can extract a standard tileset.json + hierarchy from a .3tz or .3dtiles file. Also, .3tz can be simply unzipped.

[gkjohnson]

What's your use case for this? Do you want to download the whole blob in the browser and grab the necessary content from it as needed? Or will the client application use the HTTP Range header to read specific portions of the file? How much of this is addressed by improvements to HTTP2 and the ability for browser to download multiple files over a single connection?

Either way this is something that can likely be added via plugin once a couple more hooks are added for the fetch calls.

[jo-chemla]

Use case would indeed be rather to use HTTP2 range requests to fetch through this single-file tileset - often such datasets are a few tens or hundreds of GBs, so loading the whole blob in browser would not be useful. This would be akin to replicating the way COG (cloud optimized geotiffs)/COPC (cloud optimized pointclouds) are fetched on-the-fly given the user viewpoint within the dataset by browser libraries like

• OpenLayers or geotiffjs/cog-explorer for COG
• Potree for COPC (or potree v2 format spec) (or not open-source copc-viewer from hobu team)
• or the v5.0 roadmap for loaders.gl to support COPC/COG etc

The closest already-live equivalent would be a potree scene loading a COPC or potree v2 dataset via range-requests, it runs really smoothly eg on french IGN datasets (requires an Allow-CORS plugin), example here

For the .3dtiles sqlite file, this might probably help to rely on sql.js-httpvfs (and accompanying blog post) which wraps a sqlite db behind workers performing range-request access on a virtual file-system. For .3tz, there might be existing js libs to access a specific file within the zip hierarchy.

[gkjohnson]

Use case would indeed be rather to use HTTP2 range requests to fetch through this single-file tileset

As far as I understand the Range header is supported in HTTP1, as well. If the use case is to just use range requests in the client and run the same number of requests then what's the benefit over just unpacking the tile set on the server and fetching using the typical means?

[jo-chemla]

Thanks again for the quick replies! When sending a 3dtiles tileset to the server, one can indeed either

• transfer the tileset directory which consists of hundreds of thousands of tiles,
• or pack, transfer the package, and unpack on the server.

Both operations incur an overhead - either when transferring loads of small tile files, depending on the filesystems, which is a lot slower than transferring a single same-size file, or during the packing/unpacking operation. These ops are indeed a one-time operation, but a pretty long one when compared to transferring a single package. There are also sometimes disk-size improvements when using a single package file rather than loads of tile files - I'll have to dig deeper to find the resource sharing that point of view, was probably related to COGs. Additional improvements include compression that can be added on top, although glb compression is already probably very efficient.

The Cesium discussions in the original proposal thread regarding supporting a single-file package standard for 3dtiles datasets originate from the same need that led to COG/COPC for a single-file dataset that combines, as the ecosystem libraries can read and parse such package-files:

• removing the need for these pack/unpack operations as long
• simplify dataset sharing and access,
• simplify the blob-storage/file server configuration, etc.

There does not seem at the moment to be a strong status-quo on the .3tz/.3dtiles discussion, both are used depending on the places, so the implementation should probably wait for clearer community support, just wanted to mention it and track it somewhere in your repo.

[gkjohnson]

Got it - so to be clear there are no run time benefits to this. This is purely a matter of making it easier to manage source files, is that right? If there's an official spec on how these kinds of archives are packed I'd be open to a contribution to support it via plugin.

[jo-chemla]

Indeed, no runtime benefits, just simpler to handle tilesets as a self-contained dataset file, like any massive image/pointcloud file stored as a cloud-optimized format. The Cesium proposal thread I mention links to a draft 1.0.0 3DTilesPackageSpecification. From my understanding, the 3tz specification is documented in this 3 years old repo, but from my understanding the cesium team is aiming for .3dtiles for long-term package support - I've asked them in the referenced thread.

[gkjohnson]

There's a PR in #1568 that adds in a plugin for 3tz, though I don't plan to merge it (explained in the issue). At this point I think it would be best for something like a 3tz or 3dtiles single-file format to be community-maintained plugins if anyone would like to take that up. In any case I'll close this for now.