Consider adding support for a binary container format

[mstange]

Currently our profile format uses plain JSON, often gzipped. However, using JSON is causing us some problems: We end up with files which are too large and which take too long to load.

For example, this profile is 119MB gzipped, 464MB uncompressed, and takes 900ms for TextDecoder.decode + JSON.parse. Here's a JSON size profile of its contents; this particular profile has a 246MB stackTable.

A lot of our profile data is stored in long arrays of numbers. It would save us a lot of pain if we didn't have to go through JSON strings to serialize/deserialize numbers to a file. It's just really inefficient when loading a profile:

1. First, we have to decode the Uint8Array with the file contents to a JS string, because JSON.parse only takes strings, not buffers. Decoding UTF-8 walks all the bytes.
2. Then we call JSON.parse on the JS string. This walks all the bytes again.

And for large profiles we even hit maximum string length restrictions (512MiB in V8).

It would be a lot nicer if we didn't have to "parse" those number arrays at all. For example, for stackTable.frame, we really just want an Int32Array.

I think we should consider adding support for a binary format which lets us get typed arrays (Int32Array, Float64Array) out of the profile data without walking the bytes, just by re-wrapping the underlying ArrayBuffer.

┆Issue is synchronized with this Jira Task

[mstange]

At the moment, when we discuss how to make profiles smaller, we have to worry both about three sizes:

• The compressed file size
• The uncompressed JSON size
• The size of the in-memory representation

With a binary format, the middle concern would go away. We'd only need to worry about the compressed file size and the size of the in-memory representation.

For example, should we default to compressing with zstd instead of gz? Yes, probably, but it doesn't change the time we spend in TextDecoder.decode and JSON.parse. Once we stop spending time in those, switching compression codecs has a much larger payoff.

[mstange]

I've prototyped a binary container format which has binary "slabs" for the typed array data, and which keeps using JSON for the object structure "around" typed arrays - the JSON is stored in a separate JSON slab, and it has placeholders that get substituted with the typed arrays from the separate slabs during decoding. The format is called JsonSlabs (JSLB).

Here's an npm package which supports encoding and decoding this format: https://www.npmjs.com/package/json-slabs

We could support loading from this format in addition to loading JSON. This would let creators of profiles (such as samply) output JsonSlabs files, and the front-end would be able to load them.
Today, we don't have any typed arrays in the raw profile. But we could start allowing typed arrays. For example, the frame column in RawStackTable could change its type from IndexIntoFrameTable[] to Int32Array | IndexIntoFrameTable[] (in a new format version) and then the creator of the profile file can put the stackTable.frame data in a separate slab, and it would immediately reduce the time spent in TextDecoder.decode + JSON.parse. And then we can look at using the JSLB format for profile upload / download. And we'd probably have a step that substitutes regular arrays with typed arrays during profile compaction.

This gives us the following path to reduce profile load times:

• Pick a column which can be a typed array. Some columns currently are Array<IndexIntoXYZTable | null> so we would need to change some of those to use a -1 sentinel instead of null.
• Allow a typed array (such as Int32Array) for that column in the type definition, and handle conversions in the right places. Bump the profile version.
• Keep doing this until most of the data is stored in typed arrays and the "skeleton JSON" is really small and fast to parse.

I think we should keep the ability to read and write regular JSON files, and we should share the same type definitions for the object structure between both JSON and JSLB serializations.

[mstange]

Why a new format?

I looked at some of the other binary container formats, but none of them were quite what I was looking for.

JsonSlabs hits the following points:

• Schema-less / self-describing object structure content, just like JSON (so that we don't have to generate a new parser every time we make a change to the shape) - unlike schema-based formats like flatbuffers
• Simple to parse / small dependency, mostly taking advantage of the existing JSON.parse in JS engines
• Zero-copy typed arrays - unlike compact streaming formats like Protobuf

Something like flexbuffers might work too, but it would be a bigger dependency and a more complicated format. Flexbuffers also currently doesn't appear to support zero-copy typed array mapping in its JS implementation.