Skip to content

Support reading profiles from JsonSlabs files#6037

Open
mstange wants to merge 2 commits into
firefox-devtools:mainfrom
mstange:push-pxlosymwykls
Open

Support reading profiles from JsonSlabs files#6037
mstange wants to merge 2 commits into
firefox-devtools:mainfrom
mstange:push-pxlosymwykls

Conversation

@mstange
Copy link
Copy Markdown
Contributor

@mstange mstange commented May 20, 2026

Deploy preview (load this example jslb in it)

Fixes #6024.

JsonSlabs is an alternative serialization format that's more efficient than JSON if the serialized object contains typed arrays.
See https://github.com/mstange/json-slabs/ for details.

Our profiles currently do not yet contain typed arrays. But with this in place, we can start converting more and more tables / columns to use typed arrays, and incrementally reap the efficiency benefits.

This patch only adds the reading. In the UI, uploading / downloading profiles still uses JSON.

For the profiler-edit node script, this patch also adds the "writing": If you run profiler-edit with -o some-filename-ending-in.jslb, then it will create a JSLB file (or a compressed JSLB file for .jslb.gz).


The profile format "structure" is still the same as before. The format version is still given by profile.meta.preprocessedProfileVersion. Loading a JSLB file with a profile.meta.preprocessedProfileVersion that's
higher than the currently known one will still complain, and attempt to reload the page or the service worker.

This reload behavior is the primary reason why I want to get the "reading" part taken care of first, before we start converting any parts of the profile to get efficiency wins: The sooner we land this, the more likely it will be that, in the future, if somebody has a newer JSLB file but an old cached profiler, they'll see a useful error and
get the auto-reload.

Without this patch, loading a JSLB file in the profiler gives the following error:

Error: Unserializing the profile failed: Error: The profile array
buffer could not be parsed as a UTF-8 string.


With all that said, it turns out there is already a case where this format gives an efficiency win, thanks to its "split out" feature that lets us put certain subtrees of the profile into separate JSON slabs: If the original JSON is larger 512MiB, we can split it into individual slabs which are each smaller than 512MiB, so the built-in JSON.parse will work and we don't have to use the slow streaming parse.

Example:
https://storage.googleapis.com/profiler-get-symbols-fixtures/large-speedometer3-profile.json.gz
expands to a 577.74MiB JSON file,
https://storage.googleapis.com/profiler-get-symbols-fixtures/large-speedometer3-profile.jslb.gz
expands to a 564.66MiB JSLB file (slightly smaller because compacting removed unused strings)

The JSLB file breaks down as follows:

idx  type          bytes  elements  path               
---  -------  ----------  --------  -------------------
  0  json      16.76 MiB         -  .                    (root)
  1  json     264.82 MiB         -  .shared.stackTable 
  2  json      91.01 MiB         -  .shared.frameTable 
  3  json       8.47 MiB         -  .shared.funcTable  
  4  json      28.80 MiB         -  .shared.stringArray
  5  json     154.80 MiB         -  .threads           
---  -------  ----------  --------  -------------------
     6 slabs  564.66 MiB

All of those individual JSON slabs are now under 512MiB.

Loading the JSON file, uses streaming parser: https://share.firefox.dev/3PcB9vK (19 seconds)
Loading the JSLB file, uses native JSON.parse: https://share.firefox.dev/3RkFcXs (2.3 seconds, 8.3x faster)


The example jslb next to the deploy preview link was generated as follows: yarn build-node-tools && node ./node-tools-dist/profiler-edit.js --from-hash 29apty4565r8dec0hrjfty68w88rexc6na6bn1r -o ~/Downloads/normal-profile.jslb.gz


Follow-ups:

  • Land Rust tools in the json-slabs repo to list .jslb contents / convert to JSON / expand to directory (I have most of this written already)
  • Allow uploading jslb files to profile storage
  • Add UI to the sharing panel to allow downloading as either json or jslb
  • Actually start migrating tables / columns to typed arrays
  • Change uploading to use jslb instead of json
  • Update samply / fxprof-processed-profile to support outputting to jslb

We'll want to add a binary serialization soon, but we'll also
keep the JSON serializatios as an option. This rename makes it
clear which one you're calling.
@mstange mstange requested review from canova and fatadel as code owners May 20, 2026 21:39
@codecov
Copy link
Copy Markdown

codecov Bot commented May 20, 2026

Codecov Report

❌ Patch coverage is 81.81818% with 4 lines in your changes missing coverage. Please review.
✅ Project coverage is 83.80%. Comparing base (41473d3) to head (206f8ff).
⚠️ Report is 12 commits behind head on main.

Files with missing lines Patch % Lines
src/node-tools/profiler-edit.ts 71.42% 4 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main    #6037      +/-   ##
==========================================
+ Coverage   83.77%   83.80%   +0.02%     
==========================================
  Files         329      329              
  Lines       34423    34567     +144     
  Branches     9627     9570      -57     
==========================================
+ Hits        28839    28969     +130     
- Misses       5155     5169      +14     
  Partials      429      429              

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

Fixes firefox-devtools#6024.

JsonSlabs is an alternative serialization format that's more
efficient than JSON if the serialized object contains typed arrays.
See https://github.com/mstange/json-slabs/ for details.

Our profiles currently do not yet contain typed arrays. But with this
in place, we can start converting more and more tables / columns to
use typed arrays, and incrementally reap the efficiency benefits.

This patch only adds the reading. In the UI, uploading / downloading
profiles still uses JSON.

For the profiler-edit node script, this patch also adds the "writing":
If you run profiler-edit with `-o some-filename-ending-in.jslb`, then
it will create a JSLB file (or a compressed JSLB file for .jslb.gz).

---

The profile format "structure" is still the same as before. The format
version is still given by profile.meta.preprocessedProfileVersion.
Loading a JSLB file with a profile.meta.preprocessedProfileVersion that's
higher than the currently known one will still complain, and attempt to
reload the page or the service worker.

This reload behavior is the primary reason why I want to get the
"reading" part taken care of first, before we start converting any
parts of the profile to get efficiency wins: The sooner we land this,
the more likely it will be that, in the future, if somebody has a newer
JSLB file but an old cached profiler, they'll see a useful error and
get the auto-reload.

Without this patch, loading a JSLB file in the profiler gives the
following error:

> Error: Unserializing the profile failed: Error: The profile array
> buffer could not be parsed as a UTF-8 string.

---

With all that said, it turns out there is already a case where this
format gives an efficiency win, thanks to its "split out" feature
that lets us put certain subtrees of the profile into separate JSON
slabs: If the original JSON is larger 512MiB, we can split it into
individual slabs which are each smaller than 512MiB, so the built-in
JSON.parse will work and we don't have to use the slow streaming parse.

Example:
https://storage.googleapis.com/profiler-get-symbols-fixtures/large-speedometer3-profile.json.gz
expands to a 577.74MB JSON file,
https://storage.googleapis.com/profiler-get-symbols-fixtures/large-speedometer3-profile.jslb.gz
expands to a 564.66MB JSLB file (slightly smaller because compacting removed unused strings)

The JSLB file breaks down as follows:

```
idx  type          bytes  elements  path               
---  -------  ----------  --------  -------------------
  0  json      16.76 MiB         -  .                    (root)
  1  json     264.82 MiB         -  .shared.stackTable 
  2  json      91.01 MiB         -  .shared.frameTable 
  3  json       8.47 MiB         -  .shared.funcTable  
  4  json      28.80 MiB         -  .shared.stringArray
  5  json     154.80 MiB         -  .threads           
---  -------  ----------  --------  -------------------
     6 slabs  564.66 MiB
```

All of those individual JSON slabs are now under 512MiB.

Loading the JSON file, uses streaming parser: https://share.firefox.dev/3PcB9vK (19 seconds)
Loading the JSLB file, uses native JSON.parse: https://share.firefox.dev/3RkFcXs (2.3 seconds, 8.3x faster)
@mstange mstange force-pushed the push-pxlosymwykls branch from d7175f8 to 206f8ff Compare May 20, 2026 21:49
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Consider adding support for a binary container format

1 participant