Skip to content

Share stackTable, frameTable, funcTable, resourceTable and nativeSymbols between threads#5482

Open
mstange wants to merge 9 commits intofirefox-devtools:mainfrom
mstange:share-everything
Open

Share stackTable, frameTable, funcTable, resourceTable and nativeSymbols between threads#5482
mstange wants to merge 9 commits intofirefox-devtools:mainfrom
mstange:share-everything

Conversation

@mstange
Copy link
Contributor

@mstange mstange commented Jun 4, 2025

Sharing the string table happened in #5481.
This PR makes us share the rest, and completes issue #3918.

Only samples and markers are still per-thread. I don't expect this to change.
This includes allocation samples.

To do:

  • Fix type check and test failures
  • Upgrade the URL when upgrading the profile, to remap function indexes etc. in the URL
  • When compacting during upload, apply translation tables to redux state (e.g. update focus function indexes in URL)
  • Do something to address perf impact for computing the derived thread (transforms and call node table computation happen per thread and take longer if the tables are bigger because they now contain information for all threads, should be able to reuse more of the computed outputs in the thread derivation pipeline)
  • Memoize the computation of the implementation filtered thread across threads
  • Adjust indexes in transforms when merging two different profiles
  • Add a paragraph to CHANGELOG-formats.md for the new version.
  • Add a test for merging profiles with applied transforms where we actually check the filtered call tree
  • Add a test checking that IndexIntoSourceTable indexes in the redux state are updated after profile sanitizing
  • Add a test checking that collapse-resource transforms aren't broken if profile sanitizing adds elements to the funcTable

@vlasakm
Copy link

vlasakm commented Aug 10, 2025

This is really great! Sharing this data across threads works especially well for applications where there is a large number of threads, but just a few "thread pools" whose threads are all executing the same code.

Using this allowed me to employ better caching when creating a firefox profile, save on the profile size, etc.

It also made me greedy, and I attempted to visualize a rather large profile (1602 threads, 577108 stacks, 29760 frames). Previously my experience was that Firefox Profiler would choke on say more than 300+ threads, and I hoped this change with sharing could improve the situation. Now it loads, and the UI is still quite responsive, but the load of the profile is taking long 2 minutes 15 seconds (and Firefox prompts to kill the script, etc.).

90 % of the load time is spent is in computeCallNodeTable, which I believe may be called repeatedly for each thread (and the merged thread which includes all of them), even though all these calls would process the same stack table and frame table, now that they are shared.

I know very little about React/Redux, but I see that there's some concept of memoization and thread selectors, that is still used for call nodes, even though after this change to share the stack and frame table. Very naively, would it perhaps be possible to just compute and memoize the call node information once by not using the thread selector mechanism here? But there's a great chance I am missing all the needed context.

FWIW I was and am still using this branch for quite some time. I am not sure if I am able to contribute much (I even failed to rebase it on top of more recent changes), but it would be great to see this, perhaps after all the big work-in-progress changes land (like the Typescript migration).

@mstange
Copy link
Contributor Author

mstange commented Aug 10, 2025

Yes indeed, sharing the computed call node table across threads while properly handling per thread transforms is the next thing I want to work on when I get back to this task. Thanks for the encouragement and it's great to hear that you're seeing real improvements from it!

@mstange mstange force-pushed the share-everything branch 4 times, most recently from 24a4604 to 8d957f0 Compare January 29, 2026 23:25
@codecov
Copy link

codecov bot commented Jan 29, 2026

Codecov Report

❌ Patch coverage is 85.77406% with 170 lines in your changes missing coverage. Please review.
✅ Project coverage is 85.45%. Comparing base (89082f8) to head (cf780f4).
⚠️ Report is 3 commits behind head on main.

Files with missing lines Patch % Lines
src/app-logic/url-handling.ts 11.76% 60 Missing ⚠️
src/profile-logic/transforms.ts 29.41% 48 Missing ⚠️
src/profile-logic/profile-compacting.ts 87.96% 25 Missing and 1 partial ⚠️
src/profile-logic/sanitize.ts 88.60% 9 Missing ⚠️
src/reducers/code.ts 50.00% 6 Missing ⚠️
src/profile-logic/merge-compare.ts 96.66% 5 Missing ⚠️
src/profile-logic/profile-data.ts 94.73% 5 Missing ⚠️
src/reducers/url-state.ts 80.00% 5 Missing ⚠️
src/profile-logic/index-translation.ts 90.47% 2 Missing ⚠️
src/profile-logic/symbolication.ts 95.00% 2 Missing ⚠️
... and 2 more
Additional details and impacted files
@@            Coverage Diff             @@
##             main    #5482      +/-   ##
==========================================
- Coverage   85.65%   85.45%   -0.20%     
==========================================
  Files         319      320       +1     
  Lines       31343    31737     +394     
  Branches     8546     8708     +162     
==========================================
+ Hits        26846    27120     +274     
- Misses       4066     4186     +120     
  Partials      431      431              

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@mstange mstange force-pushed the share-everything branch 4 times, most recently from a5b1248 to 4b18e81 Compare January 30, 2026 18:40
canova added a commit that referenced this pull request Feb 2, 2026
Small things I noticed while working on #5482.
@mstange mstange marked this pull request as ready for review February 2, 2026 20:00
@mstange
Copy link
Contributor Author

mstange commented Feb 2, 2026

This is now ready for review.

@mstange mstange force-pushed the share-everything branch 2 times, most recently from ce546d0 to 0d1cc61 Compare February 2, 2026 21:18
mstange added a commit that referenced this pull request Feb 3, 2026
More small stuff that makes #5482 a tiny bit easier.
@mstange mstange requested a review from canova February 3, 2026 15:35
@mstange mstange force-pushed the share-everything branch 2 times, most recently from 51823ee to d1916d3 Compare February 3, 2026 16:59
mstange and others added 5 commits February 3, 2026 12:04
…Symbols across threads.

This makes func indexes etc valid globally instead of scoped per threads.
It also allows for smaller profiles because data doesn't have to be duplicated.
Symbolication is probably faster.

Selecting multiple threads, i.e. displaying a "merged thread", is now easier because
we no longer need to compute merged funcTables etc for the merged thread; the global
tables are already "merged".
This also also means that func index mapping during symbolication now works even if
multiple threads are selected. Fixes firefox-devtools#5703.

This profile version comes both with a profile upgrader and with a URL upgrader,
because we have to adjust any func and resource indexes for transforms in the URL.
We also need to adjust any indexes in the redux state after compacting,
for example funcTable or resourceTable indexes used by transforms.

This patch also makes us adjust the source index in the URL after compaction.
Strictly speaking this fixes an existing bug, because the source table
was already being compacted.

It also fixes the bug where we weren't keeping track of the oldFuncCount properly
when splitting functions into private-window and non-private-window instances,
and weren't updating collapse-resource transforms correctly. Fixes firefox-devtools#5793.
We compute the call node table based on the filtered thread, per thread.
But now that every thread contains the full stackTable with entries used
by all threads, this computation has become much more expensive.
But most of the time we are passing the same tables from the shared profile
info, so we can just do the work once and share it across threads.

This only fixes the case where none of the threads have a transform applied.
They'll end up passing the tables from the shared data and hit the memoization
cache.

If a thread has a transform, it'll pass different tables and not hit the cache.

We don't do anything to share work across two threads that happen to have the
same transforms.

This also doesn't fix the case where there's an implementation filter applied.

Here's how loading the profile https://share.firefox.dev/4bxDqu8 compares:

- Before sharing the tables: https://share.firefox.dev/4r0imRO (1.1 seconds)
- After sharing the tables, no global memoization: https://share.firefox.dev/4qNGp6m (1.8 seconds)
- After sharing the tables, with global memoization: https://share.firefox.dev/4rwkwso (1.3 seconds)
…threads.

Here's how loading the profile https://share.firefox.dev/4kbkjrU compares:

- Before sharing the tables: https://share.firefox.dev/4kh5i84 (1.1 seconds)
- After sharing the tables, no global memoization: https://share.firefox.dev/3NLn1IK (1.6 seconds)
- After sharing the tables, with global memoization: https://share.firefox.dev/3LNNuot (1.3 seconds)
Here's how loading the profile https://share.firefox.dev/4a0b88Y compares:

- Before sharing the tables: https://share.firefox.dev/4qeiVGu (1.1 seconds)
- After sharing the tables, no global memoization: https://share.firefox.dev/3NUEi27 (2.0 seconds)
- After sharing the tables, with global memoization: https://share.firefox.dev/4qcDQty (1.3 seconds)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants