Skip to content

Stop merging ETL files during samply record on Windows - #299

Merged
mstange merged 5 commits into
mainfrom
more-unmerged-windows-stuff
Jul 15, 2024
Merged

mstange merged 5 commits into
mainfrom
more-unmerged-windows-stuff

Conversation

@mstange

@mstange mstange commented Jul 15, 2024

Copy link
Copy Markdown
Owner

This improves the performance of stopping the recording. It cuts the post-processing time almost in half.

mstange added 3 commits July 15, 2024 16:08
Instead, make functions available for recycling immediately.

Background:
A jitdump file may have multiple functions with the same name and the same code size.
For example, this can happen if the same jitted function gets recompiled with
different type information, or after it was GC'ed, or because the same script was
recompiled after a page navigation or page reload.
In the profiler assembly view, you may want to see the different assembly code for
the different recompilations.

When you want to collect more samples by running the workload multiple times,
and you relaunch the process, then each process creates its own Jitdump file.
We treat each Jitdump file as a 'library'. The Firefox Profiler creates separate
call nodes in the call tree for functions from separate libraries, even if those
functions have the same name. This means that, in order for the multi-iteration
multi-process --reuse-threads scenario to work properly, we somehow have to make
it so that function A from process X's jitdump file and function A from process
Y's jitdump file are treated as being from the same library. This is what the
JitFunctionRecycler is for: For function A from process Y, it will reuse the
information for function A from process X, making it look like both functions were
from the library we create for process X's jitdump file.

Now if you open the assembly view on such a JIT function, the sample count next
to each instruction will be an accumulated sample count over multiple rounds,
a combination of the sample counts from different processes and from different
compilations of the same function. The underlying gamble here is that each run
of the workload is similar enough that the generated assembly code is roughly the
same, modulo some built-in addresses and offsets.

So what happens if, within one process, you have multiple compilations of the
same JIT function with *different* generated assembly code, and they have the same
code size in bytes?
Before this commit, we would try to treat the separate compilations as separate
even in --reuse-threads mode, at least in the first process: The
JitFunctionRecycler would only try to match against functions from previous
processes, not from the current process.
After this commit, the JitFunctionRecycler will recycle functions even within the
same process. So you will no longer be able to see different assembly code for
recycled functions. But I think that's ok - the base assumption of --reuse-threads
is that the recompilations are similar enough that you don't need to see different
assembly code.
@mstange
mstange force-pushed the more-unmerged-windows-stuff branch from 0631a7c to dbb6a4b Compare July 15, 2024 20:08
@mstange
mstange enabled auto-merge July 15, 2024 20:09
mstange added 2 commits July 15, 2024 16:17
At the moment we always run `xperf -stop -d`, the `-d` is what makes
xperf create a "merged" trace. This is somewhat slow.

Instead, we want to process the two raw ETL files individually, and skip
the xperf merging step.

We want to process the kernel trace first, and then the user trace.
The kernel trace gives us process and thread starts and ends and their names.
The user trace gives us extra markers and jit lib mapping ops.
We want to put the information from the user trace in the right place
even if we only see it after we've processed the entirety of the kernel trace.

So we now store a bunch of process / thread data with associated timestamps
so that we can find the right place once we process the user trace, even
if the pids or tids have been reused.
This saves some time during `xperf -stop` because we now run it without
the `-d` argument.
@mstange
mstange force-pushed the more-unmerged-windows-stuff branch from dbb6a4b to 22e5ff7 Compare July 15, 2024 20:17
@mstange
mstange merged commit d33fcc9 into main Jul 15, 2024
mstange added a commit that referenced this pull request Jul 15, 2024
I introduced a bug in #299 which made it so that markers ended up
in the wrong place, because the second trace moved the reference
timestamp.

We need to make sure that all timestamps use the same reference
when they are converted to profile timestamps.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant

⌂ 换网址