Provide perfetto UI as a customizable component about perfetto HOT 24 CLOSED

We need to be able to navigate to a page from a url that contains a moniker of our own, then fetch a file from our own server. That could maybe be done in the way you're describing, if you can pass a url to open.

That one you could do today already, we do the same in our dashboards. You just fetch() the file and pass the blob via postMessage (it's super undocumented, but here's an example: https://jsfiddle.net/g7oktr0q/5/)

The other scenario is that we load local collections of logs that need to be normalized/merged into a single trace event file. That one is harder to accommodate, as we need to run our own normalization/merging code. We could do that in batch as a separate step, but we wanted to use the same tool for both scenarios.

This really depends on the full details. Merging N traces is in principle just a cat (traces are protobuf and are designed to be cat-ed together). However devil hides into details:

• Will you have colliding TIDs/PIDs?
• How about timestamps?

But maybe a semi-opaque component that can be hosted in a page and handed a trace file?

That works already (% doc): window.open() +postMessage is the way to go.
We are having a discussion on whether we should allow doing the same in an <iframe>.
Don't have a final answer there, the security model of iframes is a more subtle.

from perfetto.

Can you clarify which features you want to control as an embedder?
We have already a mode where you can communicate with the ui via windo.open() + postMessage.
RIght now the postMessage has only one API which is "please load this trace".
We could consider extending that postMessage API, but I'd like to understand how.

We can't make the all the internals of the UI a public API, that's unmaintainable

from perfetto.

Two things:

We need to be able to navigate to a page from a url that contains a moniker of our own, then fetch a file from our own server. That could maybe be done in the way you're describing, if you can pass a url to open.

The other scenario is that we load local collections of logs that need to be normalized/merged into a single trace event file. That one is harder to accommodate, as we need to run our own normalization/merging code. We could do that in batch as a separate step, but we wanted to use the same tool for both scenarios.

Totally get the surface area maintainability issue. But maybe a semi-opaque component that can be hosted in a page and handed a trace file?

from perfetto.

This really depends on the full details. Merging N traces is in principle just a cat (traces are protobuf and are designed to be cat-ed together). However devil hides into details:

In our case (a custom scenario), it isn't just merging, it's normalizing - we're ingesting text log files in various formats that we convert to trace event JSON format. The timestamps are in the same time space, PIDs/TIDs we map using trace event meta events.

We are having a discussion on whether we should allow doing the same in an <iframe>.
Don't have a final answer there, the security model of iframes is a more subtle.

That might work if there were a way to transfer the JSON across the iframe boundary. Haven't looked at iframes in years, maaybe that would work?

from perfetto.

That might work if there were a way to transfer the JSON across the iframe boundary. Haven't looked at iframes in years, maaybe that would work?

in the same ways it works for window.open, via postMessage.
the caveat is that we might add X-Frame-Options: DENY at some point in the future because of the security implications of having an iframe.
I can't speculate too much right know, there are options but we didn't take any decision yet (it boils down on whether the UI will have some oauth tokens for gdrive integration and the like)

from perfetto.

Sorry to resurrect an old issue but I'm trying to do something similar for WebPageTest as well and move the embedded trace viewer away from the vulcanized legacy viewer and to perfetto.

I can set the CORS header since I own the server that the traces are on but it looks like the CSP on ui.perfetto.dev blocks connecting to other hosts.

I can go down the window.open() embedding path and postmessage the trace files over but the main drawback with that is there is no way to share a permalink to a trace result (and it looks like iframe embedding is blocked so that won't work either).

If possible I'd rather not have to maintain a separate install of the UI on my domain and keep it up to date. Is there any way to loosen up the CSP to allow cross-domain connections or is there another way you can think of to get a permalink to work?

from perfetto.

I can set the CORS header since I own the server that the traces are on but it looks like the CSP on ui.perfetto.dev blocks connecting to other hosts.

yes IIRC the url= arg supports only fetch from GCS because of CSP. Out of curiosity, were are you contents coming from? Extending CSP is technically possible, although it's not a scalable solution.

can go down the window.open() embedding path and postmessage the trace files over but the main drawback with that is there is no way to share a permalink to a trace result (and it looks like iframe embedding is blocked so that won't work either).

Here's my thought: instead of completely denying the permalink from window.open traces, we could echo back the URL that the opener passes (via window.open, see below). That URL should point to a non-perfetto URL that would essentially guarantee to re-do the same window.open.

Recently extended the post_message_handler.ts to accept a .url argument (below)
Example:

 perfettoWindow.postMessage(perfetto: {title: 'Trace name', url: 'https://my-server.com/page_that_will_do_the_postmessage', buffer: arrayBuffer});

So the change required here would be: when a trace is pushed via window.open(), instead of completely disabling re-share, turn the "permalink" feature into something that echos back the original URL. The UI state won't be serialized (it's fixable but requires more work, and we are re-architecting that anyways so let's not go there) but the link to the trace would be valid. (@chromy)

@pmeenan would this fix your use case?

from perfetto.

The trace files are coming from http://www.webpagetest.org though any private instance will be serving them from their own domain (like https://webpagetest.httparchive.org).

I don't think echoing the url back will help much since the results pages themselvs are already sharable and permalinked.

i.e. Here is a test result for cnn.com: https://www.webpagetest.org/result/201012_DiA4_2264567da8abc26034bab714450d1382/

To the left of the waterfall is a link to download the trace file or view it in an embedded view. The viewer currently opens a build of perfetto hosted on the origin server so it can open the trace by URL and the view page URL along with the trace reference are in the URL bar and directly sharable: https://www.webpagetest.org/chrome/perfetto/index.html#!/viewer?url=https%3A%2F%2Fwww.webpagetest.org%2Fgetgzip.php%3Ftest%3D201012_DiA4_2264567da8abc26034bab714450d1382%26file%3D1_trace.json

(be patient, it's a pretty massive trace file)

from perfetto.

I don't think echoing the url back will help much since the results pages themselvs are already sharable and permalinked.

What I am suggesting is NOT echoing the url of the trace itself, rather echoing the URL of a page you own (so knows how too deal with fetch and cors) that will take care of doing the right window.open (so you are back to the nice UX of "one click to open the trace").

I am envisioning this.

1. You have a html page (can be fully client side, no need of server side code) like:
   https://www.webpagetest.org/open_trace_in_perfetto.html#trace_url=201012_DiA4_2264567da8abc26034bab714450d138

2. That page has some simple javascript that does something like:
   blob = fech(location.fragment)
   wnd = window.open(ui.perfetto.dev)
   wnd.postMessage({buffer: blob, url: location.href})

3. When you click on "share" on the perfetto UI, it will print out https://www.webpagetest.org/open_trace_in_perfetto.html#trace_url=201012_DiA4_2264567da8abc26034bab714450d1382
   That URL will bring you back to step 1 and will work effectively like permalink today.

Does it make sense?

from perfetto.

That might be workable but not a great user experience. It would need to go to a landing page of some kind that has a button on it that would open the actual trace viewer (otherwise the popover/under protections without user gesture would kick in).

That might still be workable but the experience of loading a large trace is also somewhat "less great". The trace needs to be fetched in it's entirety on the landing page and then postmessage'd to the new window while a urlfetcher load gives a nice progress indication at the top of perfetto as well.

Maybe something like this:

• Landing page starts with a grayed out button to launch the viewer
• Trace fetch runs immediately and provides progress information
• Once the trace fetch is complete then the button becomes available to open the perfetto UI
• Once perfetto loads, the in-memory trace data is posted to perfetto

It might be cleaner if there is a way to stream the data in chunks instead of having to post the whole thing. I imagine there are probably a few copies of the raw trace data kept in memory during the post otherwise.

from perfetto.

That might be workable but not a great user experience.

I know but I can't think to those many other options. If you have some alternative to propose I am all ears.
Unfortunately I don't think the answer here can be keep expanding the CSP policy

. It would need to go to a landing page of some kind that has a button on it that would open the actual trace viewer (otherwise the popover/under protections without user gesture would kick in).

You could do what most sites do, that is: tell people to allow popups in the browser for the domain if they can't be bothered clicking on an extra button.

It might be cleaner if there is a way to stream the data in chunks instead of having to post the whole thing. I imagine there are probably a few copies of the raw trace data kept in memory during the post otherwise.

Sure. That requires some change to the postmessage handler. Patches welcome

from perfetto.

Update: we are implementing the "echo-back URL" here: r.android.com/1469529

Example: https://jsfiddle.net/nahck81j/1/

see also https://perfetto.dev/docs/visualization/deep-linking-to-perfetto-ui

from perfetto.

Awesome, thanks.

from perfetto.

@primiano

We are having a discussion on whether we should allow doing the same in an <iframe>.

This would definitely be useful, since the "Load" button is now in another window than the viewer. Jumping between windows when doing a refresh get's annoying quickly.

Also, thanks for publishing the example. I am trying to replace my bundled copy of Catapult with a thin page embedding Perfetto, and the JS example helped me get bootstrapped!

from perfetto.

For posterity I just added a new doc page with a revised jsfiddle example @
https://perfetto.dev/docs/visualization/deep-linking-to-perfetto-ui

from perfetto.

@primiano
In my understanding, you explained how we pass the data to the new window(https://ui.perfetto.dev/) via postMessage above.

But I'm curious if it's possible just to pass the trace data(which I have) to perfetto and display(embed) the view in my html rather than open a new perfetto ui.

@carl-mastrangelo
Have you got to embed perfetto in your page?

from perfetto.

@primiano
In my understanding, you explained how we pass the data to the new window(https://ui.perfetto.dev/) via postMessage above.

But I'm curious if it's possible just to pass the trace data(which I have) to perfetto and display(embed) the view in my html rather than open a new perfetto ui.

@carl-mastrangelo
Have you got to embed perfetto in your page?

I am interested in this as well. Any plans to support embedding perfetto UI in an embedded iframe? Thanks

from perfetto.

Any plans to support embedding perfetto UI in an embedded iframe?

No, that will realistically never happen from the main ui.perfetto.dev instance. There are too many security risks that come from an iframe. If you need an iframe fork the codebase and host it on an origin you control.

from perfetto.

Any plans to support embedding perfetto UI in an embedded iframe?

No, that will realistically never happen from the main ui.perfetto.dev instance. There are too many security risks that come from an iframe. If you need an iframe fork the codebase and host it on an origin you control.

I was thinking of vulcanized catapult trace_viewer_full.html like version that I can embed.

from perfetto.

I was thinking of vulcanized catapult trace_viewer_full.html like version that I can embed.

It doesn't work:

• Too many features (Webusb, serviceworker, cache storage) in chrome and other browsers work only behind a secure origin (https or localhost). file:// schemes are not considered secure.
• Web workers (which we use extensively) don't work from a vulcanized file, they need to be spawned off a dedicated js entrypoint.

from perfetto.

• Too many features (Webusb, serviceworker, cache storage) in chrome and other browsers work only behind a secure origin (https or localhost). file:// schemes are not considered secure.

We can serve the trace_viewer_full.html from localhost/HTTPS, so that should not be an issue.

• Web workers (which we use extensively) don't work from a vulcanized file, they need to be spawned off a dedicated js entrypoint.

Would Embedded workers suffice?
Mentioned here: https://developer.mozilla.org/en-US/docs/Web/API/Web_Workers_API/Using_web_workers

from perfetto.

Sorry folks but we are not going to add yet another bundling option that we have to maintain, hoping that every time we add a new delendency on a web platform feature that will keep working on a vulcanized HTML.thst slows down velocity of the project for a benefit that is marginal (i don't fully understand the rationale behind the request)

If you are hosting the file on a https server then why don't you host the static site as-is, without vulcanization?

from perfetto.

Apologies - I am just trying to understand the technical feasibility of such an approach - fully understand you not wanting to add, support or maintain such a feature

from perfetto.

Closing because I think we've answered everything relevant here.

from perfetto.