Add support for search about speedscope HOT 31 CLOSED

The second major part of this is now landed and deployed to https://speedscope.app/. See #297 for details.

With this change, you should be able to search to visually highlight matching frames in the time order & left heavy views. @itarato, @tamuratak, @lptr, please give this a try and let me know if it helps address your needs!

from speedscope.

@lptr What do you think of this?

from speedscope.

Okay! All of this is now deployed to https://www.speedscope.app/ and released as part of release 1.9.0 of speedscope on npm. I'm going to close out the issue now. Thanks everyone for the feedback, and special thanks to @tamuratak for their donation!

from speedscope.

Hi @tamuratak!

Thank you so much for donating! I'm now committed to implementing this, though it might take a while. I'm currently working on #167 which has also been motivated by donation.

Will keep this thread updated with progress as I make it!

from speedscope.

Thanks @jlfwong for the reply. It makes sense.

For search (and from our perspective) the simplest scenario would be the most impactful: user enters a part of the name of the node and the viewer makes them visible either one of 2 ways: by highlighting them on the mini-map (however I can imagine this might not always work considering the scaling) or having [prev][next] buttons to position them on the main view. This satisfies investigations on particular calls.

Some less frequent searches we do:

• pattern match (eg to see how a library or package is doing overall)
• narrow search down by timespan (to find specifically long calls)
• filter to calls of calls (eg only find foo calls initiated from a bar call)

This is just for ideas, wouldn't imagine this to be broadly impactful.

Much appreciated for your work, btw. I'll check your issue link as recommended.

from speedscope.

Okay, the first part of this task is landed and deployed to https://www.speedscope.app/. You can see the details in #293.

@itarato this is helpful feedback, thank you!

For the main request you listed, identifying the relevant frames within the flamechart/flamegraph views is the next thing I'll be working on, and that hasn't been done yet.

from speedscope.

@tamuratak I started working on having access to next/prev through results in a branch here: #304

On that branch, if you're in the search results and you hit "Enter" or "Shift+Enter" you can cycle through the results. It's probably not terribly helpful yet as it is because it doesn't adjust the viewport to show you where the entry is, but aside from that I'm open to feedback about whether this would address your use case.

@lptr Are you sure highlighting in the minimap would meet your use-case? This isn't actually terribly hard to do I don't think, but I'm wondering if there might be some other better way of addressing your use case. What's this information useful to you for? Just getting a qualitative sense of how much time is spent in the code matching the query?

from speedscope.

This would also be really useful to have in some way in sandwich view

from speedscope.

This would make the tool even more useful to us. It's the only feature missing from the SVG flamegraphs we are currently using at Gradle.

from speedscope.

Oh, I thought I put a bounty on this already.

I’ll commit to implementing this feature myself if a total of $300 of donations is made to any combination of these specific charities:

• American Civil Liberties Union
• /dev/color: /dev/color is a non-profit organization whose mission is to empower Black software engineers to help one another grow into industry leaders.
• Girls Who Code: Girls Who Code was founded with a single mission: to close the gender gap in technology

If you’d like to contribute to that total, please post a screenshot of your donation receipt in this thread*.

Independent of that, I would accept a PR to implement this, regardless of donation amounts.

*: I'm planning on trying this policy on a number of issues. Donations will apply to the total of exactly one issue, so please don't post the same receipt in multiple issue threads.

from speedscope.

@jlfwong do you have any update on that donation count?

from speedscope.

The current donation count for this issue is $0 out of the target $300 :)

from speedscope.

I make this donation to express sincere thanks for the development and future development of @jlfwong's speedscope.

I hope GitHub Sponsors will make these kinds of charities much easier.

from speedscope.

If it's possible to split the work into chunks, I'd be down to help out! Search would be useful for me too.

from speedscope.

Hi all -- the previous donation-motivated change in #167 landed a while ago, so this is my next priority. I haven't had a chance to work on open source stuff in a while, but whenever I do, this change will be one of my main priorities!

from speedscope.

✋ Hey :) I've actually looked into this not long ago but got lost in the codebase (Redux and WebGL being totally new to me).
I've managed to identify a few places where the rectangles are updated and even found good starting point to mark nodes that could match a potential search term (in src/gl/flamechart-renderer.ts) - what I couldn't figure out is where to establish the search term input in the data structure (I assume this has to be in the main Redux state) and where to surface on the UI (this is the lack of my WebGL knowledge on how and where to populate buffers to shaders).

If there is any change I could be of help I'd be super keen to contribute. Is there any way to have some explanation on the structure and logic flow? I feel with that help I could start some prototype.

from speedscope.

Hi @itarato! Appreciate the willingness to help.

This change in particular requires a pretty broad sweeping change that touches a really broad surface area, so it would be pretty hard for me to guide someone else through it, especially considering I'd also need to write a product spec for this to turn out the way that I want to ship, so that's why I haven't tried to recruit help for changes here.

There's a collection of tasks here that would be much more reasonable entry points into the codebase: https://github.com/jlfwong/speedscope/issues?q=is%3Aissue+is%3Aopen+label%3A%22help+wanted%22

If you specifically want search rather than specifically wanting to contribute, then the most helpful thing to do would be provide example use cases or user-expectations to help guide what I prioritize in terms of search!

from speedscope.

Thanks @jlfwong for implementing this! I checked it out, and it looks promising. I think it's already good enough for us to switch from our standard SVG viewer. There are a few things I miss:

1. Support for searching with regular expressions. This is something I do a lot with our current workflow.

2. Highlight matches in overview. I think it would be incredibly useful to see how the matched portion relates to the rest of the timeline. This could be accomplished by adding the yellow overlay there as well.

3. The current partial yellow highlighting is a bit hard for me to notice. Maybe changing the text color too would help? And/or using bold? Or maybe the highlight could replace the background color completely (or produce an overlay on top of)?

from speedscope.

Support for searching with regular expressions. This is something I do a lot with our current workflow.

Ah, interesting. This is good feedback, but I'm going to consider that as a separate task since I think doing it properly adds a non-trivial amount of UI complexity (regex validation, toggling to support both regex & fuzzy match, etc.)

Highlight matches in overview. I think it would be incredibly useful to see how the matched portion relates to the rest of the timeline. This could be accomplished by adding the yellow overlay there as well.

By in the overview do you the minimap at the top? Or on the grey bars with the numerical markers on them?

The current partial yellow highlighting is a bit hard for me to notice. Maybe changing the text color too would help? And/or using bold? Or maybe the highlight could replace the background color completely (or produce an overlay on top of)?

What do you think about fading out the text color of all the frames that don't match instead?

from speedscope.

Support for searching with regular expressions. This is something I do a lot with our current workflow.

Ah, interesting. This is good feedback, but I'm going to consider that as a separate task since I think doing it properly adds a non-trivial amount of UI complexity (regex validation, toggling to support both regex & fuzzy match, etc.)

Is there fuzzy match already? How does that work? If not I'd say regex would be more important for me. I'd be very happy with some trivial UX, like "if it looks like /patter/ do a regex, otherwise do straight matching" or something.

Highlight matches in overview. I think it would be incredibly useful to see how the matched portion relates to the rest of the timeline. This could be accomplished by adding the yellow overlay there as well.

By in the overview do you the minimap at the top? Or on the grey bars with the numerical markers on them?

Ah, sorry, I meant the minimap.

The current partial yellow highlighting is a bit hard for me to notice. Maybe changing the text color too would help? And/or using bold? Or maybe the highlight could replace the background color completely (or produce an overlay on top of)?

What do you think about fading out the text color of all the frames that don't match instead?

I like that idea, as long as it's not a complete fade-out, but something like 50%.

Another option would be to fade out the entire non-matching frames, background, border, text alike.

from speedscope.

Is there fuzzy match already? How does that work? If not I'd say regex would be more important for me. I'd be very happy with some trivial UX, like "if it looks like /patter/ do a regex, otherwise do straight matching" or something.

Try it out :) If you type "hw" it'll match "hello world". The implementation is based on the fuzzy matching algorithm from https://github.com/junegunn/fzf.

I'm reluctant to do the /patter/ suggestion because symbol names might contain directories, so people might literally be searching for the slashes. Even with that, I would want to add a visual indication of regex compilation failure.

from speedscope.

Thank you so much @jlfwong this is amazing. Works really well. I +1 the feedback regarding visibility on the time 'time order' and 'left heavy' views. Either making the match more prominent or the rest less. Again, much appreciated!

from speedscope.

In #298, I changed the text color for umatched frames when a search is active to be a lighter grey, which should make the visually matched frames more visually prevalent. I haven't deployed this yet, but should be able to do that sometime this week.

@itarato @lptr I'm considering at the moment whether the next/prev buttons to cycle through matches and focus the view of them is worthwhile or not. It's not clear to me that it really facilitates a kind of investigation that isn't possible with just search highlighting. What do you think?

I think that given the initial description of this task as of when the bounty was laid down, I think the spirit of this task is complete. @tamuratak what do you think?

from speedscope.

I tried the current master with unmatched frames being lighter grey. I have to say that it is still difficult to find a small rectangle.

Since a function that is just waiting for an asynchronous task in a separate process to be completed has a short execution time, the rectangle is very small. I want to find such a function by searching, and want to know whether the asynchronous task in a separate process is the bottleneck.

However, I agree improving the visibility is another issue to tackle. I think this task is complete.

from speedscope.

@tamuratak To make sure I'm following, I think the issue here is that sometimes the thing you're searching for on the screen is very small, is that right? Can you show a screenshot of what you're seeing? Would having [prev/next] buttons that focus the element & zoom into it help with this scenario significantly?

I'm not opposed to implementing prev/next, I just wanted to make sure it serves a real use case and isn't just there because it "should" be.

from speedscope.

For example, I want to find parseLatexPreamble, which starts parsing LaTeX files in a child process, in the attached profile file.

I manage to find it. I would like it to become much easier.

Background

I am working on an extension of VS Code. For the development of extensions, it is recommended that heavy tasks are performed in separate processes. And, on the extension host, lots of events are fired asynchronously. Some of them are related to our extension. Others are not. The function I am looking for is buried in those events.

CPU-20200727T104328.345Z.cpuprofile.txt

from speedscope.

@jlfwong for me next/previous seems less useful. What I'm typically interested in is big (sometimes discontinuous) parts of the graph, like searching for org/mozilla/javascript on the example graph.

With the SVG flame graphs the matches would be highlighted in purple, which would allow me to judge the ratio of the graph matched quite well. With speedscope the effect is less clear, as the yellow highlight is harder to see, and it also doesn't cover the whole entry (only the matched text and some of the border of the entry).

from speedscope.

@lptr Are you sure highlighting in the minimap would meet your use-case? This isn't actually terribly hard to do I don't think, but I'm wondering if there might be some other better way of addressing your use case. What's this information useful to you for? Just getting a qualitative sense of how much time is spent in the code matching the query?

I think highlighting in the minimap would be nice, but what I care about more is a more consistent highlighting in the main window(?). And yes, the main goal for me is to get a visual indication of how much of the area actually relates to my search.

from speedscope.

Gorgeous.

from speedscope.

Okay, @lptr & @tamuratak the latest changes are on master. I'll try to deploy tomorrow night, but if you get a chance to test it out locally, I'd love to know if this is now addressing both of your use cases.

Details for what's changed in #304.

from speedscope.

Thank you @jlfwong!

from speedscope.