charmplusplus / projections Goto Github PK
View Code? Open in Web Editor NEWPerformance Analysis Tool for Charm++
License: Apache License 2.0
Performance Analysis Tool for Charm++
License: Apache License 2.0
Original issue: https://charm.cs.illinois.edu/redmine/issues/1057
It would be useful to be able to use projections just for user event tracing. For very finegrained applications like ROSS, tracing every event can be expensive and doesn't necessarily add much. The more interesting part is tracing individual phases with user events, which can only be done when the full trace infrastructure is turned on.
Original issue: https://charm.cs.illinois.edu/redmine/issues/991
By default, the Usage Profile view (and most Projections views) are sorted by ascending PE index. For the sake of improving the ease of analysis, it should be sortable on other properties, such as the amount of time spent in a particular EP on each PE. This can be extended to an arbitrary degree, allowing sorting on different aspects of the data, such as sorting on total utilization, etc.
Original issue: https://charm.cs.illinois.edu/redmine/issues/618
Some indication of the number of available tasks on each PE (i.e. for each PE, a graph of message queue size vs. time) would be useful to identify situations where a PE is starved or overworked.
Original issue: https://charm.cs.illinois.edu/redmine/issues/440
If the user selects a large number of processes or large time intervals on large logs, display some warning to ensure users don't unknowingly overstress their machines.
Original author: Nils Deppe (@nilsdeppe)
Original issue: https://charm.cs.illinois.edu/redmine/issues/1525
It would be really useful to be able to pass a vector<double>
to the traceUserBracketedEvent
function to record various numbers during runtime. These could, for example, be PAPI counters. It might make sense to ASSERT the size of all vectors for a specific event number are the same, but it could be considered a feature to let them be different.
This feature would be even more useful if analysis of the numerical values stored was possible, say average, outlier, max, min, etc.
Original issue: https://charm.cs.illinois.edu/redmine/issues/924
Will attach screenshot momentarily.
Original issue: https://charm.cs.illinois.edu/redmine/issues/1229
When scrolling around in timeline, performance can be quite bad, leading to a great deal of lag and a poor user experience.
Original author: Maxx Macica
Original issue: https://charm.cs.illinois.edu/redmine/issues/521
When converting a time to a humanReadableString, the rounding causes us to stop reading our log files early. For example, look at dcsdata/jessie/PPL/git/benchmarks/leanmd/prj_root/. The actual latest time of all the log files is 7610759, but the endTimeField in the range dialog box sets the end time to 7.61s. As a result, the end computation signal is never reached when processing the logs and we lose some of our data.
Original author: Keshav Santhanam
Original issue: https://charm.cs.illinois.edu/redmine/issues/976
No body.
Original issue: https://charm.cs.illinois.edu/redmine/issues/1004
Sanjay requested a view that would show the total number of messages in certain sized buckets and that would elide empty spaces between messages.
Histogram does this to some degree, but I think the intent is to make it a bit more sophisticated.
Original issue: https://charm.cs.illinois.edu/redmine/issues/436
The majority of people who use extrema analysis use the "least idle time" view. Change the GUI so that this is the default choice.
Original author: Jim Phillips (@jcphill)
Original issue: https://charm.cs.illinois.edu/redmine/issues/699
When tracing messages back, any message generated by a non-entry such as a periodic callback triggers tracing back from the previous entry on that PE. Tracing should stop at the periodic call, or potentially trace back to the entry that created the periodic callback.
Original author: Nikhil Jain
Original issue: https://charm.cs.illinois.edu/redmine/issues/21
Java based 3D view in projections is buggy - fix it.
Original issue: https://charm.cs.illinois.edu/redmine/issues/1283
I compiled leanmd with +tracemode summary and ran it with +sumDetail. On opening the leanmd.sum.sts file with projections visualization tool and selecting usage profile, End Time in the range selection is shown as 2441000.0000000001 s, which is invalid.
On selecting that invalid range (or a valid range like 0 - 2.4 ms), a blank usage profile is shown.
Original issue: https://charm.cs.illinois.edu/redmine/issues/518
When attempting to load a file on the command line with a path that contains a space, the parser splits on the space.
For example:
ronak`machine:~$ projections ~/Test\ Dir/test.sts
Internal Error: Path [/home/ronak/Dir] supplied for file detection is not a directory! Please report to developers!
Original author: Maxx Macica
Original issue: https://charm.cs.illinois.edu/redmine/issues/523
Call Table has some problems, including:
Original author: Nikhil Jain
Original issue: https://charm.cs.illinois.edu/redmine/issues/280
The Y axis title for processor utilization graph is not displayed correctly.
Original issue: https://charm.cs.illinois.edu/redmine/issues/1230
When loading files into Timeline (and later), there's a lot of memory usage. For some logs, e.g. prudence:/scratch/nitin/charity_proj_logs/1
, the memory usage is high enough that they take unreasonably long to load. This can probably be made more efficient.
Original issue: https://charm.cs.illinois.edu/redmine/issues/438
In the communication vs. time graph, the external received bytes count is higher than the total received bytes count. This is clearly fallacious.
Original author: Nils Deppe (@nilsdeppe)
Original issue: https://charm.cs.illinois.edu/redmine/issues/1524
Because of the structure of our code, we have only one entry method which then forwards to the correct member function for the data it received. I can trace all the user functions with Charm++ and they show up in Timelines. However, it would be very valuable to us if we could see them in the Time Profile as well. More generally, if bracketed user events could get promoted to being visible everywhere where entry methods are, say by being able to hide/show them like in Timelines that would be really awesome.
Original issue: https://charm.cs.illinois.edu/redmine/issues/415
Profile and improve the performance of projections. Look into using hardware acceleration for drawing.
Original issue: https://charm.cs.illinois.edu/redmine/issues/1021
Try loading the projections logs in prudence:/scratch/prateek/temper-projections-mar24/pr1 using the current git head of projections. Now try to open a timeline.
Nothing will be displayed, as there is a chain of errors.
Works fine using the old 07142013 binary distribution. Which is a tragedy in that it includes no changes made in the past 2 years.
On a related note, 2 years seems like a long time to go between releases.
Original issue: https://charm.cs.illinois.edu/redmine/issues/483
Overview is dropping many entry methods from its display making cores appear to be idle when they are not. These appear correctly in timeline.
For example see:
~/work/w256_4k.tracered.rhor1.t22
If you examine the interval 15.123s-16.884s (saved) for cores 2664 to 2689 (as shown in attached screen shots) the overview looks wildly different from the compact timeline.
Original author: Yanhua Sun
Original issue: https://charm.cs.illinois.edu/redmine/issues/459
In projections, if you click on a entry method which is inline or local, it can not be traced back. This is due to the nested begin_processing entries
in the log.
We want to trace it to its parent entry method.
Original author: Jim Phillips (@jcphill)
Original issue: https://charm.cs.illinois.edu/redmine/issues/698
When communication thread tracing is enabled, the "trace messages forward" feature only traces off-node messages to the sending node's communication thread. Tracing forward should cascade until the message arrives at the actual destination.
Original issue: https://charm.cs.illinois.edu/redmine/issues/1884
When I manually insert traceBegin and traceEnd calls into my application to trace say a few iterations out of many, then open Projections on those traces, Projections will display the entire time period that the application ran by default. But only a small sliver of that is stuff that I actually care about. It would be nice to by default start the time range at the first traceBegin call and end it at the last traceEnd call.
Original issue: https://charm.cs.illinois.edu/redmine/issues/995
Having 'Color by user supplied parameter' in the Timeline tool is quite useful. To make corresponding displays from other tools, it would be useful to have that option elsewhere, too.
I'll open sub-issues for the cases I'm most interested in.
Original issue: https://charm.cs.illinois.edu/redmine/issues/1170
Currently, when logs numbered beyond the number specified in the sts file exist in a folder, Projections attempts to load more and gives the exception:
Exception in thread "Thread-8" java.lang.RuntimeException: The sts file only specifies 4 PEs, but you are trying somehow to load pe 5
at projections.Tools.Timeline.Data.getData(Data.java:924)
at projections.Tools.Timeline.TimelineRunnableFileReader.run(TimelineRunnableFileReader.java:15)
There's no reason to even attempt to load these.
Original author: Yanhua Sun
Original issue: https://charm.cs.illinois.edu/redmine/issues/506
Currently, when we open timeline by default the time range is from 0 to maximum end time. However, for a lot of projections logs, only a small portion has real data.
Please refer to this log set,/dcsdata/home/jessie/PPL/NAMD/benchmarks/vesta/prj_root_2048_2. Actually this is true for most NAMD logs.
Instead, in timeline open dialog, we should be able to only pick the time range with valid data. (Begin computation, end computation) time range. This should be relatively easy to implement
Original issue: https://charm.cs.illinois.edu/redmine/issues/517
From the Usage Profile view, Tools -> Change Colors does not do anything. It should open a new window that allows the user to select new colors (cf. operation from the Time Profile graph view)
Original issue: https://charm.cs.illinois.edu/redmine/issues/515
I received an in-person request for vector graphics output (svg preferred, but we might as well support things like pdf and eps) from Projections for ease of including them in papers and allowing things like lossless zoom. Additionally, the ability to use a white background should be made more clear, as that is more amenable to inclusion in papers.
Original author: Yanhua Sun
Original issue: https://charm.cs.illinois.edu/redmine/issues/435
I do not know whether others have this problem. I found that after we added SDAG_RTS entry
in Charm, the time profile , usage profile in Projections can not display correctly.
In timeline, the SDAG_RTS seems not much time. However, when using time profile, it is totally messed up. All overhead time goes away. Instead, it is all SDAG_RTS time. This is wrong.
The projections data can be found here
/dcsdata/home/jessie/PPL/git/benchmarks/leanmd/prj_root
Original issue: https://charm.cs.illinois.edu/redmine/issues/1780
Currently Projections shows chare array element indices as 3 ints, even though up to 6D arrays are supported.
Issue for the problem in Charm's tracing: https://charm.cs.illinois.edu/redmine/issues/1694
Original author: Lukasz Wesolowski
Original issue: https://charm.cs.illinois.edu/redmine/issues/505
Given that we normally assign a communication thread in Charm++ to a separate core, it would be useful to be able to visualize the utilization of this core to determine whether it is under- or overutilized. This could help, for example, when determining how many comm threads are optimal for a particular run.
We should be able to show comm thread utilization in the usage profile view.
Original issue: https://charm.cs.illinois.edu/redmine/issues/1271
The README is out of date in at least its build instructions, maybe other things
Original issue: https://charm.cs.illinois.edu/redmine/issues/439
When in histogram views, choosing one interval shows more than one bar when it shouldn't.
Original author: Yanhua Sun
Original issue: https://charm.cs.illinois.edu/redmine/issues/507
Now, we can save time range to history and to disk so that next time when we load we can directly use them. Similarly for large scale run, it is useful to save interesting processor data set.
Original issue: https://charm.cs.illinois.edu/redmine/issues/256
Opening this so that the work is visible, both to external users and for internal workload balancing.
Original issue: https://charm.cs.illinois.edu/redmine/issues/1293
EPExecTime has a large number of zeros.
To reproduce this issue:
Compile charm with enable-tracing.
./build ChaNGa netlrts-linux-x86_64 smp --enable-tracing --with-production -j4
After downloading ChaNGa, change its makefile to add -tracemode summary to LDFLAGS.
Point ChaNGa to charm
export CHARM_DIR=/dcsdata/home/nbhat4/Work/software/changa/charm
Compile ChaNGa
make -j4
Place the input file ./snapshot1M.param in the same directory as ChaNGa. I've stored the input files (snapshot1M .param and .std) in /scratch/nitin/changa/inputFiles/ on charity).
Run changa on one core (with or without +sumDetail)
./charmrun +p1 ++ppn 1 ./ChaNGa -n 1 -p 2 -b 32 +bench +traceroot ./proj_summ_logs_32/ +sumDetail ./snapshot1M.param +pemap 1 +commap 0 > changa_output
Open the traceroot directory and try opening the ChaNGa.0.sum file and look at the EPExecTime row. (A lot of entry methods have their values as zero).
Original author: Yanhua Sun
Original issue: https://charm.cs.illinois.edu/redmine/issues/455
The logs are here
/fastscratch/Yanhua/logs_apoa1
Select 8 timesteps and display in usage profile, check for example PE 7, the utilization is wrong
Extreme of least idle is totally messed up.
Original author: Osman Sarood
Original issue: https://charm.cs.illinois.edu/redmine/issues/258
The message sizes only represent the 112 bytes header if pack unpack for messages is used. Parameter marshalling works correctly.
Example:
Original issue: https://charm.cs.illinois.edu/redmine/issues/703
For the timeline view, when a non-default color scheme (e.g. color by frequency) is chosen, doing manual color selection does not change the color of entry points on the timeline.
Original issue: https://charm.cs.illinois.edu/redmine/issues/414
Extrema Analysis shows overhead where Usage Profile/Timeline shows idle time.
This can be reproduced with:
/sharedscratch/harshitha/cosmomslb200knodelbck9/
time range 104.2 s to 105.19
load 0-31743:1024 pes
Original issue: https://charm.cs.illinois.edu/redmine/issues/457
Trace to sender, as invoked by the right click pulldown menu, fails to load for OpenAtom traces for nearly all entry methods. Failure error message is "Message was sent from outside the current time range". However, that is a clear misdiagnosis. When you have loaded an entire time step and examine methods from phases late in the time step, there is no way for the sender to have come from outside the step.
See ~bohm/work/w256_4k.t4 for example traces. It has a saved interval which matches a complete timestep. You can load a few timelines (0-3) and try to backtrace any method. The only back trace invocations which succeeded for me are for messages which where sender PE=receiver PE.
Original author: Maxx Macica
Original issue: https://charm.cs.illinois.edu/redmine/issues/514
Right now there are many different files that are used to read and process logs. We should make it more unified and simpler.
Original issue: https://charm.cs.illinois.edu/redmine/issues/386
When I load a coarse-resolution histogram (200 bins, 100 bytes each) that covers my entire run's set of message sizes, all of the counts are graphed correctly. However, if I set some finer resolutions, some of the data is omitted from the subsequent display entirely. This could be extremely misleading!
With the attached log, a setting of 105 bins, 50 bytes wide, and starting bin size 1000 renders the complete data set. Halving the width and doubling the bin count (210 bins, 25 bytes each) truncates the data above the midpoint of the display. If I further refine the requested display (e.g. 700 bins, 16 bytes wide), nearly all of the data are apparently discarded - I'm left with just three bars out of the dozens I expected.
Original issue: https://charm.cs.illinois.edu/redmine/issues/794
Make the didactic projections logs publicly available and create a freestanding tutorial page for people to learn how to use Projections by themselves.
Original author: Zhengqi Yang
Original issue: https://charm.cs.illinois.edu/redmine/issues/497
In the main window, if open up "Tools" menu, and then use "Enter" button to open a menu item, the OK button in the pop-up window will be clicked(triggered) automatically if there is one. (Tested with a few menu items)
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.