viladoman / compilescore Goto Github PK

Hey there, I'm impressed with how well this works with Unreal out of the box! That being said there's a few parts that could be improved.

1. Running "Build and Profile" on just the game project causes a full rebuild of the entire engine. I suspect this is due to Unreal being 'smart' about which toolchains it's using. We're using AutoSDK and I'm guessing CompileScore only uses the built in toolchain. Not sure if there's any easy answers to this one.
2. Need to manually disable "Unity" (Fat/Amalgamation/Combined) builds. Currently CompileScore isn't terribly useful just because the results are not accurate due to the unity builds. If CompileScore did this automatically somehow it would definitely cause a full recompile every time. Also not really sure if there's an easy answer to this.

I think Unreal can output various forms of compilation databases, which may be interesting to replay the exact same commands with the exact same compilers Unreal is using.

I think some documentation on these (or other best practices on using CompileScore with Unreal) would probably satisfy me. Worst case at least having a ticket open with some of these gotcha's is hopefully enough for anyone else wanting to try this out 👍

Thanks for making this!

What's the minimum MSVC toolchain this works with? I started data collection, ran a build which uses the compiler and linker from MSVC 2015, stopped data collection, and ended up with a 1K .scor file which when loaded seems to contain no data.

Does this have a minimum compiler version e.g. 2019 toolchains?

On "Build Solution and Profile" using an elevated instance of VS: Score Data Extractor failed to start the recording session with code -1. The current build data won't be captured. Please check the output pane for more information.

Output from Compile Score:

[18:44:53] Calling ScoreDataExtractor with -clang -start -i C:/Users/Jim Viebke/Dropbox/Code/C++/Prime Bitstrings/
[ERROR] input path is not a directory.
Execution Time: 01ms
[18:44:53] [ERROR] Score Data Extractor failed to start the recording session with code -1. The current build data won't be captured. Please check the output pane for more information.
[18:44:56] Building...
[18:45:16] Calling ScoreDataExtractor with -clang -stop -tp 100 -td 3 -d 3 -i C:/Users/Jim Viebke/Dropbox/Code/C++/Prime Bitstrings/ -o "C:/Users/Jim Viebke/Dropbox/Code/C++/Prime Bitstrings/Compile Score/compileData.scor"
[ERROR] input path is not a directory.
Execution Time: 00ms
[18:45:16] [ERROR] Score Data Extractor process failed with code -1. Please check the output pane for more information.
[18:45:22] Score generation completed!
[18:45:22] Score file main processed in 79 μs
[18:45:22] Score file globals processed in 61 μs

Output from Build:

Rebuild started...
1>------ Rebuild All started: Project: Prime Bitstrings, Configuration: Release x64 ------
1>Prime Bitstrings.vcxproj -> C:\Users\Jim Viebke\Dropbox\Code\C++\Prime Bitstrings\x64\Release\Prime Bitstrings.exe
========== Rebuild All: 1 succeeded, 0 failed, 0 skipped ==========
========== Rebuild started at 6:44 PM and took 23.600 seconds ==========

Settings:

{
  "ScoreSource": 1,
  "ScoreLocation": "C:/Users/Jim Viebke/Dropbox/Code/C++/Prime Bitstrings/Compile Score/compileData.scor",
  "ScoreGenerator": {
    "Compiler": 1,
    "InputPath": "C:/Users/Jim Viebke/Dropbox/Code/C++/Prime Bitstrings/",
    "OutputPath": "C:/Users/Jim Viebke/Dropbox/Code/C++/Prime Bitstrings/Compile Score/compileData.scor",
    "OverviewDetail": 3,
    "TimelineDetail": 3,
    "TimelinePacking": 100,
    "ExtractIncluders": true,
    "CollapseTemplateArgs": true
  }
}

Using:
Visual Studio Community 2022, 17.7.6
Compile Score 1.8.4, Data Version 9
Clang 16.0.5
Windows 10

This may be related to #10

I get a crash when stopping recording with MSVC, I debugged it and it crashes here: https://github.com/Viladoman/CompileScore/blob/master/DataExtractor/src/Extractors/MSVCScore.cpp#L45

Because both compilerPass.OutputObjectPath() and compilerPass.InputSourcePath() are nullptr. I am not well-versed enough in all this to tell you why! I can get the program to complete execution by returning "[unknown]" if those are nullptr but then most units end up not having a name.

We are generating a VS2019 solution with CMake with MSVC as compiler. Let me know if I can provide more info!

Here is what I do:

• start command prompt as administrator (btw, same behaviour when using non-elevated mode)
• start ScoreDataExtractor

C:\Users\bw\AppData\Local\Temp\ScoreExtractor>ScoreDataExtractor.exe -msvc -start
Starting MSVC recording...
Recording session started successfully!
Execution Time: 08ms

• build some C++ projects in Visual Studio 2019
• stop ScoreDataExtractor

C:\Users\bw\AppData\Local\Temp\ScoreExtractor>ScoreDataExtractor.exe -msvc -stop -o %temp%\TF_Qt.scor
Stopping MSVC recording and Generating Score...

then Application crashes

Faulting application name: ScoreDataExtractor.exe, version: 0.0.0.0, time stamp: 0x600116b4
Faulting module name: ScoreDataExtractor.exe, version: 0.0.0.0, time stamp: 0x600116b4
Exception code: 0xc0000005
Fault offset: 0x000000000000b5a3
Faulting process id: 0xf28
Faulting application start time: 0x01d6f09e4e1e6d2f
Faulting application path: C:\Users\bw\AppData\Local\Temp\ScoreExtractor\ScoreDataExtractor.exe
Faulting module path: C:\Users\bw\AppData\Local\Temp\ScoreExtractor\ScoreDataExtractor.exe
Report Id: fa6ddbe0-99c7-4837-9f42-6b58d9b2fe3c
Faulting package full name: 
Faulting package-relative application ID: 

I am using

Microsoft Visual Studio Professional 2019
Version 16.7.2
VisualStudio.16.Release/16.7.2+30413.136
Microsoft .NET Framework
Version 4.8.03752

Not a lot of information, I am afraid. Please tell me if there is anything more specific I can provide.

Thanks in advance, Benjamin

In the image below, working down from the top, it's correct up to registry.h. However, my project has two files with that name (this is hard to avoid when using lots of third party libs). The chain of files shown to be includers of registry.h (those below it in the image) are in fact includers of the other file, not the one that includes compilednodedefinition.h.

Not sure if this is a bug/oversight in CompileScore, or a limitation of the compilers? I also don't know at this point if this bug extends to the timings, or if it's just in the include graph. You would probably know right away whether or not they're backed by the same data.

Thanks for a great tool btw.

One thing that becomes clear quite quickly when analysing using CompileScore is the dependency on #include order, and how that has to be taken into account when interpreting results. By this I'm referring to the fact that a single header can have wildly different times in different contexts, due to the effect of include guards and what other headers have been included before it in one context vs another.

It seems like having an 'exclusive time' column for each header would be very useful for making it easier to identify the true compilation time offenders. So basically, the time that was spent compiling the header, minus the time spent within all its #includes. I'm not sure if the underlying compiler invocations can give you this information directly, but even if not it seems like you probably have the data you need to calculate it. Is this something you've considered?

So I think there are perhaps a couple of different issues. I loaded up the test project (built as-is with Clang 10) one further time, and without having changed anything this time it correctly displayed the data (I'd previously rebuilt the project and restarted VS a couple of times without any luck, despite the parsing reporting success). So I guess maybe there's just some unpredictability relating to VS extension initialization order? If it was possible to enable more logging it might help pin down things like that.

As for my own project, built using a source-built pre-Clang 11, I'm still seeing the source files displayed but with no numbers. Attaching an example json file from the Clang output - I can see one additional field that didn't show up in the test project output, but it doesn't look like that would be an issue. Otherwise the format appears the same, but I may be missing something.

api_alloc.cpp.json.zip

Hi,
I get the following error every time I run the profiler

[14:19:17] Calling ScoreDataExtractor with -msvc -start -d 1 -td 1
Starting MSVC recording...
Recording session started successfully!
Execution Time: 06ms
[14:19:17] Building...
[14:19:29] Calling ScoreDataExtractor with -msvc -stop -tp 100 -td 1 -d 1 -o C:\Git\Pathtracer\compileData.scor
[ERROR] Unknown output file extension provided. The MSVC generator only knows how to generate .scor or .etl files.
Execution Time: 00ms
[14:19:29] [ERROR] Score Data Extractor process failed with code 1. Please check the output pane for more information.
[14:20:45] Score generation completed!

I also get the error A recording is currently in progress on your system is preventing CompileScore from starting a new one.This can occur if you forgot to stop a CompileScore recording prior to running the start command, or if other processes have started ETW traces of their own.Please try running the CompileScore -stop command. every time I dont manually stop the profiler with tracelog -stop MSVC_BUILD_INSIGHTS_SESSION_COMPILE_SCORE

Hi Ramon,
all 5 highlighting definitions have the same name, which makes it quite hard to assign new colors :-)

Cheers, Benni

Hello,

Right clicking a project and choosing "Build and Profile fails".

This is likely because our .vcxproj files are modified to import our own .targets XML files in a bigger scheme that allows us to factor out build settings for the ~100 projects in our .sln.

Will try to come up with a repro.

There are no includes profiled, and there's no way to profile or visualize includes for my clang builds using -ftime-trace; is this expected? Or is it potentially a bug with the compiler I'm using?

TLDR

Powershell script for automating captures https://gist.github.com/ikrima/45497f8d664c35f0d656b95773426ac3

Not sure if this is useful enough to be in the repo. If so, lmk, I can make a PR, update the defaults, and strip references to CppBuildAnalyzer

Details

I got tired of starting/stopping and made a powershell script that auto-elevates. Also I always forget the steps after a week and end up hunting them down

The gist auto elevates but is annoying that the UAC prompt will come up each time.
My local version uses gsudo which caches elevation requests for 5 mins but the repo version probably shouldn't use it bc security
The two lines are just commented out in the gist

Running it is simple:

.\vcperf.ps1 -start
[build in visual studio]
.\vcperf.ps1 -stop -compilescore

It's a minor issue, but the colouring of #includes leads to confusion when using Visual Studio's built-in git diff features. At a glance, it looks like all includes are source code changes.

I wonder if the context (diff vs regular view) is exposed by VS so that the extension can optionally disable the annotations? Failing that, just a simpler way to toggle the annotation functionality (a toolbar button or keyboard shortcut) would be useful.

See #39 (comment)

This would make working in the timeline view visualization much more efficient/user-friendly compared to having to manually click on the zoom control and up/down/left/right arrows.

One thing i've incorporated for a small dev quality of life/lazyness thing is running https://github.com/aras-p/ClangBuildAnalyzer alongside CompileScore.

It dumps out an aggregate summary that I find pretty useful (ex: longest files to parse, codegen, longest template sets, etc)

Might be useful in CompileScore?

Not new bug, at least since I first started using the extension. Just finally hit it enough to cross my threshold for reporting :P

I looked into it initially and iirc, the issue was with < & > not escaped when serialized to some buffer (maybe the compile.scor file?) but it's been a months

The problematic headers are things like #include <windows>

Stack trace:

System.ArgumentException: Illegal characters in path.
   
at System.IO.Path.CheckInvalidPathChars(String path, Boolean checkAdditional)

at System.IO.Path.GetFileName(String path)

at CompileScore.ScoreGlyphTagger.CreateTrackingSpans()

at CompileScore.ScoreGlyphTagger..ctor(ITextView view, ITextBuffer sourceBuffer)

at CompileScore.ScoreGlyphTaggerProvider.CreateTagger[T](ITextView textView, ITextBuffer buffer)

at Microsoft.VisualStudio.Text.Tagging.Implementation.TagAggregator`1.InstantiateViewTagger(ITextView textView, Lazy`2 factory, ITextBuffer textBuffer, IList`1 newTaggers)
--- End of stack trace from previous location where exception was thrown ---

at Microsoft.VisualStudio.Telemetry.WindowsErrorReporting.WatsonReport.GetClrWatsonExceptionInfo(Exception exceptionObject)

I'm using VS2022. Extension version: 1.8.4. When I'm opening a solution extension crashes with error in ActivityLog:

SetSite failed for package [CompileScorePackage]Source: 'Microsoft.VisualStudio.Text.Data' Description: Specified argument was out of the range of valid values. Parameter name: length System.ArgumentOutOfRangeException: Specified argument was out of the range of valid values. Parameter name: length at Microsoft.VisualStudio.Text.Span..ctor(Int32 start, Int32 length) at CompileScore.ScoreGlyphTagger.RefreshTags() at CompileScore.Notify.Invoke() at CompileScore.CompilerData.TryNotifyDataChanged() at CompileScore.CompilerData.OnSolutionSettingsChanged() at CompileScore.CompilerData.OnEditorModeChanged() at CompileScore.Notify.Invoke() at CompileScore.EditorContext.SetMode(EditorMode input) at CompileScore.EditorContext.CheckAlreadyOpenedContext() at CompileScore.CompileScorePackage.<InitializeAsync>d__3.MoveNext() --- End of stack trace from previous location where exception was thrown --- at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw() at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task) at Microsoft.VisualStudio.Shell.AsyncPackage.<>c__DisplayClass21_0.<<Microsoft-VisualStudio-Shell-Interop-IAsyncLoadablePackageInitialize-Initialize>b__1>d.MoveNext() --- End of stack trace from previous location where exception was thrown --- at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw() at Microsoft.VisualStudio.Services.VsTask.RethrowException(AggregateException e) at Microsoft.VisualStudio.Services.VsTask.InternalGetResult(Boolean ignoreUIThreadCheck)

At first, I thought that I was due to deleting a platform but crash acts very strangely and now I have no idea why can it be. I'm creating a new empty solution - it works. But when I'm starting to add file or change number of configurations/platforms, it can start crashing.

An option to only rebuild and profile a single project would be great. On large projects iterating over an entire solution is not very efficient.

Is it possible to keep the previous compil scores, and compare it to another ?

It would be nice to have a option to show the Compile Score Timeline with the average time, if you have a lot of modules with different pch that would be really helpful.

Tried the standalone app on a big project. Very nice tool!
A few small things I noticed:

• By default the horizontal space isn't fully used. I always have to manually widen the Name tab.
• In the include graph the hover popup just shows the filename, which is redundant if it's already visible. It should (maybe in addition) show the full path. (A similar popup would be useful in other places like Units when you have several CUs with the same filename.)
• Imho the default sorting order should be Duration/Accumulated descending, as this is probably the most useful metric to find the places to start optimizing.
• Table sorting performance could be better, Optimize Function has 1 million entries and takes a while to sort. If you click multiple times the app is rather unusable for a while until it catches up.
• In Parse Template the top entry is <unknown>, not sure what to make of that. When I look at the max json file it seems fine.
• It shows total build time as around 3 minutes while the build actually took like 20. Not sure where that number comes from. The aggregated cost of multiple hours of cpu time seems about right otoh.
• Can't copy a whole row of a table.

MSVC crashes when I

• drag the Compile Score window to a docking space
• right-click a column header in the Compile Score window

Installed version is 1.5.6.

Kind regards, Benni

Currently when I build a project with IB, CompileScore will only show the results from my machine, so I can choose between partial data, or a 7 hours build time.
Is it possible to add Incredibuild support? Awesome plugin btw.

The files listed in the timeline aren't related to or included by each other -- it seems like maybe the database got corrupted, or somehow a pointer to a data structure is pointing to the wrong element.

Here, we see one cpp file ostensibly including a couple of totally separate CPP files (which in reality are not included by each other), and eventually building some bullet code -- which, again, is unrelated to everything above it in this hierarchy.

I've been using CompileScore for some time now to profile compile times in my personal projects, excellent tool and amazing job developing it.

One thing I've found is that headers with the same name but different path are hard to tell apart, e.g. a header called string.h that lives in one path and another in another path will get bundled together. However, if string.h is included via the <> then they seem to appear as separate includes.

The other issue is that even if they appeared as different in the image below, it's hard to tell which one is which, so another suggestion is to (optionally?) show absolute paths in the columns to be able to properly tell which one is which.

Thanks again!

Intermittent crash if 'Copy Full Path' or 'Copy Name' are pressed multiple times. Crashes consistently between 2-10 copies.

CompileScore.exe!CompileScore.Overview.CompileDataTable.DataGridRow_ContextMenu.AnonymousMethod__4(object a, System.EventArgs b) Line 258 C#

System.Runtime.InteropServices.COMException: 'OpenClipboard Failed (Exception from HRESULT: 0x800401D0 (CLIPBRD_E_CANT_OPEN))'

Will there also be a VSCode version at some point (clang/Linux)?
There's definitely interest: https://twitter.com/Donzanoid/status/1531934744044359681?s=20&t=0TRC9hAWTOD10VpQsn-a7Q

Edit: Though now I see that even the ScoreDataExtractor is a Windows application.

Can I simply add -ftime-trace to our compiler settings in Make? I'm only seeing an empty UI from building the VC solution, but it's mostly a call over to Make and then the files for reference and setting breakpoints.

Unsure why, but the VS extension seems to have trouble loading external files that the Compiler Score file loads properly.

Related to that, I'm not able to get any timings inline in the source files at all.

• And, "Show Timeline" is greyed out in the context menu

It seems hard to get it the .vsix extension to work at all with an externally generated score file -- it would be nice if the UI had an explicit "load from file" or "reload from file" button.

Just looked at Instance Function and there were many individual entries of the form class<T>::method<U>.
It would be nice to have the option to aggregate them by method, by class or both (displayed like class<...>::method<...>).

Hi !

Does this plugin works on Linux or Mac ? I don't see any .vsix to install it.

Cheers !

Excellent project

TLDR

Noticed this crash on:

if m_activeTU is null => m_activeTUI->timeline.name will crash. Changed it to

if (m_activeTU && category == CompileCategory::BackEnd)
{ 
  FinalizeTU(m_activeTU->timeline.name);
}

Investigation

Only did a quick perusal. Looks like OnCompilerPassEnded gets called with said translation unit before OnCompilerPassStart() for the backend phase. I added tracepoints around the specific file that causes the crash in the start phase and the finalize phase as I initially thought it might've been prematurely dealloced or removed or something but they were never triggered first

Unsure if a deeper logic error exists or if something else is malformed as i'm not too familiar with build insights or this extensions source code

https://github.com/Viladoman/CompileScore/releases/tag/v1.9.1
Looks like CompileScoreApp.zip is missing CompileScore.dll and CompileScore.runtimeconfig.json.
Took it from https://github.com/Viladoman/CompileScore/actions/runs/7615943981 and it works.

Hello,
I have noticed a problem: when 2 headers have the same filename they are consider to be the same file.
For instance all the stdafx.h of my project are shown as one file in the include graph. Another example it happens for time.h which can be the libc one or the one in my project (including as #include <MyProject/Time.h>)

PS: Compile Score is incredible, thank you!