Giter VIP home page Giter VIP logo

sharpdebug's Introduction

SharpDebug: C# debugging automation tool

Set of .NET libraries that provide access to different debugging tools. SharpDebug.Engine provides framework for writting .NET code against different debuggers/dump processing. Extension libraries provide access to dbgeng.dll for processing Windows dumps, ability to read Linux core dumps, WinDbg/VisualStudio extension with interactive scripting support. Debugging both native and managed code is supported (currently, managed code is supported only for dbgeng.dll, WinDbg and Visual Studio extensions).

Latest status

Build status Code coverage Nuget version GitHub release

Latest version of Visual Studio extension is uploaded to Open VSIX Gallery. If you want newer build than what is available in Releases page or as nuget package, you can click on Latest build, select Configuration and click on Artifacts. You can also use private nuget feed from AppVeyor CI builds.

Debugger extensions:

Quick start for using engine as standalone application for dump processing

  • Create a new .NET project (you can use Console Application)
  • Add NuGet package SharpDebug
  • Start using it:
using SharpDebug;

DebuggerInitialization.OpenDump("path_to_dump_file", "symbol_path;srv*");
// After this line, you can execute any code that can be executed in a script. For example:
foreach (Module module in Process.Current.Modules)
    Console.WriteLine(module.Name);

Take a look at Tutorials and Automate dump processing. It will come in handy :)

Building the project

Prerequisites:

  1. .NET core 2.0
  2. Visual Studio Community 2017 (only for building WinDbg extension, VisualStudio extension)

Take a look at instructions.

Supporting the project

If you like the project, use it, "star" it, share ideas on how else it can be used, file issues, send pull requests, etc...

sharpdebug's People

Contributors

dasatomic avatar grlap avatar southpolenator avatar vujova avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

sharpdebug's Issues

Still Alive?

Is this project still alive? Hasn't had commits for a few years and the Nuget package is no longer available.

Dwarf FrameAddressInstruction reader errors

case DwarfCanonicalFrameAddressInstruction.register:
dummyLocation.Register = (int)data.LEB128();
dummyLocation.Register = (int)data.LEB128();

read register twice
https://elixir.bootlin.com/linux/v5.14.14/source/arch/arc/kernel/unwind.c#L790
Its supposed to be register and register value
not register and register again

case DwarfCanonicalFrameAddressInstruction.advance_loc1:
currentAddress = data.ReadByte();
break;
case DwarfCanonicalFrameAddressInstruction.advance_loc2:
currentAddress = data.ReadUshort();
break;
case DwarfCanonicalFrameAddressInstruction.advance_loc4:
currentAddress = data.ReadUint();
break;

all these advance_loc* are wrong, they should advance the current address, not set it.

https://elixir.bootlin.com/linux/v5.14.14/source/arch/arc/kernel/unwind.c#L742

Exception in "!interactive"

After loading the extension in WinDBG, and performing the command !CsDebugScript.interactive, the following error dump is thrown:

0:024> .load csext\CsDebugScript.WinDbg.dll
0:024> !CsDebugScript.interactive
System.TypeInitializationException: The type initializer for 'CsDebugScript.Executor' threw an exception. ---> System.TypeInitializationException: The type initializer for 'CsDebugScript.ScriptCompiler' threw an exception. ---> System.IO.FileNotFoundException: The system cannot find the file specified. (Exception from HRESULT: 0x80070002)
at System.Reflection.RuntimeAssembly.nLoadFile(String path, Evidence evidence)
at System.Reflection.Assembly.LoadFile(String path)
at CsDebugScript.ScriptCompiler.GetDefaultAssemblyReferences() in C:\Users\vujova\Documents\GitHub\WinDbgCs\Source\CsDebugScript.UI\ScriptCompiler.cs:line 62
at CsDebugScript.ScriptCompiler..cctor() in C:\Users\vujova\Documents\GitHub\WinDbgCs\Source\CsDebugScript.UI\ScriptCompiler.cs:line 36
--- End of inner exception stack trace ---
at CsDebugScript.InteractiveExecution..ctor() in C:\Users\vujova\Documents\GitHub\WinDbgCs\Source\CsDebugScript.UI\InteractiveExecution.cs:line 47
at CsDebugScript.Executor..cctor() in C:\Users\vujova\Documents\GitHub\WinDbgCs\Source\CsDebugScript.UI\Executor.cs:line 19
--- End of inner exception stack trace ---
at CsDebugScript.Executor.<>c.b__4_0() in C:\Users\vujova\Documents\GitHub\WinDbgCs\Source\CsDebugScript.UI\Executor.cs:line 58
at CsDebugScript.Engine.Debuggers.DbgEngDll.ExecuteAction(Action action) in C:\Users\vujova\Documents\GitHub\WinDbgCs\Source\CsDebugScript.Engine\Engine\Debuggers\DbgEngDll.cs:line 959

The culprit lines for future reference is the following line:

var assembly = Assembly.LoadFile(@"C:\Program Files (x86)\Reference Assemblies\Microsoft\Framework\.NETCore\v4.5\Microsoft.CSharp.dll");

For the time being, the only option is to copy that file in the same location as the extension. However, that would work only for 64bit machines if someone is using a 32bit machine that solution will not work.

--halsten

Easier script execution with VS

Path for searching scripts in Visual Studio should include all project directories and subdirectories as well as a solution directory.

CodeGen doesn't play nice with constexpr

At this moment, CodeGen will throw exception for some compile time constants (mostly constexpr, but some are static const) of complex types (like std::chrono).

Template types should be generated with constants too

There should exist wrapper class and interface that will let easy generation of "constants" in generics.
interface ITemplateConstant { T Value { get; } }
class TemplateConstant<T1, T2> where T1 is ITemplateConstant, new() { static T2 Value { get; } }

There should exist a list of all used constants and referenced from all generated template classes.

Example:
static class TemplateConstants
{
class Int20 : ITemplateConstant { int Value => 20; }
}

Fast usage: TemplateConstant<TemplateConstants.Int20, int>.Value (it returns 20)

This way, template user type can have correct CodeType name in metadata instead of current hack.

VS check initialization script

Let's start with having button to verify initialization script.
It would be great to have "editor extension" for highlighting syntax, errors, etc.

DwarfCompilationUnit.ReadData causes excessive memory usage on large binaries

All the Attributes parsed in DwarfSymbolProvider.DwarfCompilationUnit.ReadData are not deduplicated/interned.
For a big binary (in my case with debug info about 900MB) this will cause extreme memory usage.
Within the first 100 compilation units my memory usage rises to 12GB and then it gets stuck there because I ran out of memory.

As a ultra ugly hotfix I added this in DwarfSymbolProvider.ParseCompilationUnits

public class StringInterner
    {
        // deduplicate strings
        // meh https://github.com/dotnet/runtime/issues/21603 https://stackoverflow.com/questions/7760364/how-to-retrieve-actual-item-from-hashsett 
        ConcurrentDictionary<object, object> stringBank = new ConcurrentDictionary<object, object>();

        public object InternObject(object str)
        {
            if (str == null) return str;

            if (stringBank.TryGetValue(str, out var result))
            {
                return result;
            }

            stringBank.AddOrUpdate(str, str, (x,y)=> x);
            return str;
        }
    }
private static DwarfCompilationUnit[] ParseCompilationUnits(byte[] debugData, byte[] debugDataDescription, byte[] debugStrings, NormalizeAddressDelegate addressNormalizer)
        {
            using (DwarfMemoryReader debugDataReader = new DwarfMemoryReader(debugData))
            using (DwarfMemoryReader debugDataDescriptionReader = new DwarfMemoryReader(debugDataDescription))
            using (DwarfMemoryReader debugStringsReader = new DwarfMemoryReader(debugStrings))
            {
                List<DwarfCompilationUnit> compilationUnits = new List<DwarfCompilationUnit>();

                StringInterner interner = new StringInterner();

                List<Task> tasksList = new List<Task>();

                while (!debugDataReader.IsEnd)
                {
                    DwarfCompilationUnit compilationUnit = new DwarfCompilationUnit(debugDataReader, debugDataDescriptionReader, debugStringsReader, addressNormalizer, interner);

                    tasksList.Add(Task.Run(() =>
                    {
                        // intern all attributes in seperate threads

                        foreach (var compilationUnitSymbol in compilationUnit.Symbols)
                        {
                            compilationUnitSymbol.Attributes = 
                                compilationUnitSymbol.Attributes
                                    .Select(x => new KeyValuePair<DwarfAttribute, DwarfAttributeValue>(x.Key, interner.InternObject(x.Value) as DwarfAttributeValue))
                                    .ToDictionary(x => x.Key, x => x.Value);
                        }
                    }));




                    compilationUnits.Add(compilationUnit);
                }

                Task.WaitAll(tasksList.ToArray());

                return compilationUnits.ToArray();
            }
        }

This keeps my memory usage at the 400th compilation unit down at 7.7GB which is atleast usable.
I originally did the interning in DwarfCompilationUnit.data but that took too much time, the data reading is already the performance bottleneck, better not add anything extra to it.
Moving it out into a seperate thread/task works well for me so far.
One could probably intern the whole attribute instead of just the attribute value, not sure if that would be better, I assume it won't.

CommonUserTypes should be improved

CommonUserTypes should have pre-calculated structure that knows how to extract needed fields, so that it is not necessary to use getting fields from Variable. This structure should be kept in TypeSelector and populated in VerifyCodeType functions.
Some of the types can use this structure to be physical user type compatible.

ElfCoreDump is not respecting 4-byte padding between Note entries

https://docs.oracle.com/cd/E23824_01/html/819-0690/chapter6-18048.html

Padding is present, if necessary, to ensure 4-byte alignment for the descriptor. Such padding is not included in namesz.

Thats done correctly here with NameSize:

int nameEnd = reader.Position + (int)note.NameSize;

But the 4-byte padding to the next note is not considered.

The first descsz bytes in desc hold the note descriptor. If no descriptor is present, descsz contains the value zero. Padding is present, if necessary, to ensure 4-byte alignment for the next note entry. Such padding is not included in descsz.

descsz also needs to be 4 byte padded same as name.

Ghetto fix

string name = reader.ReadString();
reader.Position = nameEnd;
byte[] content = reader.ReadBlock(note.n_descsz);

                            string name = reader.ReadString();
                            reader.Position = nameEnd;
                            int descEnd = reader.Position + (int)note.DescSizeWithPadding;
                            byte[] content = reader.ReadBlock(note.n_descsz);
                            reader.Position = descEnd;

Will make a pull request someday when I get the time for it

ELF module not loading PublicSymbols for function name resolution

When you load a ELF module that doesn't have Dwarf debug information, but does have exported public functions
(Like for example libpthread that you load when trying to analyze a core dump)
These functions are not considered when trying to look up a function name by instruction pointer.

But it works perfectly fine if you consider them.

I know this is not the proper way to do it, but I want to note it here in case someone who can do it properly comes along.

Add

        public PublicSymbol(string name, ulong address, ELFSharp.ELF.Sections.SymbolType type, ulong size)
        {
            Name = name;
            Address = address;
            demangledName = SimpleCache.Create(() => Demangle(name));
            Type = type;
            Size = size;
        }

        public ELFSharp.ELF.Sections.SymbolType Type { get; private set; }

        public ulong Size { get; private set; }

here https://github.com/southpolenator/SharpDebug/blob/next/Source/SharpDebug.DwarfSymbolProvider/IDwarfImage.cs#L39

and

publicSymbols.Add(new PublicSymbol(symbol.Name, symbol.Value - CodeSegmentOffset,symbol.Type, symbol.Size));

here: https://github.com/southpolenator/SharpDebug/blob/next/Source/SharpDebug.DwarfSymbolProvider/ElfImage.cs#L60

and (this is the most hacky part)

 functionsCache.AddRange(
                            publicSymbols
                                .Where(x => x.Type == SymbolType.Function && x.Size != 0 && x.Address != 0)
                                .Where(x =>
                                    functionsCache.All(f =>
                                        f.GetConstantAttribute(DwarfAttribute.LowPc) != x.Address) // #TODO binary search
                            ).Select(publicSymbol =>
                            {

                                Dictionary<DwarfAttribute, DwarfAttributeValue> attributes = new Dictionary<DwarfAttribute, DwarfAttributeValue>();
                                attributes.Add(DwarfAttribute.Name, new DwarfAttributeValue() { Type = DwarfAttributeValueType.String, Value = publicSymbol.Name });
                                // LowPC is function start
                                attributes.Add(DwarfAttribute.LowPc, new DwarfAttributeValue() { Type = DwarfAttributeValueType.Address, Value = publicSymbol.Address });
                                // HighPc can either be address (end address) or constant (offset after start address, function size)
                                attributes.Add(DwarfAttribute.HighPc, new DwarfAttributeValue() { Type = DwarfAttributeValueType.Constant, Value = publicSymbol.Size });
                                attributes.Add(DwarfAttribute.ByteSize, new DwarfAttributeValue() { Type = DwarfAttributeValueType.Constant, Value = publicSymbol.Size });
                                attributes.Add(DwarfAttribute.Type, new DwarfAttributeValue() { Type = DwarfAttributeValueType.Constant, Value = publicSymbol.Size });

                                return new DwarfSymbol { Tag = DwarfTag.Subprogram, Attributes = attributes };
                            })
                        );

here: https://github.com/southpolenator/SharpDebug/blob/next/Source/SharpDebug.DwarfSymbolProvider/DwarfSymbolProviderModule.cs#L1697

After these changes you will be able to resolve function names in libraries without debugging info.

WaitForEvent should return an exit code

When using the DbgEng library to get direct access to the debugging APIs, I expect WaitForEvent to return an exit code. However, it returns void.

See the original documentation on WaitForEvent:
https://docs.microsoft.com/en-us/windows-hardware/drivers/ddi/content/dbgeng/nf-dbgeng-idebugcontrol-waitforevent

Return code Description
S_OK The method was successful.
S_FALSE The time-out expired.
E_PENDING An exit interrupt was issued. The target is not available.
E_UNEXPECTED Either there is an outstanding request for input, or none of the targets could generate events.
E_FAIL The engine is already waiting for an event.

I want to take some action based on whether or not WaitForEvent timed out, which is possible with the original API. With WinDbgCS, I need to roll my own mechanism to figure out whether we timed out.

Ability to get StackTrace from exception minidump

I have a exception on Thread 10320.
But when I just try to read the StackTrace from the thread I get pure garbage.

devenv_JkYlc8CVH5

I can even reproduce this in WinDbg

0:010> .cxr
Resetting default scope
0:010> k
 # Child-SP          RetAddr           Call Site
00 0000008c`6c7fbe18 00007ffe`5978d23b ntdll!NtGetContextThread+0x14
01 0000008c`6c7fbe20 000001bd`63871bfc ntdll!RtlpLocateActivationContextSection+0x13f
02 0000008c`6c7fbec0 00000000`00000002 0x000001bd`63871bfc
03 0000008c`6c7fbec8 00000000`00000002 0x2
04 0000008c`6c7fbed0 0000bddf`f1f90000 0x2
05 0000008c`6c7fbed8 0000bddf`f1f95675 0x0000bddf`f1f90000
06 0000008c`6c7fbee0 00000000`00000000 0x0000bddf`f1f95675

Current thread is just garbage. But if I switch context to the exception and dump the StackTrace again

0:010> .ecxr
rax=00007ff795801fd8 rbx=0000000000000001 rcx=000001bd27a7ff50
rdx=00007ff7958cad88 rsi=000001be54f6a050 rdi=0000000000000000
rip=00007ff7950116ef rsp=0000008c6c7ff1c8 rbp=0000000000000003
 r8=0000000000000000  r9=000000000000000e r10=0000000000000004
r11=0000008c6c7ff1c0 r12=0000000000000001 r13=000001bed219ffc0
r14=0000008c6c7ffa70 r15=0000000000000001
iopl=0         nv up ei pl nz na po nc
cs=0033  ss=002b  ds=002b  es=002b  fs=0053  gs=002b             efl=00010206
Arma3_x64!XX::GetClamp+0xf:
00007ff7`950116ef 48ffa0b0000000  jmp     qword ptr [rax+0B0h] ds:00007ff7`95802088={Arma3_x64!XX (00007ff7`9434cfd0)}
0:010> k
  *** Stack trace for last set context - .thread/.cxr resets it
 # Child-SP          RetAddr           Call Site
00 0000008c`6c7ff1c8 00007ff7`94fb809d Arma3_x64!XXX::GetClamp+0xf [X]
01 0000008c`6c7ff1d0 00007ff7`94f834b4 Arma3_x64!XXX::SetTextureAndMaterial+0x1fd [X]
02 0000008c`6c7ff2e0 00007ff7`94fa1973 Arma3_x64!XXX::DoPrepareTriangle+0x1444 [X]
03 0000008c`6c7ff390 00007ff7`94a22519 Arma3_x64!XXX+0x1e3 [X]
04 0000008c`6c7ff470 00007ff7`94a75bbd Arma3_x64!XXXL+0x1e9 X]
05 0000008c`6c7ff5d0 00007ff7`94a75887 Arma3_x64!XXX+0x29d [X]
06 0000008c`6c7ff740 00007ff7`94e169d6 Arma3_x64!XXX+0x187 [X]
07 0000008c`6c7ff7f0 00007ff7`94df1048 Arma3_x64!XXX+0x376 [X]
08 0000008c`6c7ff9e0 00007ff7`94f41bde Arma3_x64!XXXk::operator()+0xe8 [X]
09 0000008c`6c7ffa70 00007ff7`94f41fb7 Arma3_x64!XXX+0x12e [X]
0a 0000008c`6c7ffab0 00007ffe`57ab7034 Arma3_x64!InheritFPUSettings+0x57 [X]
0b 0000008c`6c7ffae0 00007ffe`597c2651 kernel32!BaseThreadInitThunk+0x14
0c 0000008c`6c7ffb10 00000000`00000000 ntdll!RtlUserThreadStart+0x21

Everything works perfectly fine.

I just cannot figure out how to do it in SharpDebug, well I know how to do it, but not how to do it with the API limitations.
There doesn't seem to be any API to switch the current context to the SharpDebug.DebugEventInfo.LastEvent

My hacky workaround is

using (ThreadSwitcher switcher = new ThreadSwitcher(StateCache, thread))

Make a copy of this method and remove the thread switcher which would reset the context to the thread.

And then do a very hacky

SharpDebug.Engine.Debuggers.DbgEngDll.ExecuteAndCapture(".ecxr");
var Adbg = SharpDebug.Engine.Context.Debugger as SharpDebug.Engine.Debuggers.DbgEngDll;
var Actx = Adbg.GetStackTraceFromContext(SharpDebug.Thread.Current, IntPtr.Zero, 0);

Basically change the context to the exception inside dbgeng, and then get stack from current active context (without setting context to thread again via the ThreadSwitcher)

This way works perfectly fine. But its quite a stupid solution.

Initialization script should exist for VS project

There should exist initialization script (Init.csx or similar) that should be executed when debugging starts (or when extension is loaded).
This initialization script can define functions, user types, include other scripts, define list of types that should be visualized by custom user code, etc.

Cannot build project

Reproduce steps

  1. clone repository
  2. run dotnet build

Expected result: build success

Actual result:

C:\Program Files\dotnet\sdk\5.0.101\NuGet.targets(131,5): error : Unable to load the service index for source https://powershell.myget.org/F/powershell-core/api/v3/index.json. [C:\Users\VITALY\source\repos\SharpDebug\SharpDebug.sln]
C:\Program Files\dotnet\sdk\5.0.101\NuGet.targets(131,5): error :   Response status code does not indicate success: 404 (Feed does not exist.). [C:\Users\VITALY\source\repos\SharpDebug\SharpDebug.sln]

Dwarf parser interface

I was looking for a managed Dwarf parser. I saw comments in ElfSharp about your dwarf parser. I took a quick look at the code it seemed pretty clean so I decided to kick the tires. I tried to use the as a standalone ELF Dwarf Symbol parser.

Summary of my findings:

  • The SharpDebug.DwarfSymbolProvider public constructor is not currently usable because the SharpDebug.Engine.Module constructor is internal.
  • konrad-kruczynski/elfsharp#49
  • There was an issue with NormalizeAddress finding the correct sections.
  • There were a lots of exceptions being thrown from unimplemented location decode functionality
  • There were lots of issues around decoding extra names.
  • The parser was very slow. The libcoreclr.so.dbg dwarf info is very large and parsing takes considerable time. For my purposes I only need a small subset of the dwarf symbols so converting them all to C# is unnecessary. I need a more on demand parser.
  • The typeId generated do not correlate to the debug info offset. I found this strange as it reduced traceability. It also forced a full decode of the info.

If you are interested my branch is here https://github.com/sdmaclea/SharpDebug/tree/DwarfSymbolProvider. With those changes and the change to ElfSharp I was able to fully parse the dwarf symbols from libcoreclr.so.dbg (see ElfSharp issue above for relevant instructions).

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.