molecularmatters / raw_pdb Goto Github PK

View Code? Open in Web Editor NEW

657.0 657.0 86.0 272 KB

A C++11 library for reading Microsoft Program DataBase PDB files

License: BSD 2-Clause "Simplified" License

C++ 97.32% C 1.72% CMake 0.96%

raw_pdb's People

Contributors

Stargazers

Watchers

raw_pdb's Issues

Can't build

Latest VS2022, Win11 SDK, clang ~~14.0.0~~ 15.0.2.

Output

C:\JetBrains\apps\CLion\ch-0\222.4167.35\bin\cmake\win\bin\cmake.exe -DCMAKE_BUILD_TYPE=Debug -DCMAKE_MAKE_PROGRAM=C:/JetBrains/apps/CLion/ch-0/222.4167.35/bin/ninja/win/ninja.exe "-DCMAKE_C_COMPILER=C:/Program Files/LLVM15/bin/clang-cl.exe" "-DCMAKE_CXX_COMPILER=C:/Program Files/LLVM15/bin/clang-cl.exe" -DCMAKE_C_FLAGS=-m64 -DCMAKE_CXX_FLAGS=-m64 -G Ninja -S C:\Downloads\raw_pdb-main -B C:\Downloads\raw_pdb-main\cmake-build-debug
-- The C compiler identification is Clang 15.0.2 with MSVC-like command-line
-- The CXX compiler identification is Clang 15.0.2 with MSVC-like command-line
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working C compiler: C:/Program Files/LLVM15/bin/clang-cl.exe - skipped
-- Detecting C compile features
-- Detecting C compile features - done
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Check for working CXX compiler: C:/Program Files/LLVM15/bin/clang-cl.exe - skipped
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Configuring done
-- Generating done
-- Build files have been written to: C:/Downloads/raw_pdb-main/cmake-build-debug

Cannot get compiler information:
	Compiler exited with error code 1: C:\Program Files\LLVM15\bin\clang-cl.exe /TP -IC:/Downloads/raw_pdb-main/src/. -m64 /Zi /Ob0 /Od /RTC1 -fdeclspec -fms-extensions /YcC:/Downloads/raw_pdb-main/cmake-build-debug/src/Examples/CMakeFiles/Examples.dir/cmake_pch.hxx /FpC:/Downloads/raw_pdb-main/cmake-build-debug/src/Examples/CMakeFiles/Examples.dir/./cmake_pch.cxx.pch /FIC:/Downloads/raw_pdb-main/cmake-build-debug/src/Examples/CMakeFiles/Examples.dir/cmake_pch.hxx /clang:-fpch-preprocess -v /clang:-dD -E
	clang-cl: warning: unknown argument ignored in clang-cl: '-fdeclspec' [-Wunknown-argument]
	clang version 15.0.2
	Target: x86_64-pc-windows-msvc
	Thread model: posix
	InstalledDir: C:\Program Files\LLVM15\bin
	 (in-process)
	 "C:\\Program Files\\LLVM15\\bin\\clang-cl.exe" -cc1 -triple x86_64-pc-windows-msvc19.33.31630 -E -disable-free -clear-ast-before-backend -disable-llvm-verifier -discard-value-names -main-file-name compiler-file2444810596800211601 -mrelocation-model pic -pic-level 2 -mframe-pointer=none -relaxed-aliasing -fmath-errno -ffp-contract=on -fno-rounding-math -mconstructor-aliases -funwind-tables=2 -target-cpu x86-64 -mllvm -x86-asm-syntax=intel -tune-cpu generic -D_MT -flto-visibility-public-std --dependent-lib=libcmt --dependent-lib=oldnames -stack-protector 2 -fms-volatile -fdiagnostics-format msvc -gno-column-info -gcodeview -debug-info-kind=constructor -v "-fcoverage-compilation-dir=C:\\Downloads\\raw_pdb-main\\cmake-build-debug\\src\\Examples" -resource-dir "C:\\Program Files\\LLVM15\\lib\\clang\\15.0.2" -include C:/Downloads/raw_pdb-main/cmake-build-debug/src/Examples/CMakeFiles/Examples.dir/cmake_pch.hxx -I C:/Downloads/raw_pdb-main/src/. -D ___CIDR_DEFINITIONS_END -internal-isystem "C:\\Program Files\\LLVM15\\lib\\clang\\15.0.2\\include" -internal-isystem "C:\\Program Files\\Microsoft Visual Studio\\2022\\Enterprise\\VC\\Tools\\MSVC\\14.33.31629\\include" -internal-isystem "C:\\Program Files\\Microsoft Visual Studio\\2022\\Enterprise\\VC\\Tools\\MSVC\\14.33.31629\\ATLMFC\\include" -internal-isystem "C:\\Program Files\\Microsoft Visual Studio\\2022\\Enterprise\\VC\\Auxiliary\\VS\\include" -internal-isystem "C:\\Program Files (x86)\\Windows Kits\\10\\include\\10.0.22621.0\\ucrt" -internal-isystem "C:\\Program Files (x86)\\Windows Kits\\10\\\\include\\10.0.22621.0\\\\um" -internal-isystem "C:\\Program Files (x86)\\Windows Kits\\10\\\\include\\10.0.22621.0\\\\shared" -internal-isystem "C:\\Program Files (x86)\\Windows Kits\\10\\\\include\\10.0.22621.0\\\\winrt" -internal-isystem "C:\\Program Files (x86)\\Windows Kits\\10\\\\include\\10.0.22621.0\\\\cppwinrt" -internal-isystem "C:\\Program Files (x86)\\Windows Kits\\NETFXSDK\\4.8\\include\\um" -internal-isystem "C:\\Program Files\\Microsoft Visual Studio\\2022\\Enterprise\\VC\\Tools\\MSVC\\14.33.31629\\include" -internal-isystem "C:\\Program Files\\Microsoft Visual Studio\\2022\\Enterprise\\VC\\Tools\\MSVC\\14.33.31629\\ATLMFC\\include" -internal-isystem "C:\\Program Files\\Microsoft Visual Studio\\2022\\Enterprise\\VC\\Auxiliary\\VS\\include" -internal-isystem "C:\\Program Files (x86)\\Windows Kits\\10\\include\\10.0.22621.0\\ucrt" -internal-isystem "C:\\Program Files (x86)\\Windows Kits\\10\\\\include\\10.0.22621.0\\\\um" -internal-isystem "C:\\Program Files (x86)\\Windows Kits\\10\\\\include\\10.0.22621.0\\\\shared" -internal-isystem "C:\\Program Files (x86)\\Windows Kits\\10\\\\include\\10.0.22621.0\\\\winrt" -internal-isystem "C:\\Program Files (x86)\\Windows Kits\\10\\\\include\\10.0.22621.0\\\\cppwinrt" -internal-isystem "C:\\Program Files (x86)\\Windows Kits\\NETFXSDK\\4.8\\include\\um" -O0 -fdeprecated-macro "-fdebug-compilation-dir=C:\\Downloads\\raw_pdb-main\\cmake-build-debug\\src\\Examples" -ferror-limit 19 -fno-use-cxa-atexit -fms-extensions -fms-compatibility -fms-compatibility-version=19.33.31630 -std=c++14 -fdelayed-template-parsing -fno-inline -dD -faddrsig -object-file-name=- -o - -x c++ "F:\\Users\\user\\AppData\\Local\\Temp\\compiler-file2444810596800211601"
	clang -cc1 version 15.0.2 based upon LLVM 15.0.2 default target x86_64-pc-windows-msvc
	ignoring duplicate directory "C:\Program Files\Microsoft Visual Studio\2022\Enterprise\VC\Tools\MSVC\14.33.31629\include"
	ignoring duplicate directory "C:\Program Files\Microsoft Visual Studio\2022\Enterprise\VC\Tools\MSVC\14.33.31629\ATLMFC\include"
	ignoring duplicate directory "C:\Program Files\Microsoft Visual Studio\2022\Enterprise\VC\Auxiliary\VS\include"
	ignoring duplicate directory "C:\Program Files (x86)\Windows Kits\10\include\10.0.22621.0\ucrt"
	ignoring duplicate directory "C:\Program Files (x86)\Windows Kits\10\\include\10.0.22621.0\\um"
	ignoring duplicate directory "C:\Program Files (x86)\Windows Kits\10\\include\10.0.22621.0\\shared"
	ignoring duplicate directory "C:\Program Files (x86)\Windows Kits\10\\include\10.0.22621.0\\winrt"
	ignoring duplicate directory "C:\Program Files (x86)\Windows Kits\10\\include\10.0.22621.0\\cppwinrt"
	ignoring duplicate directory "C:\Program Files (x86)\Windows Kits\NETFXSDK\4.8\include\um"
	#include "..." search starts here:
	#include <...> search starts here:
	 C:/Downloads/raw_pdb-main/src/.
	 C:\Program Files\LLVM15\lib\clang\15.0.2\include
	 C:\Program Files\Microsoft Visual Studio\2022\Enterprise\VC\Tools\MSVC\14.33.31629\include
	 C:\Program Files\Microsoft Visual Studio\2022\Enterprise\VC\Tools\MSVC\14.33.31629\ATLMFC\include
	 C:\Program Files\Microsoft Visual Studio\2022\Enterprise\VC\Auxiliary\VS\include
	 C:\Program Files (x86)\Windows Kits\10\include\10.0.22621.0\ucrt
	 C:\Program Files (x86)\Windows Kits\10\\include\10.0.22621.0\\um
	 C:\Program Files (x86)\Windows Kits\10\\include\10.0.22621.0\\shared
	 C:\Program Files (x86)\Windows Kits\10\\include\10.0.22621.0\\winrt
	 C:\Program Files (x86)\Windows Kits\10\\include\10.0.22621.0\\cppwinrt
	 C:\Program Files (x86)\Windows Kits\NETFXSDK\4.8\include\um
	End of search list.
	F:\Users\user\AppData\Local\Temp\compiler-file2444810596800211601(2,9): warning: macro name is a reserved identifier [-Wreserved-macro-identifier]
	#define ___CIDR_FEATURES_START
	        ^
	F:\Users\user\AppData\Local\Temp\compiler-file2444810596800211601(2085,26): error: too few arguments provided to function-like macro invocation
	#if __has_attribute(cdecl)
	                         ^
	F:\Users\user\AppData\Local\Temp\compiler-file2444810596800211601(2,9): warning: macro is not used [-Wunused-macros]
	#define ___CIDR_FEATURES_START
	        ^
	2 warnings and 1 error generated.
	
	
	Compiler exited with error code 1: C:\Program Files\LLVM15\bin\clang-cl.exe /TP -IC:/Downloads/raw_pdb-main/src/. -m64 /Zi /Ob0 /Od /RTC1 -fdeclspec -fms-extensions /YuC:/Downloads/raw_pdb-main/cmake-build-debug/src/Examples/CMakeFiles/Examples.dir/cmake_pch.hxx /FpC:/Downloads/raw_pdb-main/cmake-build-debug/src/Examples/CMakeFiles/Examples.dir/./cmake_pch.cxx.pch /FIC:/Downloads/raw_pdb-main/cmake-build-debug/src/Examples/CMakeFiles/Examples.dir/cmake_pch.hxx /clang:-fpch-preprocess -v /clang:-dD -E
	clang-cl: warning: unknown argument ignored in clang-cl: '-fdeclspec' [-Wunknown-argument]
	clang version 15.0.2
	Target: x86_64-pc-windows-msvc
	Thread model: posix
	InstalledDir: C:\Program Files\LLVM15\bin
	 (in-process)
	 "C:\\Program Files\\LLVM15\\bin\\clang-cl.exe" -cc1 -triple x86_64-pc-windows-msvc19.33.31630 -E -disable-free -clear-ast-before-backend -disable-llvm-verifier -discard-value-names -main-file-name compiler-file2444810596800211601 -mrelocation-model pic -pic-level 2 -mframe-pointer=none -relaxed-aliasing -fmath-errno -ffp-contract=on -fno-rounding-math -mconstructor-aliases -funwind-tables=2 -target-cpu x86-64 -mllvm -x86-asm-syntax=intel -tune-cpu generic -D_MT -flto-visibility-public-std --dependent-lib=libcmt --dependent-lib=oldnames -stack-protector 2 -fms-volatile -fdiagnostics-format msvc -gno-column-info -gcodeview -debug-info-kind=constructor -v "-fcoverage-compilation-dir=C:\\Downloads\\raw_pdb-main\\cmake-build-debug\\src\\Examples" -resource-dir "C:\\Program Files\\LLVM15\\lib\\clang\\15.0.2" -include C:/Downloads/raw_pdb-main/cmake-build-debug/src/Examples/CMakeFiles/Examples.dir/cmake_pch.hxx -I C:/Downloads/raw_pdb-main/src/. -D ___CIDR_DEFINITIONS_END -internal-isystem "C:\\Program Files\\LLVM15\\lib\\clang\\15.0.2\\include" -internal-isystem "C:\\Program Files\\Microsoft Visual Studio\\2022\\Enterprise\\VC\\Tools\\MSVC\\14.33.31629\\include" -internal-isystem "C:\\Program Files\\Microsoft Visual Studio\\2022\\Enterprise\\VC\\Tools\\MSVC\\14.33.31629\\ATLMFC\\include" -internal-isystem "C:\\Program Files\\Microsoft Visual Studio\\2022\\Enterprise\\VC\\Auxiliary\\VS\\include" -internal-isystem "C:\\Program Files (x86)\\Windows Kits\\10\\include\\10.0.22621.0\\ucrt" -internal-isystem "C:\\Program Files (x86)\\Windows Kits\\10\\\\include\\10.0.22621.0\\\\um" -internal-isystem "C:\\Program Files (x86)\\Windows Kits\\10\\\\include\\10.0.22621.0\\\\shared" -internal-isystem "C:\\Program Files (x86)\\Windows Kits\\10\\\\include\\10.0.22621.0\\\\winrt" -internal-isystem "C:\\Program Files (x86)\\Windows Kits\\10\\\\include\\10.0.22621.0\\\\cppwinrt" -internal-isystem "C:\\Program Files (x86)\\Windows Kits\\NETFXSDK\\4.8\\include\\um" -internal-isystem "C:\\Program Files\\Microsoft Visual Studio\\2022\\Enterprise\\VC\\Tools\\MSVC\\14.33.31629\\include" -internal-isystem "C:\\Program Files\\Microsoft Visual Studio\\2022\\Enterprise\\VC\\Tools\\MSVC\\14.33.31629\\ATLMFC\\include" -internal-isystem "C:\\Program Files\\Microsoft Visual Studio\\2022\\Enterprise\\VC\\Auxiliary\\VS\\include" -internal-isystem "C:\\Program Files (x86)\\Windows Kits\\10\\include\\10.0.22621.0\\ucrt" -internal-isystem "C:\\Program Files (x86)\\Windows Kits\\10\\\\include\\10.0.22621.0\\\\um" -internal-isystem "C:\\Program Files (x86)\\Windows Kits\\10\\\\include\\10.0.22621.0\\\\shared" -internal-isystem "C:\\Program Files (x86)\\Windows Kits\\10\\\\include\\10.0.22621.0\\\\winrt" -internal-isystem "C:\\Program Files (x86)\\Windows Kits\\10\\\\include\\10.0.22621.0\\\\cppwinrt" -internal-isystem "C:\\Program Files (x86)\\Windows Kits\\NETFXSDK\\4.8\\include\\um" -O0 -fdeprecated-macro "-fdebug-compilation-dir=C:\\Downloads\\raw_pdb-main\\cmake-build-debug\\src\\Examples" -ferror-limit 19 -fno-use-cxa-atexit -fms-extensions -fms-compatibility -fms-compatibility-version=19.33.31630 -std=c++14 -fdelayed-template-parsing -fno-inline -dD -faddrsig -object-file-name=- -o - -x c++ "F:\\Users\\user\\AppData\\Local\\Temp\\compiler-file2444810596800211601"
	clang -cc1 version 15.0.2 based upon LLVM 15.0.2 default target x86_64-pc-windows-msvc
	ignoring duplicate directory "C:\Program Files\Microsoft Visual Studio\2022\Enterprise\VC\Tools\MSVC\14.33.31629\include"
	ignoring duplicate directory "C:\Program Files\Microsoft Visual Studio\2022\Enterprise\VC\Tools\MSVC\14.33.31629\ATLMFC\include"
	ignoring duplicate directory "C:\Program Files\Microsoft Visual Studio\2022\Enterprise\VC\Auxiliary\VS\include"
	ignoring duplicate directory "C:\Program Files (x86)\Windows Kits\10\include\10.0.22621.0\ucrt"
	ignoring duplicate directory "C:\Program Files (x86)\Windows Kits\10\\include\10.0.22621.0\\um"
	ignoring duplicate directory "C:\Program Files (x86)\Windows Kits\10\\include\10.0.22621.0\\shared"
	ignoring duplicate directory "C:\Program Files (x86)\Windows Kits\10\\include\10.0.22621.0\\winrt"
	ignoring duplicate directory "C:\Program Files (x86)\Windows Kits\10\\include\10.0.22621.0\\cppwinrt"
	ignoring duplicate directory "C:\Program Files (x86)\Windows Kits\NETFXSDK\4.8\include\um"
	#include "..." search starts here:
	#include <...> search starts here:
	 C:/Downloads/raw_pdb-main/src/.
	 C:\Program Files\LLVM15\lib\clang\15.0.2\include
	 C:\Program Files\Microsoft Visual Studio\2022\Enterprise\VC\Tools\MSVC\14.33.31629\include
	 C:\Program Files\Microsoft Visual Studio\2022\Enterprise\VC\Tools\MSVC\14.33.31629\ATLMFC\include
	 C:\Program Files\Microsoft Visual Studio\2022\Enterprise\VC\Auxiliary\VS\include
	 C:\Program Files (x86)\Windows Kits\10\include\10.0.22621.0\ucrt
	 C:\Program Files (x86)\Windows Kits\10\\include\10.0.22621.0\\um
	 C:\Program Files (x86)\Windows Kits\10\\include\10.0.22621.0\\shared
	 C:\Program Files (x86)\Windows Kits\10\\include\10.0.22621.0\\winrt
	 C:\Program Files (x86)\Windows Kits\10\\include\10.0.22621.0\\cppwinrt
	 C:\Program Files (x86)\Windows Kits\NETFXSDK\4.8\include\um
	End of search list.
	F:\Users\user\AppData\Local\Temp\compiler-file2444810596800211601(2,9): warning: macro name is a reserved identifier [-Wreserved-macro-identifier]
	#define ___CIDR_FEATURES_START
	        ^
	F:\Users\user\AppData\Local\Temp\compiler-file2444810596800211601(2085,26): error: too few arguments provided to function-like macro invocation
	#if __has_attribute(cdecl)
	                         ^
	F:\Users\user\AppData\Local\Temp\compiler-file2444810596800211601(2,9): warning: macro is not used [-Wunused-macros]
	#define ___CIDR_FEATURES_START
	        ^
	2 warnings and 1 error generated.
	
	

[Failed to reload]

Qt6WebEngineCore.pdb error

6.52 Qt6WebEngineCore.pdb encountered a crash while parsing this PDB using this library, and the PDB exceeded 1g in size

Can a pdb file be modified？

just a question

a lot of repo tell me how read a .pdb file.
can we modify a .pdb file and make msbuild work successful?

Just to name one example.
if i modify a .obj path in .pdb file, can we be allowed?

can this work in Linux?

We are thinking of using this under Linux to parse PDB files. Has this library been compiled and run under Linux by anyone?

Support for function parameters

It seems there is no support for function parameters when enumerating function symbols. This would be a great addition to identify overloaded functions for example.

Running example "Lines"
| | | Reading image section stream
| | | ---> done in 0.079ms
| | | Reading module info stream
| | | ---> done in 0.131ms
| | | Reading names stream
| | | ---> done in 0.133ms
| | | Storing lines from modules

https://ufile.io/jki1dcht

ExampleTypes assert in DisplayFields()

This code asserts:

#include <fstream>

namespace
{
	struct empty : std::ifstream
	{
		empty() {}
	};
}

int main()
{
	return 0;
}

Due to no support for LF_IVBCLASS. If you change the struct to a class, works fine.

From what I can tell here with this limited test case, replicating https://github.com/microsoft/microsoft-pdb/blob/master/include/cvinfo.h#L2521 seems like it works, but I am not quite sure that's the whole picture. I haven't had any luck in narrow down a more representative test case from what I'm getting when running across our codebase here, so figured it best to just post an issue to see what you thought.

Why are so many rva the same, but the functions are different

RVA 0x1E5280

S_GPROC32 Function 'v8_inspector::V8InspectorClient::beginUserGesture' | RVA 0x1E5280
S_GPROC32 Function 'v8_inspector::V8InspectorClient::canExecuteScripts' | RVA 0x1F63E0
S_GPROC32 Function 'v8_inspector::V8InspectorClient::cancelTimer' | RVA 0x1E5280
S_GPROC32 Function 'v8_inspector::V8InspectorClient::consoleAPIMessage' | RVA 0x1E5280
S_GPROC32 Function 'v8_inspector::V8InspectorClient::consoleClear' | RVA 0x1E5280
S_GPROC32 Function 'v8_inspector::V8InspectorClient::consoleTime' | RVA 0x1E5280
S_GPROC32 Function 'v8_inspector::V8InspectorClient::consoleTimeEnd' | RVA 0x1E5280
S_GPROC32 Function 'v8_inspector::V8InspectorClient::consoleTimeStamp' | RVA 0x1E5280
S_GPROC32 Function 'v8_inspector::V8InspectorClient::currentTimeMS' | RVA 0x26E5F0
S_GPROC32 Function 'v8_inspector::V8InspectorClient::descriptionForValueSubtype' | RVA 0x229CA0
S_GPROC32 Function 'v8_inspector::V8InspectorClient::dispatchError' | RVA 0x1E5280
S_GPROC32 Function 'v8_inspector::V8InspectorClient::endEnsureAllContextsInGroup' | RVA 0x1E5280
S_GPROC32 Function 'v8_inspector::V8InspectorClient::endUserGesture' | RVA 0x1E5280
S_GPROC32 Function 'v8_inspector::V8InspectorClient::ensureDefaultContextInGroup' | RVA 0x229CA0
S_GPROC32 Function 'v8_inspector::V8InspectorClient::generateUniqueId' | RVA 0x201980
S_GPROC32 Function 'v8_inspector::V8InspectorClient::installAdditionalCommandLineAPI' | RVA 0x1E5280
S_GPROC32 Function 'v8_inspector::V8InspectorClient::isInspectableHeapObject' | RVA 0x1F63E0
S_GPROC32 Function 'v8_inspector::V8InspectorClient::maxAsyncCallStackDepthChanged' | RVA 0x1E5280
S_GPROC32 Function 'v8_inspector::V8InspectorClient::memoryInfo' | RVA 0x229CA0
S_GPROC32 Function 'v8_inspector::V8InspectorClient::muteMetrics' | RVA 0x1E5280
S_GPROC32 Function 'v8_inspector::V8InspectorClient::quitMessageLoopOnPause' | RVA 0x1E5280
S_GPROC32 Function 'v8_inspector::V8InspectorClient::resourceNameToUrl' | RVA 0x229CA0
S_GPROC32 Function 'v8_inspector::V8InspectorClient::runIfWaitingForDebugger' | RVA 0x1E5280
S_GPROC32 Function 'v8_inspector::V8InspectorClient::runMessageLoopOnInstrumentationPause' | RVA 0x3009C0
S_GPROC32 Function 'v8_inspector::V8InspectorClient::runMessageLoopOnPause' | RVA 0x1E5280
S_GPROC32 Function 'v8_inspector::V8InspectorClient::serializeToWebDriverValue' | RVA 0x229CA0
S_GPROC32 Function 'v8_inspector::V8InspectorClient::startRepeatingTimer' | RVA 0x1E5280
S_GPROC32 Function 'v8_inspector::V8InspectorClient::unmuteMetrics' | RVA 0x1E5280
S_GPROC32 Function 'v8_inspector::V8InspectorClient::valueSubtype' | RVA 0x229CA0

Default MSFStream Type - CoalescedMSFStream vs DirectMSFStream

I've hit a bit of an performance/memory usage issue with RawPDB.

I'm using the TPIStream with a 1.11 GB clang debug PDB and just creating the stream takes around 300 ms and uses around 240 MiB memory. The PDB has 2.962.141 type records and I only need to inspect some of them.

The issue is that the TPIStream uses CoalescedMSFStream internally and the TPIStream in this PDB is not continuous, so the entire stream is read into memory block by block. This works great if you want to read all the type records. But for my use case I would prefer to have it use DirectMSFStream instead and pay for the cost of reading the individual records lazily.

I'm looking for feedback on how to make a change to RawPDB that would support both use cases and fit with the overall design philosophy of the library.

Here are a some suggestions.

Somehow make TPIStream support both CoalescedMSFStream and DirectMSFStream. Perhaps at compile time with a template parameter or runtime with a flag?
Add a TPIDirectStream in addition to the existing TPIStream that uses DirectMSFStream.
Change TPIStream to use DirectMSFStream and make the user code/example read the type records they need and cache them there instead for fast lookup. E.g. moving the cost of reading all types from creation of the TPIStream in RawPDB to user code.

Also worth considering whether whatever approach is taken here could be used for other streams as well.

Look forward to hearing your thoughts :-)

Rewrite PDBs?

Hi, I'm new to the PDB file format. I found your project as well as Microsoft's. I found your project interesting b/c it actually compiles. I am poking around the example code now.

I'm looking for some advice about rewriting PDB files. I posted a question on Stackoverflow, hoping to get some advice from someone with more experience. Do you think it's possible to read a PDB file and write a new PDB with the source file paths changed? If so, how would you recommend going about this?

Any advice you can offer would be great, thanks.

Microsoft.VisualStudio.Coverage.Monitor.pdb bug

S_LPROC32 Function 'DuplicateLoggerBuffer' | RVA 0x1C70
S_LOCAL: 'CProfileMonitorControl' -> 'ProfileMonitorControl'
S_DEFRANGE_REGISTER: Register 0x 14A
S_DEFRANGE_REGISTER: Register 0x 14C
S_LOCAL: 'CLoggerBuffer' -> 'SourceBuffer'
S_DEFRANGE_REGISTER: Register 0x 14B
S_DEFRANGE_REGISTER: Register 0x 14E
S_LOCAL: '_GUID' -> 'ExecGuid'
S_DEFRANGE_REGISTER: Register 0x 150
S_DEFRANGE_REGISTER: Register 0x 156
S_LOCAL: '_BUFFERHEADER' -> 'DuplicateBufferHeader'
S_DEFRANGE_REGISTER: Register 0x 151
S_LOCAL: 'CLoggerBuffer' -> 'DuplicateBuffer'
S_DEFRANGE_REGISTER: Register 0x 14D
S_LOCAL: 'PUCHAR' -> 'SourceCoverageData'
S_DEFRANGE_REGISTER: Register 0x 14B
S_LOCAL: 'PUCHAR' -> 'DuplicateCoverageData'
S_DEFRANGE_REGISTER: Register 0x 14A
S_LOCAL: '_BUFFERHEADER' -> 'SourceBufferHeader'
S_DEFRANGE_REGISTER: Register 0x 14B
S_INLINESITE: Parent 0x 5A7C
S_INLINESITE: Parent 0x 5C60
S_LOCAL: 'ULONG' -> 'dwHandle'
S_DEFRANGE_REGISTER: Register 0x 14

https://ufile.io/846wqrra

Crash when trying to read symbols from `LLVMCodeGen.pdb`

When trying the "function symbol example" on the LLVMCodeGen.pdb (can be provided if necessary), there is a crash in AreBlockIndicesContiguous (blockIndices seem to be empty), when trying to call CreateImageSectionStream.

What is interesting: I added a CHECK(dbiStream.HasValidImageSectionStream(rawPdbFile) == PDB::ErrorCode::Success);, which succeeds.

When trying to open that PDB file with LLVM, hasPDBDbiStream is returning false.
The "dia2dump" example is printing a lot of function symbols, however, the "global function symbols" seem to be empty.

So maybe there is also something wrong with the pdb.

Dealing with directories being larger than a single block

Both raw_pdb and Llvm can't deal with pdbs that have more block indices than what fits in a single block. The DIA SDK seems to handle this case fine however. Although this rarely happens, we are dealing with pdb files that fail this (directoryBlockCount > blockIndicesPerBlock) validation check. This problem led us to move away from the Llvm implementation in favor of the DIA SDK's.

After some investigation on the problematic pdb files, which we unfortunately can not share, we found a workaround.

We assume that block indices will spill to the next block(s). Inspecting the block values confirmed this assumption with our problematic pdbs. For example, a pdb file that has 824 blocks in the directory but can only fit 256 indices per block will use 4 consecutives blocks for storing indices. In our case, all the indices are consecutives, even crossing block boundaries, and the last block of indices is zero-filled after the last index.

directoryBlockCount = 824
blockIndicesPerBlock = 256
superBlock->blockSize = 1024
superBlock->freeBlockMapIndex = 2
superBlock->blockCount = 210763
superBlock->directorySize = 843148
superBlock->unknown = 0
superBlock->directoryIndicesBlockIndex = 210754
superBlock->directorySize / superBlock->blockSize = 823
num_indices_blocks = 4

The workaround currently consists of making sure that blocks are contiguous in memory (which I think is always the case thanks to the CoalescedMSFStream(?)). This way, indexing from the address of the first index block will safely overflow to the subsequent index block(s). We could add some additional validation, like making sure the indices are in a valid range, and that the last block is indeed zero-filled after the last index. In any case, the fix seems to be working for us, but I'm curious to get your thoughts on the approach.

Do you have pdb files that trigger the check? I'll keep searching for such pdb files which I could share.

Thanks!

Examples project: NULL pointer dereference from appending NULL string

After compiling and running the example project with no modifications, it is crashing with a NULL pointer dereference as a result of appending a NULL char * in ExampleTypes.cpp on line 428.

PDB: https://groups.google.com/a/chromium.org/g/chromium-dev/c/6eJ_GVhjCmE
(.\RetrieveSymbols.exe D86C58DD-F8A3-6DD3-4C4C-44205044422E 1 chrome.dll.pdb)

Crash location

methodPrototype += typeName; // Line 428: typeName=nullptr

Callstack

Examples.exe!std::_Narrow_char_traits<char,int>::length(const char * const _First) Line 398	C++
Examples.exe!std::string::append(const char * const _Ptr) Line 3170	C++
Examples.exe!std::string::operator+=(const char * const _Ptr) Line 3112	C++
Examples.exe!GetMethodPrototype(const PDB::TPIStream & tpiStream, const PDB::CodeView::TPI::Record * methodRecord, std::string & methodPrototype) Line 429	C++
Examples.exe!DisplayFields(const PDB::TPIStream & tpiStream, const PDB::CodeView::TPI::Record * record) Line 669	C++
Examples.exe!ExampleTypes(const PDB::TPIStream & tpiStream) Line 769	C++
Examples.exe!main(int argc, char * * argv) Line 153	C++

Parsing the structure info

Is there any way to iterate over structs to get field names and offsets?

Unhandled record kind 0x1179

I'm getting the above error when running the examples through ExampleFunctionVariables(). This trivial sample will reproduce it:

#include <utility>
#include <vector>

namespace
{
	using whatever_vector = std::vector<std::pair<int32_t, int32_t>>;
}

int main()
{
	whatever_vector temp;
	temp.emplace_back( 1, 2 );
	temp.emplace_back( 3, 4 );

	for ( auto& [one, two] : temp )
	{
		one += two;
	}

	return 0;
}

It's struggling with the variables used in the structured bindings. I can't find any documentation on 0x1179, and can't seem to figure out what it is. Any ideas?

It only happens if I try to examine a debug build, release works fine. It was last night's Twitter post about looking at pdb sizes that piqued my interest 😄

(I'm using Visual Studio 2022 version 17.7.3 here, Platform Toolset v143 - haven't been able to check other versions)

Examples fail to compile on Ubuntu 22.04.2

Compiling examples on Ubuntu:

$ lsb_release -a
No LSB modules are available.
Distributor ID: Ubuntu
Description:    Ubuntu 22.04.2 LTS
Release:        22.04
Codename:       jammy

$ git log
commit c5308d53a66349f986e97c359a5a96ba12ee1003 (HEAD -> main, origin/main, origin/HEAD)
Merge: 3d876c8 dd0af39
Author: MolecularMatters <[email protected]>
Date:   Thu Dec 15 17:57:42 2022 +0100

    Merge pull request #40 from jeremyong/patch-1

    Resolve typo in GetModifierName

$ ninja
[1/2] Building CXX object src/Examples/CMakeFiles/Examples.dir/ExampleLines.cpp.o
FAILED: src/Examples/CMakeFiles/Examples.dir/ExampleLines.cpp.o
/usr/bin/c++  -I/home/user/proj/raw_pdb/src/. -O3 -DNDEBUG -std=gnu++11 -Winvalid-pch -include /home/user/proj/raw_pdb/build/src/Examples/CMakeFiles/Examples.dir/cmake_pch.hxx -MD -MT src/Examples/CMakeFiles/Examples.dir/ExampleLines.cpp.o -MF src/Examples/CMakeFiles/Examples.dir/ExampleLines.cpp.o.d -o src/Examples/CMakeFiles/Examples.dir/ExampleLines.cpp.o -c /home/user/proj/raw_pdb/src/Examples/ExampleLines.cpp
/home/user/proj/raw_pdb/src/Examples/ExampleLines.cpp: In function ‘void ExampleLines(const PDB::RawFile&, const PDB::DBIStream&, const PDB::InfoStream&)’:
/home/user/proj/raw_pdb/src/Examples/ExampleLines.cpp:183:38: error: ‘memcpy’ is not a member of ‘std’; did you mean ‘wmemcpy’?
  183 |                                 std::memcpy(line.checksum, checksumHeader->checksum, checksumHeader->checksumSize);
      |                                      ^~~~~~
      |                                      wmemcpy
ninja: build stopped: subcommand failed.

Adding #include <cstring> at the top of ExampleLines.cpp fixes the issue, but I was wondering if there's a more canonical place to put the include?

Support for /DEBUG:FASTLINK

I've been using DIA previously to get all function symbols of a .exe, and just noticed that while DIA spits out functions symbols for FastLink EXEs, raw_pdb just gives up. So for now, we still have the DIA fallback, but if this is something that will be added in the future, I'd appreciate it. No rush, though.

How to get type size

example:
unsigned __int64 get_size(const wchar_t* name)
{
unsigned __int64 uSize = 0;
do {
constexpr auto k_size = sizeof(SYMBOL_INFOW) + MAX_SYM_NAME * sizeof(wchar_t);
unsigned char buf[k_size]{};
auto info = reinterpret_cast<SYMBOL_INFOW*>(buf);
info->SizeOfStruct = k_size;
info->MaxNameLen = MAX_SYM_NAME;
if (SymGetTypeFromNameW(m_process_handle, m_symbol_base, name, info) == false)
break;
SymGetTypeInfo(m_process_handle, m_symbol_base, info->TypeIndex, TI_GET_LENGTH, &uSize);
} while (false);
return uSize;
};

What is the equivalent of IDiaSymbol::get_length?

To get the length in bytes of a function from its corresponding DIA symbol, we can use the get_length accessor. How could we retrieve the same information in raw_pdb?

Examples project: NULL pointer dereference

After compiling and running the example project with minor modifications to get it to build (diff included below), it is crashing with a NULL pointer dereference in ExampleTypes.cpp on line 184.

I've uploaded a crash dump here: https://easyupload.io/tode9c

PDB: https://groups.google.com/a/chromium.org/g/chromium-dev/c/6eJ_GVhjCmE
(.\RetrieveSymbols.exe 4E2A10AD-EED9-9C7F-4C4C-44205044422E 1 chrome.dll.pdb)

Crash location

case PDB::CodeView::TPI::TypeRecordKind::LF_MODIFIER:
    *modifierRecord = typeRecord;
    return GetTypeName(tpiStream, typeRecord->data.LF_MODIFIER.type, pointerLevel, nullptr, nullptr);
case PDB::CodeView::TPI::TypeRecordKind::LF_POINTER:
    ++pointerLevel;
    *referencedType = typeRecord; // Line 184: referencedType=nullptr (LF_MODIFIER calls with nullptr).

Callstack

Examples.exe!GetTypeName(const PDB::TPIStream & tpiStream, unsigned int typeIndex, unsigned char & pointerLevel, const PDB::CodeView::TPI::Record * * referencedType, const PDB::CodeView::TPI::Record * * modifierRecord) Line 184	C++
Examples.exe!GetTypeName(const PDB::TPIStream & tpiStream, unsigned int typeIndex, unsigned char & pointerLevel, const PDB::CodeView::TPI::Record * * referencedType, const PDB::CodeView::TPI::Record * * modifierRecord) Line 181	C++
Examples.exe!DisplayFields(const PDB::TPIStream & tpiStream, const PDB::CodeView::TPI::Record * record) Line 508	C++
Examples.exe!ExampleTypes(const PDB::TPIStream & tpiStream) Line 750	C++
Examples.exe!main(int argc, char * * argv) Line 153	C++

diff (moved two includes and commented out two unused variables.)

diff --git a/src/Examples/ExampleFunctionSymbols.cpp b/src/Examples/ExampleFunctionSymbols.cpp
index 41a8711..3ca3de9 100644
--- a/src/Examples/ExampleFunctionSymbols.cpp
+++ b/src/Examples/ExampleFunctionSymbols.cpp
@@ -208,8 +208,8 @@ void ExampleFunctionSymbols(const PDB::RawFile& rawPdbFile, const PDB::DBIStream
                                continue;
                        }

-                       const FunctionSymbol& nextSymbol = functionSymbols[i + 1u];
-                       const size_t size = nextSymbol.rva - currentSymbol.rva;
+                       //const FunctionSymbol& nextSymbol = functionSymbols[i + 1u];
+                       //const size_t size = nextSymbol.rva - currentSymbol.rva;
                        ++foundCount;
                }

diff --git a/src/Examples/ExampleTypes.cpp b/src/Examples/ExampleTypes.cpp
index 8c6bacf..82b50d5 100644
--- a/src/Examples/ExampleTypes.cpp
+++ b/src/Examples/ExampleTypes.cpp
@@ -1,6 +1,3 @@
-#include <cstring>
-#include <cinttypes>
-
 #include "Examples_PCH.h"
 #include "ExampleTimedScope.h"
 #include "PDB_RawFile.h"
@@ -8,6 +5,9 @@
 #include "PDB_DBIStream.h"
 #include "PDB_TPIStream.h"

+#include <cstring>
+#include <cinttypes>
+
 #pragma warning(push)
 #pragma warning(disable : 4061)
 #pragma warning(disable : 4774)

VS2022 build errors of unmodified project

2>D:\raw_pdb\src\Examples\ExampleTypes.cpp(4,10): error C2220: the following warning is treated as an error
2>D:\raw_pdb\src\Examples\ExampleTypes.cpp(4,10): warning C4654: Code placed before include of precompiled header line will be ignored. Add code to precompiled header.
2>D:\raw_pdb\src\Examples\ExampleTypes.cpp(723,19): error C2146: syntax error: missing ')' before identifier 'PRIu64'
2>D:\raw_pdb\src\Examples\ExampleTypes.cpp(723,10): warning C4473: 'printf' : not enough arguments passed for format string
2>D:\raw_pdb\src\Examples\ExampleTypes.cpp(723,10): message : placeholders and their parameters expect 1 variadic arguments, but 0 were provided
2>D:\raw_pdb\src\Examples\ExampleTypes.cpp(723,10): message : the missing variadic argument 1 is required by format string '%s'
2>D:\raw_pdb\src\Examples\ExampleTypes.cpp(723,19): error C2146: syntax error: missing ';' before identifier 'PRIu64'
2>D:\raw_pdb\src\Examples\ExampleTypes.cpp(723,19): error C2065: 'PRIu64': undeclared identifier
2>D:\raw_pdb\src\Examples\ExampleTypes.cpp(723,26): error C2143: syntax error: missing ';' before 'string'
2>D:\raw_pdb\src\Examples\ExampleTypes.cpp(723,26): warning C4548: expression before comma has no effect; expected expression with side-effect
2>D:\raw_pdb\src\Examples\ExampleTypes.cpp(723,47): error C2059: syntax error: ')'
2>D:\raw_pdb\src\Examples\ExampleFunctionSymbols.cpp(212,17): error C2220: the following warning is treated as an error
2>D:\raw_pdb\src\Examples\ExampleFunctionSymbols.cpp(212,17): warning C4189: 'size': local variable is initialized but not referenced
2>Done building project "Examples.vcxproj" -- FAILED.

What is the equivalent of IDiaDataSource::loadDataForExe?

ExampleFunctionSymbols trampling

Hi, I was playing around with the ExampleFunctionSymbols function when I accessed a corrupt frameProc pointer in the functionSymbols array at the end of the function.

For me it looks like, that the const PDB::CodeView::DBI::Record* record is freed (or trampled) in the const PDB::ModuleSymbolStream moduleSymbolStream (line 68) dtor - but stored in the functionSymbols array which has a longer lifetime.

Unhandled TypeRecordKind 0x110B

Tons of warnings emitted when building with clang 13.0.1

Due to the following code in PDB_Warnings.h, I get a 30+ warnings when compiling most of the .cpp files with clang 13.0.1

#elif PDB_COMPILER_CLANG
	// turn on absolutely all available Clang warnings
#	pragma clang diagnostic warning "-Wall"
#	pragma clang diagnostic warning "-Wextra"
#	pragma clang diagnostic warning "-Weverything"
#	pragma clang diagnostic warning "-Wpedantic"
...

I can fix the warnings by adding the following suppressions:

#   pragma clang diagnostic ignored "-Wextra-semi"
#   pragma clang diagnostic ignored "-Wzero-length-array"
#   pragma clang diagnostic ignored "-Wnewline-eof"
#   pragma clang diagnostic ignored "-Wreserved-identifier"
#   pragma clang diagnostic ignored "-Wused-but-marked-unused"
#   pragma clang diagnostic ignored "-Wmissing-field-initializers"
#   pragma clang diagnostic ignored "-Wmissing-prototypes"

Before I open a PR with these changes, I wondering if there is a reason for enabling all clang warnings and whether the code should be fixed instead or perhaps the warning levels should be relaxed? There are no warnings when building in VS2022.

Why do you not parse line information?

It seems fairly critical to me, if someone is making a debugger or anything like that.

Unhandled TypeRecordKind 0x1609

When running the Example.exe application on Windows, it always "dies" with the error message Unhandled TypeRecordKind 0x1609. Tested against multiple PDB files created with Visual Studio 2019.

I am not sure I understand the validity of PDB_NO_ALIAS

Hello !

I was trying to compile your project with llvm + MinGW, which requires some change in your code.

(I made it work so expect a PR soon :) )

I found the PDB_NO_ALIAS macro but I am not sure I understand why it is correct in your use case because the pointer returned by ArrayView::Decay is the same as the one provided to its constructor so it is an alias ?

Or did I misunderstand the very definition of aliasing ?

Cheers,
Scr3am

How is this library tested?

There are no tests shipped with the source code, are there internal tests and if so wouldn't it be better to ship them as well? High quality code is something that comes with tests. Memory safety is also a huge issue with this library, I wrote a small fuzzer for this library and it doesn't look too good. Considering this is supposed to replace DIA then it should at least have basic bound checks in place since people may load PDBs of untrusted sources.

Its probably best to include in the description that this library is potentially dangerous when PDBs are used from unknown sources.

molecularmatters / raw_pdb Goto Github PK

raw_pdb's People

Contributors

Stargazers

Watchers

Forkers

raw_pdb's Issues

Recommend Projects

Recommend Topics

Recommend Org