[Hub] Artifact and cache size about imhex HOT 11 OPEN

A complex but definitive solution to artifact size would be to store the debug info of release versions ourselves instead of bundling it in artifacts, and make ImHex upload stacktraces with code offsets to our server, where we could map them to files/lines again.

from imhex.

Ohh, I never knew symbol servers existing. That could be a way to solve the problem indeed. But I don't plan to do it right now. It someone wants to build a PoC, please do so. I'm imagining something like a function in ImHex that calls the symbol server when crashing, or an implementation in our the API web server when they receive raw "stacktraces" without symbols from ImHex instances

Some links that seem useful:
https://stackoverflow.com/a/35556262
https://docs.sentry.io/platforms/apple/data-management/debug-files/symbol-servers/
https://wiki.archlinux.org/title/Debuginfod (used by ArchLinux for downloading debug info for the libraries in pacman)

I think the pdbs are needed by the stack tracer implementation used , so that the debuggers servers are not something that can be used here.

I'm sure this can be worked around

from imhex.

LTO seems to influence the ccache cache sizes a lot
the ArchLinux ccache cache is 150MB with LTO, and 24MB without
See https://gist.github.com/iTrooz/740f00f0935e365534f5a76dab0e7738 to measure section sizes for ELF

from imhex.

building in Release mode instead of RelWithDebInfo helps a lot with artifacts size. For example, Ubuntu 22.04 DEB went from 132MB to 16.2MB. Windows Installer went from 217MB to 24.2MB
More information: https://github.com/iTrooz/ImHex/actions/runs/9231528139 and https://github.com/iTrooz/ImHex/actions/runs/9231536431

from imhex.

using -gz=zlib (or fallback on -gz doesn't seem to improve cache sizes (Checked on Ubuntu 22.04 and ArchLinux builds)

Artifacts sizes do not improve either. In fact, AppImage seems to have gone from 141MB to 162MB
Windows and MacOS do not support this option.

Note that actual ELF files produced drastically reduce in size (e.g. 140.6MiB to 56.4MiB for libimhex for Ubuntu 22.04). The reason we are not observing changes in artifacts is because packages formats (e.g. .deb, .rpm, .tar.zst..) are already compressed.

NOTE: This means that this optimisation would still be useful once the package installed

More information: https://github.com/iTrooz/ImHex/actions/runs/9231528139 https://github.com/iTrooz/ImHex/actions/runs/9235290159

from imhex.

Some software provides separate pdb file downloads for debugging, is this approch possible for ImHex?

from imhex.

Probably, but your approach is missing some details. Who would download and use these separate debugging files ?

I offer an answer to this in my last comment

from imhex.

Who would download and use these separate debugging files ?

AFAIK, WinDbg, "who" keep downloads symbol files automatically, until the disk is filled

from imhex.

If you have a source please share it, but I'm doubtful it would do that, because its not its purpose. WinDbg is a debugger, why would it even be installed on a user machine, and why would it manage storage

from imhex.

I think Crystal-Rain Slide means that debuggers can have symbol servers defined and when you try to debug code it downloads pdbs for libraries and things you may need. Those are microsoft servers though but you can use any server like a folder or an http address. I think the pdbs are needed by the stack tracer implementation used , so that the debuggers servers are not something that can be used here.

from imhex.

I don't know much about how the process of creating useful stack traces, but if symbol servers can be used for them then I suppose it would be the natural choice. symbol servers are not exclusive to windows, gdb also supports them and there may be linux servers that can be used as well. Im not 100% sure but i think it is likely.

from imhex.