Autofill Memory Limits exceeded on relatively small database about keepassium HOT 5 OPEN

Dumping the prettyfied XML structure seems to clock in at around 18MB. This does not seem that much for the database itself objectively but it seems this is already too much, as autofill just dies silently.

I agree, such an XML this should not be too big of a deal. Dying silently means running out of memory at XML parsing or processing stage.

As much as I would love to skip custom icons and attachments while loading in AutoFill, this is quite problematic. Due to an early design mistake, KeePassium uses a DOM parser (which loads the whole XML tree to memory), rather than an event-based SAX parser (which parses XML in small chunks and can make decisions during the process). To my defense, at the time of this decision AutoFill was not even announced, so memory consumption was not that important.

Well, now memory became important, but optimization requires rewriting the whole database processing code. Which is a large all-or-nothing task that cannot be done incrementally. It is indeed on the list, but there is always something more urgent…

</rant>

there are some 300+kB favicons in there which is ridiculous.

This might be the reason.

When adding larger favicons, KeePassium specifically downsizes them to 128x128 PNG. But when showing the existing ones, resizing is done only in UI. Which means, according to this post, that the original image, in full resolution and colors, is decompressed to memory for displaying. I guess a 300 kB compressed PNG turns into quite a heavy bitmap…

(By the way, were the large favicons added by KeePassium? If so, there is probably a separate issue in the downsizing code.)

from keepassium.

I agree, such an XML this should not be too big of a deal. Dying silently means running out of memory at XML parsing or processing stage.

Thank you for confirming. This was my understanding as well.

As much as I would love to skip custom icons and attachments while loading in AutoFill, this is quite problematic. Due to an early design mistake, KeePassium uses a DOM parser (which loads the whole XML tree to memory), rather than an event-based SAX parser (which parses XML in small chunks and can make decisions during the process). To my defense, at the time of this decision AutoFill was not even announced, so memory consumption was not that important.

Right. DOM Parser seems like a sensible choice. If there were no memory pressure. Which there now unfortunately is. Bummer.

Well, now memory became important, but optimization requires rewriting the whole database processing code. Which is a large all-or-nothing task that cannot be done incrementally. It is indeed on the list, but there is always something more urgent…

Of course. But hey, easter is coming up, what better time to relax, ponder past sins and rewrite such code?! 😆

there are some 300+kB favicons in there which is ridiculous.

This might be the reason.

When adding larger favicons, KeePassium specifically downsizes them to 128x128 PNG. But when showing the existing ones, resizing is done only in UI. Which means, according to this post, that the original image, in full resolution and colors, is decompressed to memory for displaying. I guess a 300 kB compressed PNG turns into quite a heavy bitmap…

Interesting. Just read that post and if that's the case that would indeed explain the problem I'm seeing.

(By the way, were the large favicons added by KeePassium? If so, there is probably a separate issue in the downsizing code.)

I am not certain. I've been using KeepassXC and KeePassium in parallel and do not remember exactly, which icons were added where.

When I was looking for testcases for large icons in my database, I noticed a weird thing though:
The large icons in my db are all icons with 384 x 384px resolution.
But the source websites only offer smaller icons:

• https://www.wago.com/ - Their apple touch icon is only 180x180px: https://www.wago.com/_ui/static_2024-03-20_09-58/desktop/common/images/apple-touch-icon.png
• https://www.elektroradar.de/ - Their largest touch icon is 144px.

So it looks like something is actually saving the icons in a larger format than needed? Could that someone be KeePassium?

from keepassium.

Could that someone be KeePassium?

Guess what, you are absolutely right! KeePassium successfully "downscales" 180x180px icons into 128x128pt, which happens to be 384x384px on modern devices :) I've created a separate issue for that: #354

from keepassium.

Of course. But hey, easter is coming up, what better time to relax, ponder past sins and rewrite such code?! 😆

See, there is always something smaller and more urgent :)

from keepassium.

See, there is always something smaller and more urgent :)

🤯

I wiped the 384x384px icons from the database.
Database is now back down to about 1MB and Autofill is not dying anymore with that size.
So I guess this is a win? 😆

Of course, the overall problem is still unsolved, autofill should not be dying over icons...

from keepassium.