Poor performance about base64-bytestring HOT 2 CLOSED

While there was indeed some extra performance to be squeezed out of base64-bytestring, I'd be astonished if that's what is making your application slow. My tweaked version of decodeLenient can decode in excess of 100MB/sec of data.

As far as I can tell, you've got something like 19 SSL connections in your trace above. I think that all those half million calls to decodeLenient are coming from Data.PEM.Parser, which means they're decoding lines that are about 72 bytes long. The old version of decodeLenient can decode these at 40MB/sec, since there's added overhead from the small lines and the sloppy implementation of Data.PEM.Parser. (The new version is about 50% faster.)

Since decodeLenient is being called half a million times, you're decoding about 20MB of PEM data, and it's taking roughly half a second (about 20 milliseconds per connection). That is a ludicrous amount of work to be doing for 19 SSL connections. I infer that something - probably a lot of somethings - is or are broken inside the tls code that you're using.

from base64-bytestring.

Yes, it looks like the problem is actually in the PEM parser, which is being called way too many times. For reference, here is the response from the author, Vincent Hanquez:

One of the problem of the current certificate verification is the design is terrible (stupid algorithm category). It's basically listing all files in /etc/ssl/certs and parsing them all one by one until the certificate you want is found. This is terribly wasteful, and this is a problem i'm trying to deal with, with the new CertificateStore object, so that we can have a "o(1)" access to the certificate we need for verification. This is not a easy thing to implement, but would solve the performance problem to a greater extend than speeding up PEM parsing.

from base64-bytestring.