nginx worker processes segfaulting about autom8n HOT 12 CLOSED

@dotsam - Please see https://www.nginx.com/resources/wiki/start/topics/tutorials/debugging/#

You can build your own binary on your cpanel server by running

git clone https://github.com/AnoopAlias/nDeploy.git
cd nDeploy/rpm_buildtree/
./nDeploy-nginx_build_script.centos7.sh

The script tries to rsync the binary over to rpm.piserve.com which you can ignore. The rpm will be present in cd nDeploy/rpm_buildtree/nginx-pkg-64-centos7

Which you can rpm -ivh

from autom8n.

This should most probably be something wrong in your os setup as the same binary works fine in many servers without issue and this is the first case of segfault reported.

Let me know if you were able to nail it down

from autom8n.

@dotsam - Were you able to find the cause?.

from autom8n.

The process that's segfaulting isn't doing a core dump for some reason, and the debug logs haven't been very revealing as of yet. Oddly enough, the segfaults never seem to result in any failed connections that I can see, but none the less, I don't like processes just segfaulting. I'm going to try and simplify down my configuration (currently wordpress fastcgi/php-fpm/naxsi/ssl/ipv6) and see if I can narrow it down that way.

from autom8n.

So I have this narrowed down a little. The segfault happens when nginx reloads its configuration when it receives a SIGHUP signal. I can send the signal myself, but there's also something causing nginx to reload itself (not sure about the internal workings of nginx, so I can't say if this is normal or not).

If I rebuild nginx without pagespeed, then the segfault doesn't occur under the same conditions. If I change the version of pagespeed to 1.9.32.10 (which is tagged as the "latest" release in the pagespeed repo) then things are also fine. So somewhere in these 151 commits is the change that's causing my segfault. I'm going to keep digging in to this a little more tomorrow.

from autom8n.

FYI as per https://developers.google.com/speed/pagespeed/module/build_ngx_pagespeed_from_source

We should be doing pagespeed 1.10.33.4 which is what nDeploy build script use

from autom8n.

Finally got core dumps working correctly, tracked it down to the shutdown procedure in pagespeed, and I've filed a bug with them apache/incubator-pagespeed-ngx#1120

from autom8n.

k cool

from autom8n.

The bug in ngx_pagespeed has been fixed. This was happening when the pagespeed module was compiled in, but had no config. I expect that this may actually be the case for a number of nDeploy users, but as it wasn't actually impacting anything but the workers as they shut down, nobody noticed. It's fixed in the latest 1.10.33.5 release, which is now the latest version referenced by the official Google site, so if you bump the version in nDeploy and rebuild, then this should all be fixed.

from autom8n.

Ah, so the way pagespeed tags/releases work, this isn't actually in the "release" archive, so simply changing the version number won't grab this change, probably not until they push 1.10.33.6 as the release version.

from autom8n.

Yeah I think it will get pushed in 1.10.33.6 . I will keep an eye on it and update as soon as this is released .

from autom8n.

Bug was fixed in upstream ngx_pagespeed .

from autom8n.