Add support for HTTP2 requests in Curl about aws-lambda-layers HOT 4 CLOSED

Some notes for today.

The ARM layers do have support for HTTP2.

So the problem is limited to x86 layers built from remi's repo.

Let's start from scratch (public.ecr.aws/lambda/provided:al2-x86_64) and do everything again:

$ curl --version
curl 7.79.1 (x86_64-koji-linux-gnu) libcurl/7.79.1 OpenSSL/1.0.2k-fips zlib/1.2.7 libidn2/2.3.0 libssh2/1.4.3 nghttp2/1.41.0 OpenLDAP/2.4.44
Release-Date: 2021-09-22
Protocols: dict file ftp ftps gopher gophers http https imap imaps ldap ldaps mqtt pop3 pop3s rtsp scp sftp smb smbs smtp smtps telnet tftp
Features: alt-svc AsynchDNS GSS-API HSTS HTTP2 HTTPS-proxy IDN IPv6 Kerberos Largefile libz NTLM NTLM_WB SPNEGO SSL UnixSockets

$ yum install -y \
        https://dl.fedoraproject.org/pub/epel/epel-release-latest-7.noarch.rpm \
        https://rpms.remirepo.net/enterprise/remi-release-7.rpm \
        yum-utils \
        epel-release \
        curl
...

$ yum-config-manager --enable remi-php81
...

$ yum update -y && yum upgrade -y
...

$ yum install php-cli
...

$ php -v
PHP 8.1.14 (cli) (built: Jan  4 2023 06:45:14) (NTS gcc x86_64)
Copyright (c) The PHP Group
Zend Engine v4.1.14, Copyright (c) Zend Technologie

$ php -i | grep HTTP2
$ php -r "var_dump(get_defined_constants());" | grep CURL_HTTP
  ["CURL_HTTP_VERSION_1_0"]=>
  ["CURL_HTTP_VERSION_1_1"]=>
  ["CURL_HTTP_VERSION_NONE"]=>

No luck.

Note
It seems curl is already installed, we can remove yum install curl from our Dockerfiles, but that's a minor detail.

I have opened remicollet/remirepo#228 to try and get some help from Remi.

from aws-lambda-layers.

What is weird is that libnghttp2 is already installed in AL2…

ldd /lib64/php/modules/curl.so
	libcurl.so.4 => /lib64/libcurl.so.4 (0x0000004001a45000)
	libc.so.6 => /lib64/libc.so.6 (0x0000004001cdc000)
	libnghttp2.so.14 => /lib64/libnghttp2.so.14 (0x0000004002089000)
	...

And /lib64/libnghttp2.so.14 is already here in AL2. We can even try to install it:

yum install libnghttp2

...

Package libnghttp2-1.41.0-1.amzn2.x86_64 already installed and latest version

So I'm not sure what's missing here. We don't copy libnghttp2.so.14 in the Lambda layer because that file already exists on Lambda/in AL2.

from aws-lambda-layers.

Forgot to mention: the CURL version in the new layers is 7.79.1, so it should be more than enough to have the HTTP 2 constants. Something is missing but I'm not sure what.

from aws-lambda-layers.

So I'm not sure what's missing here. We don't copy libnghttp2.so.14 in the Lambda layer because that file already exists on Lambda/in AL2.

@mnapoli are you experiencing on x86, arm or both?
I've quickly checked and yes at least in the docker image looks like libnghttp2 is installed by default in AL2:

❯ docker run --rm -it --entrypoint= public.ecr.aws/lambda/provided:al2-x86_64 find / -iname 'libnghttp*'
/usr/lib64/libnghttp2.so.14.20.0
/usr/lib64/libnghttp2.so.14
/usr/share/licenses/libnghttp2-1.41.0

So... are you experiencing on the true runtime lambda layer, AL2 docker image or both? If only on rumtime lambda layer, worth to double check there's no diff between AL2 docker image & true lambda runtime layers.

Then, despite the libnghttp2 is provided, it then depends if installed libcurl has been compiled with libnghttp2 support or not. And it seems that libcurl is not provided by default in AL2 layer. And finally, it depends if php itself has been eventually compiled with libnghttp2.

And looks like... yes (at least on x86):

❯ docker run --rm -it --entrypoint=bash bref/php-81:2 ldd /opt/bref/extensions/curl.so | grep nghttp
        libnghttp2.so.14 => /lib64/libnghttp2.so.14 (0x00007f8915550000)

So, maybe Remi compiles php with php support included instead of letting it as a separate extension? So even if the extension looks like to support http2, in fact this extension is unused? Looks like no, because the extension is loaded:

bash-4.2# php -m | grep curl
curl

I've also tried to not load the extension and then php -i | grep -i curl returns nothing, so no curl built into php binary by default.

I have to admit, this drives me nuts!

from aws-lambda-layers.