brefphp / aws-lambda-layers Goto Github PK

View Code? Open in Web Editor NEW

29.0 4.0 22.0 1.15 MB

Bref 2 scripts to build AWS Lambda layers

Home Page: https://bref.sh/docs/runtimes/

License: MIT License

Dockerfile 71.26% Shell 4.87% PHP 13.65% Makefile 7.77% HCL 2.45%

aws-lambda bref php

aws-lambda-layers's Issues

OpenSSL 3.2.0 update breaks connections to Aurora Postgres

Description:

When the container images were updated to include OpenSSL 3.2.0 in #143, subsequent deployments broke Aurora RDS connections to Postgres with the following error:

SQLSTATE[08006] [7] connection to server at "cluster.rds.amazonaws.com" (x.x.x.x), port 5432 failed: SSL error: ssl/tls alert handshake failure connection to server at "cluster.rds.amazonaws.com" (x.x.x.x.), port 5432 failed: FATAL:  no PostgreSQL user name specified in startup packet

How to reproduce:

Update the container image to include OpenSSL 3.2.0, RDS connection is broken
Lock the specific container image sha to a previous version, connection is restored

Missing CA file on latest beta

Description:

As discussed on Slack, the latest build (beta11) is missing the certificate file to handle SSL connections.

Stack trace from the Vimeo SDK:

{
    "errorType": "Vimeo\\Exceptions\\VimeoRequestException",
    "errorMessage": "Unable to complete request. [error setting certificate verify locations:  CAfile: /opt/ssl/cert.pem CApath: none]",
    "stackTrace": [
        "#0 /var/task/vendor/vimeo/vimeo-api/src/Vimeo/Vimeo.php(154): Vimeo\\Vimeo->_request('https://api.vim...', Array)",
        "#1 redacted",
        "#2 redacted",
        "#3 redacted",
        "#4 /var/task/vendor/bref/bref/src/Runtime/Invoker.php(24): redacted->handle(Array, Object(Bref\\Context\\Context))",
        "#5 /var/task/vendor/bref/bref/src/Runtime/LambdaRuntime.php(87): Bref\\Runtime\\Invoker->invoke(Object(redacted), Array, Object(Bref\\Context\\Context))",
        "#6 /var/task/vendor/bref/bref/src/FunctionRuntime/Main.php(32): Bref\\Runtime\\LambdaRuntime->processNextEvent(Object(redacted))",
        "#7 /opt/bref/bootstrap.php(17): Bref\\FunctionRuntime\\Main::run()",
        "#8 {main}"
    ]
}

How to reproduce:

Create a new function
Make a cURL request to an HTTPS endpoint

CI (build on every push)

Support all regions already supported by Bref 1

We need to sync all the regions here: https://github.com/brefphp/aws-lambda-layers/blob/main/utils/lambda-publish/Makefile with the ones here https://github.com/brefphp/bref/blob/master/runtime/layers/regions.json

Error using MongoDB on php 8.2

Description:

Error message output from Lambda

Warning: PHP Startup: Unable to load dynamic library 'mongodb.so' (tried: /opt/bref/extensions/mongodb.so (libssl.so.1.1: cannot open shared object file: No such file or directory), /opt/bref/extensions/mongodb.so.so (/opt/bref/extensions/mongodb.so.so: cannot open shared object file: No such file or directory)) in Unknown on line 0

How to reproduce:

functions:
  test:
    handler: public/test.php
    description: ''
    runtime: php-82
    layers:
      - ${bref-extra:mongodb-php-82}

In order for me to workaround this error on local docker, this is the command I added to Dockerfile

FROM bref/php-82-fpm-dev:2

# Fix "tried: /opt/bref/extensions/mongodb.so (libssl.so.1.1: cannot open shared object file: No such file or directory)" error
RUN yum -y install openssl11-libs.x86_64

"Integrity check" on all files

This action prevents earthquakes.

Would you merge a workflow that digs down to byte-level and pats every byte's back?

File permissions
EditorConfig
Git conflict markers
Move TODO-s into GitHub issues
Exported files (GitHub ZIP)
And more

https://github.com/szepeviktor/byte-level-care/blob/master/.github/workflows/reusable-integrity.yml#L39

Memory issue with bref/php-81-fpm-dev:2

Description:

There is a memory issue with the current image bref/php-81-fpm-dev:2 (digest b84118767b2174fc716ec1e1a2cf6bd4db4379e210909199e51735e443aea2ec)

Please see https://github.com/chrisjenkinson/bref-fpm-dev-image-issue/blob/main/.github/workflows/blank.yml which shows a GitHub workflow with the issue, using these images:

php:8.1.19
bref/php-80-fpm-dev:2
bref/php-81-fpm-dev:2
bref/php-82-fpm-dev:2

This is the result https://github.com/chrisjenkinson/bref-fpm-dev-image-issue/actions/runs/4965042554/jobs/8885601793

Fatal error: Allowed memory size of 3154116608 bytes exhausted (tried to allocate 2147483656 bytes) in phar:///project/vendor/phpstan/phpstan/phpstan.phar/vendor/nette/di/src/DI/DependencyChecker.php on line 47

PHPStan process crashed because it reached configured PHP memory limit: 3008M
Increase your memory limit in php.ini or run PHPStan with --memory-limit CLI option.

Can we also install Pcov by default in the fpm-dev image?

Pcov has a considerable performance improvement on generating coverage compared to Xdebug.
So I think it would be great to have this installed by default just like Xdebug so people can choose what to use.

For example also see: https://dyrynda.com.au/blog/using-pcov-instead-of-xdebug-for-coverage

Make the console layer self-contained

The Bref\ConsoleRuntime\Main class could be extracted (with it's Symfony Process dependency) to a separate package. Same as FPM.

release with PHP 8.1.17

Please could you provide the docker layers for PHP 8.1.17? Noting this diff: 2.0.3...main

Release (deploy Docker and layers on release)

Benchmark performances compared to Bref 1 layers

ARM images and layers

Move `/opt/php-fpm-runtime`

Move /opt/php-fpm-runtime to /opt/bref/php-fpm-runtime (namespace everything in /opt/bref).

Add PHP 8.2 x86 layers

Check out these docs: https://github.com/brefphp/aws-lambda-layers#supporting-a-new-php-version

Review all config files (diff with `bref/bref`) and re-apply changes

The codebase for this repository was forked from https://github.com/brefphp/bref many months ago.

We need to compare all config files (ini files, php-fpm.conf, etc.) to make sure we didn't drop changes or options that exist in https://github.com/brefphp/bref and not here.

Run all builds in GitHub Actions?

@deleugpn I'm opening this issue to start the discussion on "where to build and upload layers".

In the decision log, you wrote:

AWS CodeBuild is preferred for publishing the layers because the account that holds the layers has no external access. It is dedicated exclusively for having the layers only and only Matthieu Napoli has access to it.
GitHub Actions require exposing access to an external party. Using AWS CodeBuild allows us to use IAM Assume
Role so that one "Builder Account" can build the layers and then cross-publish them onto the "Layer Account".

With GitHub OIDC, we can have the GitHub action run with an IAM role, without having to store AWS access keys.

Considering that, do you still think CodeBuild might be better here?

I wonder about this because GitHub is much easier to debug (logs are public, with a nice UI, no need to log in, etc.), and run (nothing to set up). If I use GitHub Actions for everything, at least that's consistent across all repos, and lower bus factor (everything is public/easier to replicate).

Support Europe/Spain region

Recently a new AWS region was launched in Europe/Spain
I've opened a PR to add support for "eu-south-2" and be able to push layers to that region also

Unable to load APCu extension

Description:

APCu extension is unavailable in the runtime.

How to reproduce:

Following current docs: https://bref.sh/docs/environment/php.html#extensions

upgrade to v2 plugin
(uses layer arn:aws:lambda:eu-central-1:534081306603:layer:php-81:12)
add extension=apcu to ./php/conf.d/php.ini
deploy a function which uses APCu
test the function

Error message:

{
  "errorType": "Error",
  "errorMessage": "Call to undefined function apcu_fetch()",
...

Warning: PHP Startup: Unable to load dynamic library 'apcu' (tried: /usr/lib64/php/modules/apcu (/usr/lib64/php/modules/apcu: cannot open shared object file: No such file or directory), /usr/lib64/php/modules/apcu.so (/usr/lib64/php/modules/apcu.so: cannot open shared object file: No such file or directory)) in Unknown on line 0

Cleanup directories in `/tmp` in the `-build` images

I noticed that the build images are exported with temp content in /tmp (all compilation artifacts).

We could possibly make these images much smaller by cleaning up these folders.

Rewrite ARM layers using remirepo's ARM builds?

There is no support for ARM in RHEL 7: https://rpms.remirepo.net/enterprise/7/

However there are PHP 8.0 and 8.1 packages in RHEL 9: https://rpms.remirepo.net/enterprise/9/

Any idea if we could use those in Amazon Linux 2? (cc @deleugpn maybe)

Why: we'd get PHP 8.1 support, and latest versions of PHP (Amazon Linux Extras is behind in version numbers).

So far I only did a quick try:

docker run --rm -it --entrypoint=bash public.ecr.aws/lambda/provided:al2-arm64

Then in the container, I ran:

yum install -y \
         https://dl.fedoraproject.org/pub/epel/epel-release-latest-9.noarch.rpm \
         https://rpms.remirepo.net/enterprise/remi-release-9.rpm \
         yum-utils \
         epel-release \
         curl

And got errors.

Add support for HTTP2 requests in Curl

We had that issue in Bref 1 previously:

issue
PR

We need to fix that in the new layers as well.

Here is how I checked the problem:

Run bash in a Bref container
- docker run --rm -it --entrypoint=bash bref/php-81:1.7.15
- docker run --rm -it --entrypoint=bash bref/php-81:2
Check php -i | grep HTTP2 and php -r "var_dump(get_defined_constants());" | grep CURL_HTTP

I should see these constants:

  ["CURL_HTTP_VERSION_1_0"]=>
  ["CURL_HTTP_VERSION_1_1"]=>
  ["CURL_HTTP_VERSION_NONE"]=>
  ["CURL_HTTP_VERSION_2_0"]=>
  ["CURL_HTTP_VERSION_2"]=>
  ["CURL_HTTP_VERSION_2TLS"]=>
  ["CURL_HTTP_VERSION_2_PRIOR_KNOWLEDGE"]=>

But instead I see only these:

  ["CURL_HTTP_VERSION_1_0"]=>
  ["CURL_HTTP_VERSION_1_1"]=>
  ["CURL_HTTP_VERSION_NONE"]=>

In Bref 1, we compiled LIBNGHTTP2. In Bref 2 I'd love to find a solution that doesn't involve compiling (slow), if we can install a package that's better.

@shouze do you have any idea?

OpenSSL 3.0

OpenSSL 1.1 will be end of life in September 2023. PHP 8.1 and 8.2 support OpenSSL 3.0. It would be great if we could switch over to OpenSSL 3.0. This would have required more lift on Bref v1 due to sharing of the base images, but Bref v2 does not have this same sharing, so it is going to be easier to make this change only for PHP 8.1 and 8.2, and not 8.0.

FYI, I'm opening this issue with the intention to provide a PR for this, soon, unless there is an objection to making this change.

FYI2 - unfortunately the end of life for PHP 8.0 is after the end of life for any OpenSSL version that PHP 8.0 is compatible with. Maybe we should consider dropping PHP 8.0 support a little before the PHP 8.0 EOL in November 2023, as soon as OpenSSL 1.1 no longer will receive security patches, helping to encourage people to upgrade to PHP 8.1, so they are not in a huge mess if there's a vulnerability discovered, and there's no easy upgrade path to patch it.

Push `latest` Docker images on new release

This needs to be done once Bref v2.0 is released.

Console layer

Add a default Makefile at the root

bref v2 docker container does not properly accept POST input

Description:

I spun up a new project with bref v2, when using the docker info from here, I can’t seem to get it to read any POST data ($_POST and php://input are always empty). For a sanity check I redid everything with v1 and didn’t have any issues.

How to reproduce:

composer init
composer require bref/bref:^2.0@beta
./vendor/bin/bref init
(choose Web application)

docker-compose:
(8.1 and 8.2 exhibit same behavior)

version: "3.5"

services:
  app:
    image: bref/php-82-fpm-dev:2
    ports: [ '8000:8000' ]
    volumes:
      - .:/var/task
    environment:
      HANDLER: index.php

docker-compose up

2023-03-01T03:48:11.766731159Z 01 Mar 2023 03:48:11,764 [INFO] (rapid) exec '/var/runtime/bootstrap' (cwd=/var/task, handler=)
2023-03-01T03:48:13.001214724Z ⚡️ Server is running at http://localhost:8000
2023-03-01T03:48:17.758301451Z 01 Mar 2023 03:48:17,757 [INFO] (rapid) extensionsDisabledByLayer(/opt/disable-extensions-jwigqn8j) -> stat /opt/disable-extensions-jwigqn8j: no such file or directory
2023-03-01T03:48:17.758334744Z 01 Mar 2023 03:48:17,757 [WARNING] (rapid) Cannot list external agents error=open /opt/extensions: no such file or directory
2023-03-01T03:48:17.758713178Z START RequestId: 8dd01c10-b6d4-4e8a-a5f4-e59f1081350b Version: $LATEST
2023-03-01T03:48:18.409674374Z [01-Mar-2023 03:48:18] NOTICE: fpm is running, pid 31
2023-03-01T03:48:18.409692325Z [01-Mar-2023 03:48:18] NOTICE: ready to handle connections
2023-03-01T03:48:18.484529603Z END RequestId: 8dd01c10-b6d4-4e8a-a5f4-e59f1081350b
2023-03-01T03:48:18.484551424Z REPORT RequestId: 8dd01c10-b6d4-4e8a-a5f4-e59f1081350b	Init Duration: 0.49 ms	Duration: 726.00 ms	Billed Duration: 727 ms	Memory Size: 3008 MB	Max Memory Used: 3008 MB	

2023-03-01T03:51:14.041644317Z START RequestId: fe57f0f0-e5d1-4dde-8d9d-e1eb01c38124 Version: $LATEST
2023-03-01T03:51:14.057204662Z END RequestId: fe57f0f0-e5d1-4dde-8d9d-e1eb01c38124
2023-03-01T03:51:14.057288146Z REPORT RequestId: fe57f0f0-e5d1-4dde-8d9d-e1eb01c38124	Duration: 15.54 ms	Billed Duration: 16 ms	Memory Size: 3008 MB	Max Memory Used: 3008 MB

index.php:

<?php
var_dump($_POST);
var_dump(file_get_contents('php://input'));
var_dump($_SERVER);

Output of POST to localhost:8000

array(0) {
}
string(0) ""
array(57) {
["HOSTNAME"]=>
string(12) "314f399aa5fa"
["AWS_LAMBDA_FUNCTION_VERSION"]=>
string(7) "$LATEST"
["AWS_SESSION_TOKEN"]=>
string(0) ""
["PHP_INI_SCAN_DIR"]=>
string(25) ":/var/task/php/conf.dev.d"
["AWS_LAMBDA_LOG_GROUP_NAME"]=>
string(21) "/aws/lambda/Functions"
["LD_LIBRARY_PATH"]=>
string(94) "/var/lang/lib:/lib64:/usr/lib64:/var/runtime:/var/runtime/lib:/var/task:/var/task/lib:/opt/lib"
["LAMBDA_TASK_ROOT"]=>
string(9) "/var/task"
["AWS_LAMBDA_LOG_STREAM_NAME"]=>
string(7) "$LATEST"
["AWS_LAMBDA_RUNTIME_API"]=>
string(14) "127.0.0.1:9001"
["HANDLER"]=>
string(9) "index.php"
["AWS_LAMBDA_FUNCTION_NAME"]=>
string(13) "test_function"
["PATH"]=>
string(52) "/var/lang/bin:/usr/local/bin:/usr/bin/:/bin:/opt/bin"
["_"]=>
string(12) "/opt/bin/php"
["PWD"]=>
string(9) "/var/task"
["AWS_SECRET_ACCESS_KEY"]=>
string(0) ""
["LAMBDA_RUNTIME_DIR"]=>
string(12) "/var/runtime"
["LANG"]=>
string(11) "en_US.UTF-8"
["TZ"]=>
string(15) ":/etc/localtime"
["AWS_ACCESS_KEY_ID"]=>
string(0) ""
["SHLVL"]=>
string(1) "2"
["HOME"]=>
string(1) "/"
["_HANDLER"]=>
string(9) "index.php"
["RUNTIME_CLASS"]=>
string(20) "Bref\FpmRuntime\Main"
["AWS_LAMBDA_FUNCTION_MEMORY_SIZE"]=>
string(4) "3008"
["USER"]=>
string(6) "nobody"
["CONTENT_LENGTH"]=>
string(1) "0"
["CONTENT_TYPE"]=>
string(80) "multipart/form-data; boundary=--------------------------926422385474604171512351"
["SERVER_PROTOCOL"]=>
string(8) "HTTP/1.1"
["SERVER_NAME"]=>
string(14) "localhost:8000"
["SERVER_PORT"]=>
string(2) "80"
["SERVER_ADDR"]=>
string(9) "127.0.0.1"
["REMOTE_PORT"]=>
string(2) "80"
["REMOTE_ADDR"]=>
string(9) "127.0.0.1"
["SERVER_SOFTWARE"]=>
string(4) "bref"
["SCRIPT_FILENAME"]=>
string(19) "/var/task/index.php"
["REQUEST_URI"]=>
string(1) "/"
["REQUEST_METHOD"]=>
string(4) "POST"
["GATEWAY_INTERFACE"]=>
string(11) "FastCGI/1.0"
["HTTP_CONTENT_LENGTH"]=>
string(3) "161"
["HTTP_CONTENT_TYPE"]=>
string(80) "multipart/form-data; boundary=--------------------------926422385474604171512351"
["HTTP_CONNECTION"]=>
string(10) "keep-alive"
["HTTP_ACCEPT_ENCODING"]=>
string(17) "gzip, deflate, br"
["HTTP_HOST"]=>
string(14) "localhost:8000"
["HTTP_POSTMAN_TOKEN"]=>
string(36) "8b9192d8-3d44-49e6-b857-86fb27725cb2"
["HTTP_CACHE_CONTROL"]=>
string(8) "no-cache"
["HTTP_ACCEPT"]=>
string(3) "*/*"
["HTTP_USER_AGENT"]=>
string(21) "PostmanRuntime/7.30.0"
["LAMBDA_REQUEST_CONTEXT"]=>
string(329)
"{"http":{"method":"POST","path":"\/","protocol":"http","sourceIp":"::ffff:172.23.0.1","userAgent":"PostmanRuntime\/7.30.0"},"accountId":"123456789012","apiId":"api-id","domainName":"localhost","domainPrefix":"","requestId":"id","routeKey":"$default","stage":"$default","time":"2023-02-27T15:19:14.936Z","timeEpoch":1677511154936}"
["LAMBDA_INVOCATION_CONTEXT"]=>
string(179)
"{"awsRequestId":"1b62cfe1-f560-4a03-a037-fe237968de7f","deadlineMs":1677511454939,"invokedFunctionArn":"arn:aws:lambda:us-east-1:012345678912:function:test_function","traceId":""}"
["QUERY_STRING"]=>
string(0) ""
["PATH_INFO"]=>
string(1) "/"
["FCGI_ROLE"]=>
string(9) "RESPONDER"
["PHP_SELF"]=>
string(1) "/"
["REQUEST_TIME_FLOAT"]=>
float(1677511154.942073)
["REQUEST_TIME"]=>
int(1677511154)
["argv"]=>
array(0) {
}
["argc"]=>
int(0)
}

Add the zip extension by default for parity with Bref 1

New Bref 2 layers miss the zip extension (compared to Bref 1).

OpenSSL conf missing in non-dev Image

It seems like the openssl.cnf file is missing from the regular images (bref/php-83 or bref/php-83-fpm) but it is present in the ...-dev images.

> docker run -it --entrypoint /bin/sh bref/php-83-fpm-dev -c "ls /opt/bref/ssl/"

cert.pem  ct_log_list.cnf       misc         openssl.cnf.dist
certs     ct_log_list.cnf.dist  openssl.cnf  private

> docker run -it --entrypoint /bin/sh bref/php-83-fpm -c "ls /opt/bref/ssl/"

cert.pem

I noticed that openssl is tested in the line below, but the random bytes method works independently of the configuration. A test like creating private keys might be more reliable?

aws-lambda-layers/tests/test_2_extensions.php

Line 61 in 1ecad62

'openssl' => strlen(openssl_random_pseudo_bytes(1)) === 1,

brefphp / aws-lambda-layers Goto Github PK

aws-lambda-layers's Issues

Recommend Projects

Recommend Topics

Recommend Org