Bug: sam local start-api --warm-containers doesn't work on docker about aws-sam-cli HOT 8 CLOSED

Hi @cford1080, thanks for raising this issue. I was able to reproduce the problem and I've raised a PR #6697 that should hopefully resolve it.

There is another factor here, however, which is how Docker runs SAM CLI inside the container. You need to ensure that it is being run as the main process and that it's not being wrapped by sh. In those cases, the SIGTERM interrupt will never reach the SAM CLI process and there's nothing that SAM CLI can do to handle it. This article goes into more detail.

from aws-sam-cli.

We were able to find the root cause. It has to do with the --docker-volume-basedir overriding the file path that the observer watches which happens to be on the host machine instead of inside the container.

I've raised a draft PR which should solve this issue too but the integration tests for this might take me some more time.

from aws-sam-cli.

These two commands use the same underlying logic for starting and running containers, including file watching and volume mounting. The PRs I've raised above should address the issue for both start-api and start-lambda commands.

from aws-sam-cli.

@mildaniel
Thanks. I see that addresses the issue of spawned containers not being cleaned up, but what of the issue of warm-containers not working inside of a docker container and producing a FileObserverException?

from aws-sam-cli.

When using the warm-containers option to start-api running inside a docker container, it will also fail to resolve http://host.docker.internal and a timeout will result.

Omitting the warm-containers for either EAGER or LAZY resolves the issue.

from aws-sam-cli.

@ryanb77

When using the warm-containers option to start-api running inside a docker container, it will also fail to resolve http://host.docker.internal and a timeout will result.

This is mitigated by adding the extra_hosts directive:

extra_hosts:
    - host.docker.internal:host-gateway

Omitting the warm-containers for either EAGER or LAZY resolves the issue.

I am currently omitting this option, but I wish to use the feature. If you review the Steps to reproduce, you may notice that I have provided 2 workarounds where warm-containers is working (with exception for cleaning up spawned containers), but requires some suboptimal volume mapping. In the provided logs (not using any workaround, where warm-containers is failing), you may also notice the following:

sam-local-1  | 2024-02-13 22:48:13,235 | Resolving code path. Cwd=/home/cford1080/github/sample, CodeUri=./src
sam-local-1  | 2024-02-13 22:48:13,236 | Resolved absolute path to code is /home/cford1080/github/sample/src

I am at a loss as to why this is using the host directories rather than the container's.

from aws-sam-cli.

Thank you, @mildaniel. As I mentioned, this isn't causing any showstopping issues because the option can be left omitted for now, but some testing could be sped up significantly if the container can remain warm throughout.

Does this impact other sam local commands which use these same options? For example, if I open another project which uses start-lambda instead of start-api, would I run into the same issues with the same options, and if so, will the solutions in motion now resolve the issues for these options on all commands, assuming they resolve the issues reported for start-api? Or do the options have different implementations per command that would all need to be addressed individually if issues are present? Hope that all makes sense; I got lost myself along the way and rewrote that a few times.

from aws-sam-cli.

Patch is released in v1.113.0. Closing

from aws-sam-cli.