aws-sdk dependency about fhir-works-on-aws-deployment HOT 9 CLOSED

Hi,

The zip file will indeed include the aws-sdk. This is expected behavior.

The aws-sdk is used by the persistence package to call DDB, S3, etc. The deployment package uses it for a few minor things related to bulk export(uploadGlueScriptsToS3.ts#L7)

It looks like the search package does not actually use the aws-sdk package, so we could clean up and remove it from the dependencies. This won't reduce the zip size though, since yarn resolves the dependencies for all packages and includes a single copy of the aws-sdk for the entire project.

You are right in that one of the main changes in the aws-sdk v3 is the ability to import individual modules for different AWS services whereas v2 is a single import that includes ALL aws services. Switching to v3 would reduce the our zip size since we only use clients for a handful of aws services (DDB, S3, Lambda, Step Functions, ES, etc. ). V3 is fairly recent (it went GA on Dec 15 2020) and it is not a drop in replacement for v2. We haven't prioritized making the upgrade.

It's worth noting that with a recent update(#240) to use webpack, the size of the zip package has been significantly reduced.

from fhir-works-on-aws-deployment.

Hi,
Is there a reason why aws-sdk is packed into the zip file - it is already available in the default Lambda layer when the function executes so doesn't need to be there?
Thanks,
Allan

from fhir-works-on-aws-deployment.

For apps in production it is a good practice to bundle the aws-sdk in order to pin it to the exact version that you are using to build and test your code.

from fhir-works-on-aws-deployment.

Yes that is a good point.
Looking at it pragmatically though the version of aws-sdk provided by the Lambda layer will always be 2.x.x so it is very unlikely that there will be a breaking change if semver is correctly followed by the sdk team.
Plus the dependency in each of the package.json files is specified using ^ so the minor & patch versions of the sdk used during fhirworks component test & release may well be different to that used when the component is installed in the deployment package.

from fhir-works-on-aws-deployment.

A breaking change is unlikely indeed, but there may be still be subtle changes in the sdk behavior. The recommended best practice is to fully control the dependencies in your function's deployment package.
https://docs.aws.amazon.com/lambda/latest/dg/best-practices.html#function-code

The yarn.lock file pins all packages to a specific version that fulfills the version ranges from the package.jsonfiles. This ensures that the exact same versions are used for building/testing and in prod
https://classic.yarnpkg.com/en/docs/yarn-lock/

from fhir-works-on-aws-deployment.

OK how about this as an approach:
If the FWoA sub components (persistence-ddb, search-es etc) are made independent of aws-sdk then we can follow best practice by including the sdk in the deployment component.
And then if a developer chooses to use the built in aws-sdk (and accept the potential pitfalls of this), they can do this by not including it in the deployment component.

from fhir-works-on-aws-deployment.

If the FWoA sub components (persistence-ddb, search-es etc) are made independent of aws-sdk then we can follow best practice by including the sdk in the deployment component.

Risk here will come from versioning ddb and es components, as say version 1 of ddb requires aws-sdk of 2.5 and ES requires 3.2 and then the deployment package dictates aws-sdk of 3.4. This will lead to either breaks or inconsistencies if the sub packages can't declare their own dependencies.

By default we want each FWoA packages to have all the dependencies they need. That said if the developer wants to remove it themselves they certainly can, but the mainline packages will keep the dependencies.

Thoughts?

from fhir-works-on-aws-deployment.

Hi Robert,
Yes I totally agree with what you are saying here, it is best to have full control of your dependencies.

My main rationale behind not wanting the aws-sdk inside the zipfile is to reduce package size to achieve better cold start performance, as I believe that this is the most significant factor in slow start ups.

I'm taking a pragmatic line here rather than following textbook - I don't believe that we'll ever see a version of aws-sdk greater than v2.x because v3 of the sdk is now being published with the @aws-sdk scope. You may know more than this though! Once the FWoA components are migrated v3.x code-size becomes less of an issue because of the modular approach, I haven't looked at this enough to know what impact it has though.

Thanks,
Allan

from fhir-works-on-aws-deployment.

+1 yeah I think migrating to v3 will be a good middle ground :)

from fhir-works-on-aws-deployment.