async-aws / aws Goto Github PK
View Code? Open in Web Editor NEWAWS SDK with readable code and async responses
Home Page: https://async-aws.com
License: MIT License
AWS SDK with readable code and async responses
Home Page: https://async-aws.com
License: MIT License
The following code:
require_once __DIR__.'/vendor/autoload.php';
$s3 = new \AsyncAws\S3\S3Client([
'region' => 'eu-central-1',
]);
$bucket = 'async-aws-test';
$result = $s3->putObject([
'Bucket' => $bucket,
'Key' => 'foo/',
]);
$result->resolve();
Will give:
Uncaught AsyncAws\Core\Exception\Http\ClientException: HTTP/1.1 411 Length Required returned for "https://s3.eu-central-1.amazonaws.com/async-aws-test/foo/".
Ie PutObjectRequest
.
When trying to mock SqsClient
from the official AWS SDK i get the following error:
Trying to configure method "sendMessage" which cannot be configured because it does not exist, has not been specified, is final, or is static
because the method does not belong class but is somehow added to it. Please keep this in mind when releasing the stable version and dont make the same structure as the official SDK.
We look for "structure" or "list" on many places, but we fail to look for "map".
We should always consider these 3 types of combined types.
That allows us to test other clients more easily.
I have a Bref based lambda function processing files stored in an s3 bucket on object creation events. Files are accessed via Flysystem v2.
Composer require:
"bref/bref": "^0.5.16",
"async-aws/flysystem-s3": "^0.2.0",
When creating the S3Client I'm using these params:
$client = new S3Client([
'accessKeyId' => $config['username'],
'accessKeySecret' => $config['password'],
'region' => $config['region'],
]);
When trying to list files I'm getting this error
Code: InvalidToken
Message: The provided token is malformed or otherwise invalid.
From what I see in async-aws/core Configuration.php falls back to all env vars if an option is not specified. The Lambda has env vars for
AWS_ACCESS_KEY_ID
AWS_SECRET_ACCESS_KEY
AWS_SESSION_TOKEN
but I'm using a different user for accessing the s3 bucket when creating the client. I think when doing so the Configuration should not use the sessionToken from env vars.
I like the way symfony/http-client handle async calls, by processing the call when needed and hiding the complexity to the user.
as a user, I'd love to use something like:
$bucket = '...';
$client = (new AwsClient())->s3();
$objects = $client->listObjects(['bucket' => $bucket]); // async
// do something else...
foreach ($objects as $object) { // listObjects resolved here
$client->deleteObject(['bucket' => $bucket, 'key' => $objet->getKey()]); // async
}
// deletions resolved at destruct
For DX, and IDE autocompletion, I think that each method should return an Dedicated Object with Getter TypeHinted.
$bucket = '...';
$key = '...';
$client = (new AwsClient())->s3();
$object = $client->getObject(['bucket' => $bucket, 'key' => $key']); // async
$object->getContentLength(); // resolved here
I added it to the baseline, but it should be fixed.
There is a FIXME in the code in IniFileProvider::loadProfiles()
.
See #53 (comment)
The command:
./generate update —all
Should update all operations for all services.
├── CHANGELOG.md
├── composer.json
├── docs
│ ├── authentication.md
│ ├── ...
│ └── waiter.md
└── src
├── CodeGenerator
├── Core
├── Service
│ ├── CloudFormation
│ ├── Lambda
│ ├── S3
│ ├── Ses
│ ├── Sns
│ └── Sqs
└── Integration
├── Flysystem
| └── S3
└── Symfony
├── SqsTransport
└── Bundle
provides:
The AsyncAws\Core\Result has some private properties and some protected. Should we be consistent with these?
Should we be defensive and do all private?
Or are we okey with using protected?
If an exception is thrown on ->resolve()
, it can be caught. But then on __destruct()
, we will call resolve again and a new message is thrown.
The automated test generate stubs for Body, but in RestJson and RestXml, Uri and Headers are important too.
see for instance https://github.com/async-aws/aws/blob/master/src/Service/Lambda/tests/Unit/Input/AddLayerVersionPermissionRequestTest.php
provide a VarExporter Caster to created readable dumps. (without request, client, ...)
ie. tests are put in
src/Service/S3/src/Tests/Unit/Input/GetObjectAclRequestTest.php
instead of
src/Service/S3/tests/Unit/Input/GetObjectAclRequestTest.php
If someone submit a PR with a patch for a bug, how can that be tested before the PR is merged?
Each client is different. Some need ext-SimpleXML
and some need ext-json
. The composer.json should be updated accordingly.
How do we refer to the name of the organization?
I created #161 to make sure we are consistent.
What do you think is best?
We should simplify doc blocks by doing something like:
/**
* @param FooEnum::* $value
*/
See https://psalm.dev/docs/annotating_code/typing_in_psalm/
This will require us to replace our FooEnum::AVAILABLE_...
with a static function. Ie FooEnum::exists(string $value): bool
Currently, the only way to stream a response is to:
$result = ...;
$info = $result->getInfo();
foreach($httpClient->stream($info['response']) as $chunk) {
// ...
}
Do we want to provide a better way?
Should we treat these the same?
'Key' => 'foo.png',
'Key' => '/foo.png',
Currently we expect no leading slash. The problem occurs when we copy objects:
https://docs.aws.amazon.com/AmazonS3/latest/API/API_CopyObject.html#API_CopyObject_Examples
That assumes we do one with leading slash and one without:
$input = new CopyObjectRequest([
'Key' => 'my-second-image.jpg',
'Bucket' => 'my-bucket',
'CopySource' => '/bucket/my-image.jpg',
]);
Should we do it the symfony way? Ie both changelog in root and individual packages?
Alternatively we can just put changlog on the packages.
more information: peter-evans/create-pull-request#48 (comment)
We should remove it after the result is initialised.
This will make the Result object more pretty but we will possibly loose debugging info after resolving a successful request.
Related to #97
We could write integrations to
I just list a few of them here. I dont say we should focus on any of them.
This I think core is pretty much feature complete.
We could do automatic minor releases every Wednesday if all of this conditions are true:
On the S3 HeadObjectOutput there is a parameter "Metadata" that is never set.
I think, that all parameters on the object is the metadata..
Notging prevent dumping an invalid file which leads to uncompilable-file.
This could be avoided by linting the code (php -l
) after dumping it, and reverting the file in case of issue.
Add a GithubAction that dry-run the generation based on the latest available version, and trigger an alert/or create a PR when something changes.
That should fixes #257 (comment)
Looking at these examples: https://docs.aws.amazon.com/AmazonS3/latest/API/API_PutObjectAcl.html#API_PutObjectAcl_Examples
If one is using the x-amz-acl
header, then the request body should be empty. Currently we get an error because the body contains:
<?xml version="1.0" encoding="UTF-8"?>
Like everything else marked as a "timestamp"
It is just a minor thing. We should remove return type hint on parameters like PutObjectRequest::getBody()
.
We should also update the constructor doc block.
The correct signature is array|string|resource|\Traversable|\Closure
The official SDK contains JSON file describing every method/request/response
To provide a library compatible with official Documentation, I suggest to use those files to generates our DTO/Objects.
The content of simple-aws
is:
Optionnaly it will also provides:
$s3->putFile($path)
) (ie like psr/cache vs psr/simple-cache)What do you think?
This should not be valid:
$input = new \AsyncAws\S3\Input\GetObjectRequest();
$input->setBucket('bref-example');
$result = $s3->getObject($input);
Currently it will be a "listObject", which is way wrong..
The api2-json file provides hints on enum. (ie. the Runtime Type in in Lambda)
We should:
class ListLayerVersionsRequest
{
/**
* A runtime identifier. For example, `go1.x`.
*
* @var string|null
*/
private $CompatibleRuntime;
public function validate(): void
{
foreach ($this->CompatibleRuntime as $value) {
if (!isset(Runtime::AVAILABLE_RUNTIMES[$value])) {
throw new InvalidArgument(sprintf('The CompatibleRuntime "%s" in "%s" is not a Valide "Runtime". Available values are "%"', $value, __CLASS__, array_keys(Runtime::AVAILABLE_RUNTIMES)));
}
}
}
}
// Class is named "Runtime" because of the name of the shape
class Runtime
{
public const NODEJS = 'nodejs';
public const NODEJS_4_3 = 'nodejs4.3';
public const NODEJS_6_10 = 'nodejs6.10';
public const NODEJS_4_3_EDGE = 'nodejs4.3-edge';
// ..
// = "AVAILABLE_" + className.upper()
public const AVAILABLE_RUNTIME = [
self::NODEJS = true,
self::NODEJS_4_3 = true,
self::NODEJS_6_10 = true,
self::NODEJS_4_3_EDGE = true,
// ..
];
}
What if:
Day 0: Update in official SDK
Day 1: Bot creates a PR with updates
Day 2: We still dont merge that PR
Day 3: Update in official SDK
Day 4: ??
Will the PR be updated?
Will a new PR be created?
What if we merge the PR but not delete the branch?
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.