Comments (7)
bentsku
commented on May 26, 2024
2
@zachtayl no need for the reproducer, I've found the issue. Thanks for the report, I'll start working on it right away. I'll keep you updated here when the fix will be available in latest.
from localstack.
zachtayl
commented on May 26, 2024
1
@bentsku, the quick resolution on this is greatly appreciated!
I can confirm that the S3_TRUNCATE_KEYS does now function as expected and provides a fix for the original error.
from localstack.
bentsku
commented on May 26, 2024
Hello @zachtayl and thanks for your report!
We're aware of this issue and this will be resolved in the next release (3.1.0).
In the meantime, we have a feature flag available to fix this issue: S3_TRUNCATE_KEYS=1 available in the latest version.
Starting LocalStack with S3_TRUNCATE_KEYS=1 should solve your issue.
Beware however that if you are using PERSISTENCE=1 or cloud pods, you will need to delete and regenerate those, as the underlying format for S3 persisted data has changed.
Could you confirm this solve your issue? Thanks! And really sorry for the regression on this.
from localstack.
zachtayl
commented on May 26, 2024
Thanks for the quick response @bentsku!
Attempting to use the S3_TRUNCATE_KEYS=1 flag caused regression. When attempting to create s3 resources via Terraform Local, files with keys that seem to be longer than the given truncation are unable to be found thus apply fails. Other files with shorter file names do not seem to be affected in the same manner.
localstack_main | During handling of the above exception, another exception occurred:
localstack_main |
localstack_main | Traceback (most recent call last):
localstack_main | File "/opt/code/localstack/.venv/lib/python3.11/site-packages/localstack/aws/chain.py", line 99, in handle
localstack_main | handler(self, self.context, response)
localstack_main | File "/opt/code/localstack/.venv/lib/python3.11/site-packages/localstack/aws/handlers/service.py", line 112, in __call__
localstack_main | handler(chain, context, response)
localstack_main | File "/opt/code/localstack/.venv/lib/python3.11/site-packages/localstack/aws/handlers/service.py", line 82, in __call__
localstack_main | skeleton_response = self.skeleton.invoke(context)
localstack_main | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
localstack_main | File "/opt/code/localstack/.venv/lib/python3.11/site-packages/localstack/aws/skeleton.py", line 154, in invoke
localstack_main | return self.dispatch_request(serializer, context, instance)
localstack_main | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
localstack_main | File "/opt/code/localstack/.venv/lib/python3.11/site-packages/localstack/aws/skeleton.py", line 168, in dispatch_request
localstack_main | result = handler(context, instance) or {}
localstack_main | ^^^^^^^^^^^^^^^^^^^^^^^^^^
localstack_main | File "/opt/code/localstack/.venv/lib/python3.11/site-packages/localstack/aws/skeleton.py", line 118, in __call__
localstack_main | return self.fn(*args, **kwargs)
localstack_main | ^^^^^^^^^^^^^^^^^^^^^^^^
localstack_main | File "/opt/code/localstack/.venv/lib/python3.11/site-packages/localstack/aws/api/core.py", line 160, in operation_marker
localstack_main | return fn(*args, **kwargs)
localstack_main | ^^^^^^^^^^^^^^^^^^^
localstack_main | File "/opt/code/localstack/.venv/lib/python3.11/site-packages/localstack/services/s3/v3/provider.py", line 640, in put_object
localstack_main | s3_stored_object = self._storage_backend.open(bucket_name, s3_object)
localstack_main | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
localstack_main | File "/opt/code/localstack/.venv/lib/python3.11/site-packages/localstack_ext/services/s3/v3/storage.py.enc", line 114, in open
localstack_main | except FileNotFoundError:G=open(F,'w+b')
localstack_main | ^^^^^^^^^^^^^
localstack_main | FileNotFoundError: [Errno 2] No such file or directory: '/tmp/localstack/state/s3/omitomit/omitomitomi/omitomitomitomitomitomi/om.p_null_b821e1d92e1aa45295278e1d61ec204c'
localstack_main | 2024-01-10T18:44:30.247 INFO --- [ asgi_gw_9] localstack.request.aws : AWS s3.PutObject => 500 (InternalError)
We are able to workaround this issue given a fallback to the legacy_v2 s3 provider if there is no further resolution available. Thanks again!
from localstack.
bentsku
commented on May 26, 2024
Thanks for the quick feedback! We would really like to get this issue fixed for it to be able to be default for 3.1.0.
This regression is stumping me a bit, as we truncate on both "write" and "read" operation (Put and Get), so the logic should be similar between both.
Might I bother you a bit more, could you share a minimal reproducible sample? Thanks
from localstack.
bentsku
commented on May 26, 2024
Hello @zachtayl and thanks again for the report!
I believe the issue is now fixed in the latest docker image. Could you give it a try and see if that fixes your issue? I hope you can make your project work without using the legacy_v2 provider.
from localstack.
bentsku
commented on May 26, 2024
Awesome, thanks a lot! This should become default with 3.1.0, where you will be able to remove the feature flag. Thank you so much again for the report! I'll close the issue in the meantime (if somebody else encounters it, I'll link this issue), the tentative release date for 3.1.0 should be January 25th. Thanks!
from localstack.
Related Issues (20)
- bug: Does not return deleted output when use with AWS SDK HOT 1
- bug: Error running LocalStack in Docker on Windows 11 - initScript.sh not found despite being present HOT 2
- bug: `NoneType` object is not a mapping when deploying cdk stack HOT 1
- bug: Localstack still does not support import a key material when key usage is SIGN_VERIFY HOT 2
- bug: Kinesis shard iterator ShardIteratorType.AFTER_SEQUENCE_NUMBER not always honoring startingSequenceNumber HOT 1
- bug: StepFunctions: Doesnt support ErrorPath HOT 5
- bug: no availability zones available in region eu-central-2 HOT 1
- enhancement request: FIPS Configuration Support HOT 1
- kms generate-data-key appears to ignore keyspec HOT 1
- bug: AWS Java SDK 2 returns a 403 error HOT 3
- bug: mature and well-tested template.yml has multiple errors HOT 2
- bug: <title> CreateRepository operation: API for service 'ecr' not yet implemented HOT 2
- Error while creating AWS::SNS:Subscription through Cloudformation HOT 3
- Support Textract service HOT 1
- bug: localstack ignores docker volume mount in docker-compose file when running in Github actions HOT 4
- bug: EventBridge does not trigger target lambdas with the `event_pattern` pattern containing filtering by `detail` HOT 1
- bug: When using localstack with serverless always got Invalid token HOT 1
- bug: S3 putObject fails when invoked more than 1000 times in a row HOT 7
- bug: Cognito `TRIGGER_CONFIRM_SIGNUP`, json.decoder.JSONDecodeError: Expecting value: line 1 column 1
- bug: Unable to update stack "RemovePermission operation: No policy is associated with the given resource." HOT 8
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from localstack.