Comments (3)
egorksv
commented on August 27, 2024
More details after looking further into log files:
Pod start: 2024-02-29T18:27:06.547+04:00
First appearance of credentials refresh error: 2024-03-01T21:19:21.410+04:00
Log fragments when the error first appears:
Last successful AWS call querying DynamoDB table:
2024-03-01T21:17:18.365+04:00
Sync status: {'last_total_items': Decimal('677'), 'last_synced': '2024-03-01T17:15:18Z', 'batch_name': 'XXXXXXXX', 'last_page_size': Decimal('100'), 'last_total_pages': Decimal('7'), 'status': 'WAITNEWITEMS', 'last_page': Decimal('7')}
2024-03-01T21:19:21.410+04:00
Refreshing temporary credentials failed during advisory refresh period.
Traceback (most recent call last):
File "/app/.venv/lib/python3.11/site-packages/botocore/credentials.py", line 1964, in fetch_creds
response = self._fetcher.retrieve_full_uri(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/app/.venv/lib/python3.11/site-packages/botocore/utils.py", line 3072, in retrieve_full_uri
return self._retrieve_credentials(full_url, headers)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/app/.venv/lib/python3.11/site-packages/botocore/utils.py", line 3116, in _retrieve_credentials
return self._get_response(
^^^^^^^^^^^^^^^^^^^
File "/app/.venv/lib/python3.11/site-packages/botocore/utils.py", line 3138, in _get_response
raise MetadataRetrievalError(
botocore.exceptions.MetadataRetrievalError: Error retrieving metadata: Received non 200 response 400 from container metadata: [5de44b40-d111-48f7-9845-d1a50e8d8577]: (ExpiredTokenException): The token included in the request is expired: current date/time 2024-03-01T17:19:20.399912Z must be before the expiration date/time 2024-03-01T14:26:31Z., fault: client
Events (date/times converted to GMT):
Pod start: 2024-02-29T14:27:06.547Z
Token issued, initial expiry date/time: 2024-03-01T14:26:31Z # 24-hour token
Last successful call to dynamodb: 2024-03-01T17:17:18.365Z # Session still up?!
First attempt to refresh credentials during "advisory refresh period": 2024-03-01T17:19:21.410Z # 3 hours AFTER token was supposed to have expired?!
I strongly suspect there was a programming error somewhere around "advisory refresh period", as it was probably supposed to refresh 3 hours BEFORE expiration, but instead started refreshing 3 hour AFTER.
This does not look like time zone issue as I'm in GMT+4, three hours kind of "does not compute"
from botocore.
nateprewitt
commented on August 27, 2024
Hi @egorksv,
It looks like you might be encountering #3114 which was fixed a few weeks ago in 1.34.41. Have you tried updating to a more recent version of Boto3/Botocore?
from botocore.
egorksv
commented on August 27, 2024
Oh, thanks, I was on .33, will test on .56
from botocore.
Related Issues (20)
- Client.head_object raises ClientError instead of NoSuchKey HOT 2
- Add support for fast fail when token bucket empty
- The request signature we calculated does not match the signature you provided: Check your AWS Secret Access Key and signing method HOT 2
- RecursionError through botocore/endpoint_provider.py:572 HOT 3
- Support environment variable for S3 addressing_style config HOT 3
- Add GlobalSign Root R6 to cacert.pem HOT 2
- Waiter documentation about error behavior is incorrect HOT 2
- Retry on `IllegalLocationConstraintException` HOT 2
- Stubber.add_response for query method validate against an invalid schema HOT 3
- datetime.datetime.utcnow() is deprecated as of Python 3.12 HOT 2
- Memory leak tests use substantially more memory on Python 3.13 HOT 1
- Arguments of Bedrock's client.create_guardrail not working as expected HOT 2
- Race condition in botocore's use of JSONFileCache for credential loading HOT 7
- Difference in SigV4 calculation between S3 and botocore? HOT 2
- Using Generated Presigned URLs with CRC32C checksums results in 400 from S3 HOT 10
- Max attempts error in "certificate_issued" waiter HOT 3
- Detected blocking call inside the event loop HOT 1
- Boto3 Performance degradation after ugprade from python 3.8 to 3.12 HOT 4
- ParamValidationError for invalid bucket name when Config.parameter_validation is False HOT 3
- Ability to skip JSON parsing for DynamoDB get_item, query, and the like HOT 5
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from botocore.