Comments (2)
deltarod
commented on June 27, 2024
I posted this in slack but putting here for extra context for anyone else:
2024.06.03 18:23:17.868481 [ 702 ] {} <Information> AWSClient: Failed to make request to: https://removed: Poco::Exception. Code: 1000, e.code() = 0, No message received, Stack trace (when copying this message, always include the lines below):
0. Poco::Net::HTTPResponse::read(std::basic_istream<char, std::char_traits<char>>&) @ 0x0000000014c8570b
1. Poco::Net::HTTPClientSession::receiveResponse(Poco::Net::HTTPResponse&) @ 0x0000000014c7bf08
2. DB::EndpointConnectionPool<Poco::Net::HTTPSClientSession>::PooledConnection::receiveResponse(Poco::Net::HTTPResponse&) @ 0x000000000cdc5249
3. DB::S3::PocoHTTPClient::makeRequestInternalImpl(Aws::Http::HttpRequest&, DB::ProxyConfiguration const&, std::shared_ptr<DB::S3::PocoHTTPResponse>&, Aws::Utils::RateLimits::RateLimiterInterface*, Aws::Utils::RateLimits::RateLimiterInterface*) const @ 0x000000000f997f64
4. DB::S3::PocoHTTPClient::MakeRequest(std::shared_ptr<Aws::Http::HttpRequest> const&, Aws::Utils::RateLimits::RateLimiterInterface*, Aws::Utils::RateLimits::RateLimiterInterface*) const @ 0x000000000f994132
5. std::shared_ptr<Aws::Http::HttpResponse> smithy::components::tracing::TracingUtils::MakeCallWithTiming<std::shared_ptr<Aws::Http::HttpResponse>>(std::function<std::shared_ptr<Aws::Http::HttpResponse> ()>, String const&, smithy::components::tracing::Meter const&, std::map<String, String, std::less<String>, std::allocator<std::pair<String const, String>>>&&, String const&) @ 0x0000000014dff26f
6. Aws::Client::AWSClient::AttemptExhaustively(Aws::Http::URI const&, Aws::AmazonWebServiceRequest const&, Aws::Http::HttpMethod, char const*, char const*, char const*) const @ 0x0000000014df1d37
7. Aws::Client::AWSXMLClient::MakeRequest(Aws::Http::URI const&, Aws::AmazonWebServiceRequest const&, Aws::Http::HttpMethod, char const*, char const*, char const*) const @ 0x0000000014e0f4f6
8. Aws::Utils::Outcome<Aws::S3::Model::ListObjectsV2Result, Aws::S3::S3Error> std::__function::__policy_invoker<Aws::Utils::Outcome<Aws::S3::Model::ListObjectsV2Result, Aws::S3::S3Error> ()>::__call_impl<std::__function::__default_alloc_func<Aws::S3::S3Client::ListObjectsV2(Aws::S3::Model::ListObjectsV2Request const&) const::$_0, Aws::Utils::Outcome<Aws::S3::Model::ListObjectsV2Result, Aws::S3::S3Error> ()>>(std::__function::__policy_storage const*) @ 0x0000000014fa2242
9. Aws::S3::S3Client::ListObjectsV2(Aws::S3::Model::ListObjectsV2Request const&) const @ 0x0000000014f11648
10. DB::S3::Client::ListObjectsV2(DB::S3::ExtendedRequest<Aws::S3::Model::ListObjectsV2Request>&) const @ 0x000000000f95d257
11. Aws::Utils::Outcome<Aws::S3::Model::ListObjectsV2Result, Aws::S3::S3Error> std::__function::__policy_invoker<Aws::Utils::Outcome<Aws::S3::Model::ListObjectsV2Result, Aws::S3::S3Error> ()>::__call_impl<std::__function::__default_alloc_func<DB::StorageS3Source::DisclosedGlobIterator::Impl::listObjectsAsync()::'lambda'(), Aws::Utils::Outcome<Aws::S3::Model::ListObjectsV2Result, Aws::S3::S3Error> ()>>(std::__function::__policy_storage const*) @ 0x00000000119af609
12. std::__packaged_task_func<std::function<std::future<Aws::Utils::Outcome<Aws::S3::Model::ListObjectsV2Result, Aws::S3::S3Error>> (std::function<Aws::Utils::Outcome<Aws::S3::Model::ListObjectsV2Result, Aws::S3::S3Error> ()>&&, Priority)> DB::threadPoolCallbackRunner<Aws::Utils::Outcome<Aws::S3::Model::ListObjectsV2Result, Aws::S3::S3Error>, std::function<Aws::Utils::Outcome<Aws::S3::Model::ListObjectsV2Result, Aws::S3::S3Error> ()>>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(std::function<Aws::Utils::Outcome<Aws::S3::Model::ListObjectsV2Result, Aws::S3::S3Error> ()>&&, Priority)::operator()(std::function<Aws::Utils::Outcome<Aws::S3::Model::ListObjectsV2Result, Aws::S3::S3Error> ()>&&, Priority)::'lambda'(), std::allocator<std::function<std::future<Aws::Utils::Outcome<Aws::S3::Model::ListObjectsV2Result, Aws::S3::S3Error>> (std::function<Aws::Utils::Outcome<Aws::S3::Model::ListObjectsV2Result, Aws::S3::S3Error> ()>&&, Priority)> DB::threadPoolCallbackRunner<Aws::Utils::Outcome<Aws::S3::Model::ListObjectsV2Result, Aws::S3::S3Error>, std::function<Aws::Utils::Outcome<Aws::S3::Model::ListObjectsV2Result, Aws::S3::S3Error> ()>>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(std::function<Aws::Utils::Outcome<Aws::S3::Model::ListObjectsV2Result, Aws::S3::S3Error> ()>&&, Priority)::operator()(std::function<Aws::Utils::Outcome<Aws::S3::Model::ListObjectsV2Result, Aws::S3::S3Error> ()>&&, Priority)::'lambda'()>, Aws::Utils::Outcome<Aws::S3::Model::ListObjectsV2Result, Aws::S3::S3Error> ()>::operator()() @ 0x00000000119b7367
13. void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<Aws::Utils::Outcome<Aws::S3::Model::ListObjectsV2Result, Aws::S3::S3Error>> (std::function<Aws::Utils::Outcome<Aws::S3::Model::ListObjectsV2Result, Aws::S3::S3Error> ()>&&, Priority)> DB::threadPoolCallbackRunner<Aws::Utils::Outcome<Aws::S3::Model::ListObjectsV2Result, Aws::S3::S3Error>, std::function<Aws::Utils::Outcome<Aws::S3::Model::ListObjectsV2Result, Aws::S3::S3Error> ()>>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(std::function<Aws::Utils::Outcome<Aws::S3::Model::ListObjectsV2Result, Aws::S3::S3Error> ()>&&, Priority)::operator()(std::function<Aws::Utils::Outcome<Aws::S3::Model::ListObjectsV2Result, Aws::S3::S3Error> ()>&&, Priority)::'lambda0'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000119b78ac
14. ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000000cc7b379
15. void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000000cc7ebda
16. void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000000cc7d9ed
17. ? @ 0x00007f7955922609
18. ? @ 0x00007f7955847353
(version 24.3.2.23 (official build))
2024.06.03 18:23:17.868546 [ 702 ] {} <Information> AWSClient: AWSXmlClient: HTTP response code: -1
Resolved remote host IP address: 16.182.97.209:443
Request ID:
Exception name:
Error message: Poco::Exception. Code: 1000, e.code() = 0, No message received (version 24.3.2.23 (official build))
0 response headers:
2024.06.03 18:23:17.868560 [ 702 ] {} <Information> AWSClient: If the signature check failed. This could be because of a time skew. Attempting to adjust the signer.
2024.06.03 18:23:17.868566 [ 702 ] {} <Information> AWSClient: Request failed, now waiting 0 ms before attempting again.
the flink loader I am using does create temp files. Maybe it doesn't properly handle file renames? That's the general form of my files so ordered mode probably wouldn't work unless I'm understanding the functionality incorrectly? the general multipart form of the file name is part-{shard_num}-{file_num} .
an example full s3 path for a single file would look like folder/table/2024-06-03--22/part-0-182 .
https://github.com/ClickHouse/ClickHouse/tree/master/src/Storages/S3Queue
Maybe I'm blind but I dont really see the logic of how it discovers new files here. Is that in keeper? A point in the right direction would be nice :)
from clickhouse.
deltarod
commented on June 27, 2024
#64349 fixes. Closing.
from clickhouse.
Related Issues (20)
- 23.11 segfault HOT 1
- FINAL on table with is_deleted works 10-30 times slower than expected HOT 3
- 24.4+ SSL config disableProtocols is overridden by requireTLSv1_2
- The data retrieved with the same query conditions fluctuates up and down, sometimes more and sometimes less! HOT 8
- [RFC] Checking backups checksums during RESTORE
- ALTER TABLE xxx MATERIALIZE COLUMNS/INDEXES/PROJECTIONS
- Low uniqMerge performance issue in Materialized aggregatingMergetree HOT 3
- file table function: an option to rename failed processing files
- Temporary stopping automatic dictionary updates HOT 3
- Direct dictionary + short_circuit_function_evaluation: Logical error / stack-buffer-overflow
- Logical error: Queries with distributed_group_by_no_merge=1 should be processed to Complete stage
- MergeTree: more efficient read operations for small data volumes HOT 4
- isIPv4String Function validating strings that are not proper IPv4 strings HOT 3
- error install clickhouse-server deb repo HOT 6
- LOGICAL_ERROR: 'Unexpected node type for table expression. Expected table, table function, query, union, join or array join. Actual IDENTIFIER'
- Code: 47. DB::Exception: Unknown expression or function identifier in scope WITH
- Clickhouse Server data is deleted when Kubernetes Cluster is restarted HOT 3
- Cannot cast to FixedString in a subquery HOT 1
- Replicated database: Stuck DDL after a failed DROP DATABASE HOT 2
- Unify calling of calling `client` or `local` with `clickhouse` and `ch` shortcuts
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from clickhouse.