Secor failure upload policy about secor HOT 4 CLOSED

The upload should not be delayed indefinitely. In the worst case, there could be a delay of 48h. For details, see https://groups.google.com/forum/#!topic/secor-users/wxO0Eb83_R4

from secor.

I understand how secor uses the age of newest file in partition for the upload policy, but if I configure secor to use a DateMessageParser, when the new day comes(the timestamp of messages extract local machine time), a new file will be created in the same partition to store messages of the new date, will this behavior set file age zero everyday? Will it constrain secor upload policy?

from secor.

I think you are right. It is rather unconventional to set the upload delay to a value larger than the partition time interval but there is a corner case here that is not supported.

from secor.

Thanks for confirmation. I was thinking about some use cases that when the total message throughput is small, it is better to batch messages for longer time to create larger and lesser file on object store server, cause when the number of objects is large on S3 or other object store such as Swift, the performance may suffer.

from secor.