Bug in progress-listener? about amazonica HOT 6 CLOSED

I'm guessing it's a race condition. The start event is fired before the progress listener is actually registered. When you call upload the first time, TransferManager creates an UploadCallable, which is submitted to the TransferManager's thread pool in the constructor of UploadMonitor on the next line, which when called is going to fire the start event.

It's counterintuitive to me. I would think the the first upload would in fact see the started event fired, as the progress listener had time to be attached while the fixed thread pool was spinning up the 10 threads, and all subsequent calls to upload would "miss" the started event being fired. But threads seem to behave counterintuitively more often than not, ALIME.

from amazonica.

Thanks for the reply.

Based on your response, it seems like the appropriate thing to do would be to see if this behavior is the same when using pure java. Does that sound right to you? I started going down that road by using the AWS SDK directly, but then didn't want to rewrite all of the nice marshalling/unmarshalling and so on that you've added to amazonica.

My java is a bit weak, so I'd probably spend more time trying to test this than I'd be comfortable with, given the workaround of adding the inc to pending works fine when I do it in the body of the let, and not on the :started event.

I don't want to impose, but could you (or one of the lovely people following the issues list on amazonica 👋) help me verify this is the behavior in pure java as well? If I know that's the case, I could open a ticket with Amazon and see what they have to say about it. Before I do that I want to make sure this isn't a clojure-specific or amazonica-specific issue.

Thanks for your time,
Devin

from amazonica.

I can't reproduce with the code from your gist. pending is always zero for me, with large or small files. Do you get zero if you remove the :failed case?

from amazonica.

No, locally.

from amazonica.

@mcohen01 I wonder if this is only a race condition on EC2, where the upload times are significantly faster. I will test it from my local machine. By the time I go to add the progress listener, the upload may have already finished?

EDIT: Then again, it seems odd that it only happens for the first upload.

One other question: Did you test from a REPL, or by running lein run or something?

from amazonica.

I was guessing that the start event is fired before the listener is attached. Maybe making the connection to S3 happens much quicker when you're in EC2 such that this is the case, but not outside AWS. I've been testing in the repl.

from amazonica.