Timeout issue for large upload about cmda HOT 6 CLOSED

I guess there's a chance that the config settings in checkBucketAccess are somehow messing with the global S3 config settings?

See if this works (it would actually be cleaner this way):

async function checkBucketAccess({ bucket = CMDA_BUCKET }) {
  if (!bucket) throw new Error('bucket cannot be empty')
  const s3 = new S3({ maxRetries: 0, httpOptions: { timeout: 2000, connectTimeout: 2000 } })
  await s3.headBucket({ Bucket: bucket }).promise()
}

from cmda.

Updating the lambda function code for S3 timeout to 5 minutes will fixed the issue
async function checkBucketAccess({ bucket = CMDA_BUCKET }) { if (!bucket) throw new Error('bucket cannot be empty') const s3 = new S3() s3.config.maxRetries = 0 const httpOptions = s3.config.httpOptions || {} httpOptions.timeout = 300000 httpOptions.connectTimeout = 300000 await s3.headBucket({ Bucket: bucket }).promise() }

from cmda.

But checkBucketAccess isn't involved in the upload – it's only there to ensure the lambda function has access to the bucket. You sure it's to do with the size of the file (50MB isn't very big)?

How slow is the internet connection you're on?

from cmda.

If the timeout on checkBucketAccess is changed to be 5 minutes, and lambda can't access to the bucket, then it will take 5 minutes before that feedback can be given to the user

from cmda.

I thought as well that the checkBucketAccess wasn't the cause, but after I update the lambda function with the 5 minutes, I was able to upload 512mb of data, where previously was failing consistently.

from cmda.

Fixed (hopefully) in v1.2.0

from cmda.