jed / dynamo-client Goto Github PK

Hi,

I an facing error response from dynamo-client while using query. The error response comes very often. But, sometimes I get a proper response also. So, its not a data issue. I get error in dynamo-client, when it parses the json response.

[2017-05-11 21:10:18:701] SyntaxError: Unexpected token S
at Object.parse (native)
at IncomingMessage. (/home/chenoa/Desktop/02-05-2017/v4api_07_20_16/v4api/node_modules/dynamo-client/dynamo_client.js:92:29)
at emitNone (events.js:72:20)
at IncomingMessage.emit (events.js:166:7)
at endReadableNT (_stream_readable.js:905:12)
at doNTCallback2 (node.js:441:9)
at process._tickDomainCallback (node.js:396:17)
[2017-05-11 21:10:18:702] Error while using query: SyntaxError: Unexpected token S
[2017-05-11 21:10:18:728] Error: HTTP/1.1 200

Is it that, response is yet not completed and the control moved to res.end method or is it something to do with concurrency.
The issue des not happen always but, it does happen very frequently.

Thanks.

I see an interesting phenomenon if I change the tests to use the ap-southeast-2 region, which is the closest region to me at the moment, whereby the concurrency/ProvisionedThroughputExceededException test takes around 50 seconds to complete - this is compared with around 10 secs or so for us-east-1, which I have a much higher latency to.

This is counter intuitive at first glance, but I think what is happening (yet to confirm) is that because the latency is so much lower for me to ap-southeast-2, the 50 parallel requests can be made a lot faster, and therefore they trigger the ProvisionedThroughputExceededException much sooner and end up in a cycle of contention as all 50 requests slowly back off.

What I'm thinking could be a good idea to attempt to reduce this contention would be to have a per-table queue or limit on how many parallel requests can be made (similar to https://github.com/caolan/async#queue) - perhaps this queue only starts getting used once ProvisionedThroughputExceededExceptions start getting thrown.

Thoughts?

Hi Jed

I'm a bit confused. By looking at the package.json dependencies and reading the README this seems to be a driver implementation from scratch, as opposed to a thin wrapper over Amazon's SDK.

Yet, you deprecated your https://github.com/jed/dynamo project a couple of years ago and adviced people into using Amazon's SDK.

Is there something I'm missing?
Thanks