vercel / async-retry Goto Github PK
View Code? Open in Web Editor NEWRetrying made simple, easy and async
Home Page: https://npmjs.com/async-retry
License: MIT License
Retrying made simple, easy and async
Home Page: https://npmjs.com/async-retry
License: MIT License
I am currently trying to retry requests made to Notion. If I am rate limited, their API returns the number of seconds I should wait before retrying. I would like to use this value in onRetry
(or anywhere else, though here it makes sense) to dynamically alter how long to wait before the next retry. If I am not mistaken, this kind of behaviour is not supported by async-retry
at the moment.
Right now, code like this will actually throw the error 'RetryOperation timeout occurred' which originates in node-retry. It should throw the error I made ("HI" + Math.random()
), but doesn't because of a timeout error.
import * as retry from 'async-retry'
async function throwsErrors(){
await delay(1000)
throw new Error("HI" + Math.random())
}
retry(
async (bail, attempt) => {
await throwsErrors()
} , {
maxRetryTime: 100,
minTimeout: 2,
randomize: true,
retries: 100,
onRetry: console.log
}
).then(console.log)
async function delay(milliseconds: number) {
return new Promise(resolve => setTimeout(resolve, milliseconds))
}
I made an issue in node-retry that describes why this happens in detail, but the gist is that if you call retryOperation.retry(err)
and there's a timeout error, node-retry drops err
. I'm not sure if this is something that node-retry needs to support or if it's something that you can/should handle here.
The documentation currently says this:
onRetry
: an optional Function that is invoked after a new retry is performed. It's passed the Error that triggered it as a parameter.
But it should say this:
onRetry
: an optional Function that is invoked before a new retry is performed. It's passed the Error that triggered it as a parameter.
It's a small but significant difference.
I thought onRetry
was useless because the docs say it runs after the retry. Then I tested and found it actually runs before the retry, which is much more useful. The docs should reflect that.
When will a new version get published to npm?
Hello,
I have these warnings when the retry is trigerred :
(node:27189) UnhandledPromiseRejectionWarning: Unhandled promise rejection (rejection id: 44): TypeError: Cannot read property 'onRetry' of undefined
This is how I use retry
:
const _retryGeocode = (address) => (
retry((bail) => (
_geocode(address).catch(e =>
e === 'OVER_QUERY_LIMIT'
? Promise.reject(e)
: bail(e)
)
))
)
I'm on node v6.9.1
As I don't provide any option to the retry function, I don't understand why these warnings occur as the opt
is initialized as an empty object by default.
Hello
I am getting this error while using retry
(0 , async_retry_1.retry) is not a function
Here is my code:
await retry(
async () => {
await this.myMethod(arguments)
},
{ retries: 2 },
);
Could you please help?
I find myself doing this a lot when using async-retry:
await Promise.race(
retry(/* ... */),
new Promise((_, reject) => {
setTimeout(() => {
reject(new Error('FAILED to complete async retry in less than 1 minute'))
}, 60000)
}),
)
It would be awesome if there were a clean way to provide an option (retryTimeout
?) that allowed you to say ("time out after the retry went for this long, regardless of the attempt number").
I recently added async-retry
to the project I'm working on because I was getting a lot of ChunkLoadError Loading chunk 6 failed
in Sentry. This is how I currently use it:
import loadable from '@loadable/component'
import retry from 'async-retry'
const Dashboard = loadable(() =>
retry(() => import(/* webpackChunkName: "Dashboard" */ 'containers/pages/Dashboard'), {
factor: 1,
maxTimeout: 10000,
})
)
I'm now getting a new error on Sentry from async-retry
package. I'm not sure to understand it:
function runAttempt(num) {
32 var val;
33
34 try {
35 val = fn(bail, num);
36 } catch (err) {
37 onError(err, num);
38 return;
39 }
40
41 Promise.resolve(val)
42 .then(resolve)
43 .catch(function catchIt(err) {
44 onError(err, num);
45 });
46 }
According to Sentry the problem is on line 35 of this function. I get 3 different error messages:
TypeError error e is not a function
TypeError error Function expected
TypeError e is not a function. (In 'e(c,t)', 'e' is an instance of Promise)
Am I doing something wrong or is it a bug?
I would be happy to help either way.
It would be nice to allow the called function to override the amount of time to delay between invocations, for the next invocation only. The rationale for this request is that some REST APIs will return a retry-after
header for some calls indicating how long one should wait before retrying the request. There is currently no way to communicate that directive to this module.
I haven't followed the details of whether the type of @types/retry has changed, but I got an error that option has no "retries", and I couldn't use this package.
const retry = require('async-retry');
const fetch = require('node-fetch');
await retry(
async (bail) => {
// if anything throws, we retry
const res = await fetch('https://google.com');
if (403 === res.status) {
// don't retry upon 403
bail(new Error('Unauthorized'));
return;
}
const data = await res.text();
return data.substr(0, 500);
},
{
// Not Found
retries: 5,
}
);
it would be nice to change the signature to this onRetry: (err, i) => {}
Throwing an error in onRetry
seems like a weird behavior to me.
const retry = require("async-retry");
(async () => {
try {
await retry(
(bail, attempt) => {
throw new Error(`inside error at ${attempt}`);
},
{
onRetry: (e, attempt) => {
console.error(e.message);
throw new Error(`onRetry error at ${attempt}`);
},
},
);
} catch (e) {
console.error(e.message);
}
})();
Result:
inside error at 1
onRetry error at 1
inside error at 2
/home/xxxxx/playground.js:12
throw new Error(`onRetry error at ${attempt}`);
^
Error: onRetry error at 2
at Object.onRetry (/home/xxxxx/playground.js:12:17)
at onError (/home/xxxxx/node_modules/async-retry/lib/index.js:33:17)
at RetryOperation.runAttempt [as _fn] (/home/xxxxx/node_modules/async-retry/lib/index.js:43:9)
at Timeout._onTimeout (/home/xxxxx/node_modules/retry/lib/retry_operation.js:81:10)
at listOnTimeout (node:internal/timers:557:17)
at processTimers (node:internal/timers:500:7)
I expected that it caught once without retry like:
inside error at 1
onRetry error at 1
(finish)
otherwise, with retry like:
inside error at 1
inside error at 2
inside error at 3
inside error at 4
inside error at 5
inside error at 6
onRetry error at 6
(finish)
Is this a bug?
await retry(async bail => {
// if anything throws, we retry
const res = await fetch('https://google.com')
// IF THIS CODE THROWS THEN THE CODE UNDERNEATH WON'T EVER RUN RIGHT?
// SO WHY IS THERE LOGIC TO BAIL IN THERE, BECAUSE SURELY TO RUN THAT CODE
// THE PROMISE WOULD HAVE RESOLVED CORRECTLY AND THEREFORE NOT NEED
// TO BE RETRIED?
if (403 === res.status) {
// don't retry upon 403
bail(new Error('Unauthorized'))
return
}
const data = await res.text()
return data.substr(0, 500)
}, {
retries: 5
})
my question is in caps above π
I faced this today, with my repo that uses firebase-admin
.
Would like to hear your comments about it. Is the use of eslintConfig
in package.json
supposed to be dev/internal detail? Currently (async-retry
1.3.3), It's not.
And make it optional.
This is most likely because I am not familiar enough with promises... but, how can I pass a function in a variable and then retry it if it fails the first time?:
// works ok:
await retry(
async (bail) => {
const res = await fetch('https://google.com')
}
);
// does not work
const Fn = fetch('https://google.com')
await retry(
async (bail) => {
const res = await Fn; // if fails first time, it is not called again
}
);
Are you interested in types? We made some for using your library and I can share them. I can make a pull request, otherwise I can try and toss it up on DefinitelyTyped.
Thanks!
"dependencies": {
"@azure/event-hubs": "^5.6.0",
"@azure/eventgrid": "^4.5.0",
"@azure/eventhubs-checkpointstore-blob": "^1.0.1",
"@azure/storage-blob": "^12.8.0",
"@hapi/joi": "^17.1.1",
"@nestjs/common": "^8.2.6",
"@nestjs/core": "^8.2.6",
"@nestjs/mongoose": "^8.0.1",
"@nestjs/platform-express": "^8.2.6",
"@nestjs/swagger": "^5.2.0",
"@sendgrid/client": "^7.4.2",
"async-retry": "1.3.1",
"axios": "^0.21.1",
"azure-eventgrid": "^1.6.0",
"azure-iothub": "^1.14.1",
"azure-storage": "^2.10.4",
"class-transformer": "^0.5.1",
"class-validator": "^0.13.2",
"config": "^3.3.6",
"flat": "^5.0.2",
"fs-extra": "^10.0.0",
"hot-shots": "^8.3.0",
"mongoose": "^5.13.9",
"ms-rest-azure": "^3.0.0",
"nanoid": "^3.3.1",
"nestjs-pino": "^2.5.0",
"pino-noir": "^2.2.1",
"qs": "^6.10.1",
"raw-body": "^2.4.1",
"reflect-metadata": "^0.1.13",
"rimraf": "^3.0.2",
"rxjs": "^7.5.4",
"swagger-ui-express": "^4.1.6"
},
"devDependencies": {
"@nestjs/testing": "^8.1.2",
"@types/express": "^4.17.13",
"@types/flat": "5.0.2",
"@types/fs-extra": "^9.0.13",
"@types/hapi__joi": "^17.1.4",
"@types/jest": "^27.0.2",
"@types/multer": "^1.4.2",
"@types/nanoid": "^2.1.0",
"@types/node": "^14.17.20",
"@types/pino": "^6.3.11",
"@types/supertest": "^2.0.10",
"@typescript-eslint/eslint-plugin": "^3.7.1",
"@typescript-eslint/parser": "^3.7.1",
"eslint": "^7.8.1",
"eslint-plugin-import": "^2.24.2",
"eslint-plugin-prefer-arrow": "^1.2.2",
"jest": "^27.2.4",
"prettier": "^1.15.3",
"supertest": "^4.0.2",
"ts-jest": "^27.0.5",
"ts-node": "9.0.0",
"tsc-watch": "^4.2.9",
"tsconfig-paths": "3.9.0",
"typescript": "^4.0.2",
"wait-on": "^5.3.0"
},
"jest": {
"moduleFileExtensions": [
"js",
"json",
"ts"
],
"rootDir": "src",
"testRegex": ".spec.ts$",
"transform": {
"^.+\.(t|j)s$": "ts-jest"
},
"coveragePathIgnorePatterns": [
"/src/core/validators"
],
"coverageDirectory": "../coverage",
"testEnvironment": "node"
}
please refer to my above configurations. I am facing an issue while testing the async-retry library as below.
const flushPromises = (): unknown => new Promise(resolve => setImmediate(resolve));
it('should retry method execution till success', async () => {
const verifyMethod = jest.fn().mockResolvedValue('verified');
const testObject = {
testKey: 'testValue',
methodToExecute : jest.fn()
.mockRejectedValueOnce(new errors.ThrottlingError('ThrottlingError'))
.mockRejectedValueOnce(new errors.ThrottlingError('ThrottlingError'))
.mockImplementationOnce(function(input) {
// eslint-disable-next-line no-invalid-this
return verifyMethod(input, this.testKey);
})
};
const promise = throttlingErrorHandler.handleOnError(
testObject.methodToExecute.bind(testObject),
[ 'testInput' ],
{ iothub: 'iothub1', deviceId: 'testdevice', operation: 'verifyMethod' }
);
await flushPromises();
jest.runAllTimers();
await flushPromises();
jest.runAllTimers();
await flushPromises();
const result = await promise;
expect(result).toEqual('verified');
expect(verifyMethod).toHaveBeenCalledWith('testInput', testObject.testKey);
expect(testObject.methodToExecute).toHaveBeenCalledTimes(3);
expect(mockMailService.sendMail).not.toHaveBeenCalled();
});
while executing this getting jest timeout exceed error.
below are my code snipped.
public async handleOnError(
asyncMethodToExecute: (...args) => Promise,
methodArguments: unknown[],
context: { iothub: string; deviceId: string; operation: string },
runForever = false
): Promise {
this.appLoggerService.warn(`Throttling/IotHubQuotaExceededError on ${context.iothub}`, ThrottlingErrorHandler.name);
const throttlingErrorConfiguration = await this.throttlingErrorConfigurationDbService.getConfiguration();
if (!throttlingErrorConfiguration) {
this.appLoggerService.warn('Throttling error configuration not available', ThrottlingErrorHandler.name);
return null;
}
const isMaxRetriesEnabled = throttlingErrorConfiguration.maxRetriesEnabled;
const maxRetries = isMaxRetriesEnabled ? throttlingErrorConfiguration.maxRetries : -1;
// fixed interval in seconds after which operation is tried again
const maxJitterIntervalInSeconds = throttlingErrorConfiguration.maxJitterIntervalInSeconds;
const sendMailAfterNumberOfAttempts = throttlingErrorConfiguration.sendMailAfterNumberOfAttempts;
const milliseconds = 1000;
return await retry(async (bail, attempt) => {
try {
const result = await asyncMethodToExecute(...methodArguments); // method context (this) should already be set by caller
return result;
} catch (error) {
if (error instanceof errors.ThrottlingError
|| error instanceof errors.IotHubQuotaExceededError) {
this.appLoggerService.warn(`Error: ${error.name} on ${context.iothub}`, ThrottlingErrorHandler.name);
if (attempt % sendMailAfterNumberOfAttempts === 0 && throttlingErrorConfiguration.sendGrid.isEnabled) {
this.appLoggerService.error(`Error: ${error.name} on ${context.iothub}, sending mail`, ThrottlingErrorHandler.name);
void this.mailService.sendMail(
throttlingErrorConfiguration.sendGrid.mailTo,
throttlingErrorConfiguration.sendGrid.mailCc,
throttlingErrorConfiguration.sendGrid.templateId,
{
iothub: context.iothub,
deviceId: context.deviceId,
operation: context.operation,
error: error.name
}
);
}
throw error;
} else {
bail(error);
}
}
},
{
retries: maxRetries > 0 && !runForever ? maxRetries - 1 : undefined, // subtract 1 as initial attempt itself is a retry
forever: !isMaxRetriesEnabled || runForever,
maxTimeout: maxJitterIntervalInSeconds * milliseconds,
randomize: true
});
}
the above retry function doesn't retry when an error is thrown.
kindly review and suggest what is wrong. it was working with jest 25.* version but not working with 27.* version.
my node version is 16.11.0 & npm version is 8.0.0
From the example:
// Packages
const retry = require('async-retry')
const fetch = require('node-fetch')
await retry(async bail => {
// if anything throws, we retry
const res = await fetch('https://google.com')
if (403 === res.status) {
// don't retry upon 403
bail(new Error('Unauthorized'))
// return <---- don't immediately return here
throw new Error('Throw after bail');
}
const data = await res.text()
return data.substr(0, 500)
}, {
retries: 5
})
Calling bail will immediately reject the promise returned by retry
, but if the promise returned by the retirer function is then itself rejected, attempts will continue running in the background until e.g. the max retry count is reached.
This can be surprising and result in obscure bugs.
is there a way to check a thrown error
before deciding to retry? I tried wrapping my await
line in a try/catch
& bail()
/throw
depending on error.message
, but that's making my unit tests go nuts for reasons that aren't clear to me
In example,
// ...
return await retry(async (bail) => {
// ...
if (403 === res.status) {
// don't retry upon 403
bail(new Error('Unauthorized'));
}
// ...
}
// ...
Shouldn't that be return bail
? Otherwise, it passes through, and attempts to also return the text substring, which wastes resources.
When parsing the options I wandered why they differ from the ones seen in vercel retry.
Then I checked that this library uses retry instead of vercel retry. Any reason why?
adding async-retry into our package dependencies increases bundle size by 10kb.Any suggestions to reduce this size?
the way i am importing is import retry from 'async-retry'; the version of the package i am using is "async-retry": "1.3.1". Please let me know.
I know there's feature to bail out conditionally, but I think it would be useful to have a function that is called as soon as the operation fails, and the return value of that function should decide whether or not the next retry should happen.
So if (!op.retry(err)) {
would just become something like if (!options.shouldRetry(err) || !op.retry(err)) {
.
Current alternative to this would be to have a try/catch in the retrier
function, decide in catch, throw if need to retry, and swallow and bail
if needed. But this is a lot of boilerplate and can not be extracted into a separate function.
While having a function in option would be much easier.
Did it make sense?
Hi
i want to retry the http request if the response data i am receiving is empty. The backend service takes some time to process the data, it return me an empty array if it has not processed yet. The array will hold data if it is successfully processed.
public async processClassification(instanceId: any, requestId: any): Promise<any> {
const url = this.config.backendUrl + "/check/classification";
const options = {
uri: url,
headers: {
"X-IDCHECK-SESSION_ID": instanceId,
},
body: {},
json: true,
resolveWithFullResponse: true,
};
const data = await retry(async (bail) => {
const res = await request.get(options);
if (res.body.data.classification.length > 0) {
bail("got data");
return;
}
return res;
}, {
retries: 5,
});
return data;
}
Also to note i using the following library https://github.com/request/request-promise-native to make the actual request
I have this situation where I download the file, and want to delete this file (async operation) if the downloaded file is corrupted and needs retry.
return retry(
async (_, attempt) => {
await downloadFileFromInternet();
},
{
onRetry: err => {
// It happens that downloaded file has an error
// I want to delete the downloaded corrupted file
await fs.unlink(pathToCorruptedFile);
},
retries: 3
}
);
Is it possible to have async-retry retry the next attempt only once the onRetry resolves (if the return value is a Promise)?
A declarative, efficient, and flexible JavaScript library for building user interfaces.
π Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. πππ
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google β€οΈ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.