retry-async

retry-async.ts

/**
 * Retry-status object type, for use with RetryCB.
 */
export type RetryStatus = {
    /**
     * Retry index, starting from 0.
     */
    index: number,
    /**
     * Retry overall duration, in milliseconds.
     */
    duration: number,
    /**
     * Last error, if available;
     * it is undefined only when "retryAsync" calls "func" with index = 0.
     */
    error?: any
};

/**
 * Retry-status callback type.
 */
export type RetryCB<T> = (s: RetryStatus) => T;

/**
 * Type for options passed into retryAsync function.
 */
export type RetryOptions = {
    /**
     * Maximum number of retries (infinite by default),
     * or a callback to indicate the need for another retry.
     */
    retry?: number | RetryCB<boolean>,
    /**
     * Retry delays, in milliseconds (no delay by default),
     * or a callback that returns the delays.
     */
    delay?: number | RetryCB<number>,
    /**
     * Error notifications.
     */
    error?: RetryCB<void>
};

/**
 * Retries async operation returned from "func" callback, according to "options".
 */
export function retryAsync<T>(func: RetryCB<Promise<T>>, options?: RetryOptions): Promise<T> {
    const start = Date.now();
    let index = 0, e: any;
    let {retry = Number.POSITIVE_INFINITY, delay = -1, error} = options ?? {};
    const s = () => ({index, duration: Date.now() - start, error: e});
    const c = (): Promise<T> => func(s()).catch(err => {
        e = err;
        typeof error === 'function' && error(s());
        if ((typeof retry === 'function' ? (retry(s()) ? 1 : 0) : retry--) <= 0) {
            return Promise.reject(e);
        }
        const d = typeof delay === 'function' ? delay(s()) : delay;
        index++;
        return d >= 0 ? (new Promise(a => setTimeout(a, d))).then(c) : c();
    });
    return c();
}

Cool

Tips

• Use of delay as a callback allows for any delay strategy at any point.
• To deactivate delay, it needs to be negative, as setTimeout for 0 still creates a delay.
• If you need a timeout for your retries, use retry as a callback, and check for duration it receives.

Example

Below is a comprehensive example, which uses callbacks for all retry options (copied from repo):

import {retryAsync, RetryStatus} from './retry-async';

let failCount = 4; // let's fail our requests 4 times

// our async function, result from which we intend to re-try, when it fails:
async function makeRequest(s: RetryStatus) {
    if (--failCount >= 0) {
        throw new Error(`Request failed ${s.index + 1} time(s)`);
    }
    return `Request succeeded after ${s.duration}ms and ${s.index + 1} attempt(s)`;
}

// use delays with 0.5s increments:
const delay = (s: RetryStatus) => (s.index + 1) * 500;

// retry for up to 5 times, with duration not exceeding 4s:
const retry = (s: RetryStatus) => s.index < 5 && s.duration <= 4000;

const error = (s: RetryStatus) => {
    const info = {index: s.index, duration: s.duration, error: s.error.message};
    console.error('Handling:', info);
}

(function test() {
    retryAsync(makeRequest, {retry, delay, error})
        .then(data => console.log('SUCCESS:', data))
        .catch(err => console.error('FAILED:', err));
})();

Output:

Handling: { index: 0, duration: 0, error: 'Request failed 1 time(s)' }
Handling: { index: 1, duration: 514, error: 'Request failed 2 time(s)' }
Handling: { index: 2, duration: 1525, error: 'Request failed 3 time(s)' }
Handling: { index: 3, duration: 3033, error: 'Request failed 4 time(s)' }
SUCCESS: Request succeeded after 5034ms and 5 attempt(s)

☝️ In case you are wondering, how come we get so many extra ms (14->11->8) for something so well-optimized. It is because setTimeout is quite inaccurate, for one reason, and for another, console output is heavy in NodeJS. If we switch off console output inside error, and pass in delay: -1 (or none at all), then we will see a consistent output like this:

SUCCESS: Request succeeded after 0ms and 5 attempt(s)

---

@steinathan And it only got cooler since 😄 Thanks for the first post 😸

Afterthoughts

I noticed how many libraries out there bunch up various delay/timeout strategies right into their libraries for async retry, which never made any sense to me, because a good solution should just provide a way to set any such strategy dynamically, based on such thing as RetryStatus above, and not incorporate some specific solutions, which needlessly complicate and limit their own.

You can see how easy it is, from the example above, to set a delay+retry strategy that we use:

// use delays with 0.5s increments:
const delay = (s: RetryStatus) => (s.index + 1) * 500;

// retry for up to 5 times, with duration not exceeding 4s:
const retry = (s: RetryStatus) => s.index < 5 && s.duration <= 4000;

But you can easily build any strategy on this, including exponential or random progressions, if you like 😄

If we change the delay for the example above, to use a simple exponential progression:

// use delays of 7ms, with exponential increase:
const delay = (s: RetryStatus) => 7 ** (s.index + 1); // Delays: 7, 49, 343 and 2401 ms

Then the output will change to this:

Handling: { index: 0, duration: 0, error: 'Request failed 1 time(s)' }
Handling: { index: 1, duration: 28, error: 'Request failed 2 time(s)' }
Handling: { index: 2, duration: 90, error: 'Request failed 3 time(s)' }
Handling: { index: 3, duration: 445, error: 'Request failed 4 time(s)' }
SUCCESS: Request succeeded after 2862ms

That's 2800ms projected vs 2862ms actual, so extra 62ms due to timeout inaccuracy + console logging.

---

A good usage pattern within any app would include a helper like one below, to centralize various resilience strategies:

export class AsyncResilience {

    // aggressive async retry strategy:
    // retries 10 times with 1s delays
    static get aggressive(): RetryOptions {
        return {
            retry: 10,
            delay: 1000
        }
    }

    // endless retry strategy, with delays as follows:
    // 500, 1000, 1500, 2000, 2500, (3000)
    static get endless(): RetryOptions {
        return {
            delay(s: RetryStatus) {
                return s.index > 5 ? 3000 : (s.index + 1) * 500;
            }
        }
    };
}

So when you can use AsyncResilience.aggressive, for example, whenever a retry strategy is needed, so all your resilience strategies for async retries are in one place.

Further on, you may add strategies, for example, specific to HTTP requests, so the retry logic depends on HTTP error codes that you are getting, etc.

I have created a retry-async repo for this, because this gist environment seems too simplistic. So, if you want to follow up on this gist, you should do it there instead ;)

I have created a retry-async repo for this, because this gist environment seems too simplistic. So, if you want to follow up on this gist, you should do it there instead ;)

how does it handle functions that require parameters?

how does it handle functions that require parameters?

@infinity-matrix You are always expected to use a local wrap for the original function logic:

async function createOriginalAsyncRequest(a1: any, a2: any, a3: any) {

    // wrap request logic into a local function:
    const createAsyncRequest = async (s: RetryStatus) => {
        // all request logic here, using a1, a2, a3
    }

    return retryAsync(createAsyncRequest, {delay: 100, retry: 5});
}

...with the exception of when the original function takes no arguments, then you can use it directly, if you want, though a wrap always provides a cleaner approach, because then you can just call the original function name, while hiding the retry logic inside it.

Thanks for the quick response - can you please also point out how to get the return value from the original function - thanks!

Thanks for the quick response - can you please also point out how to get the return value from the original function - thanks!

You are already getting it above. What's the issue there?

In case anyone else comes across this and also wants to print details of each retry, the return line of the retryAsycnc function (line 59) can be changed to the following:

// return d >= 0 ? (new Promise(a => setTimeout(a, d))).then(t) : t();
const mypromise = (a => {console.log('### retryAsync  func: ' + func.name + ' # retry: ' + r + ' # delay: ' + d + ' # index: ' + index); setTimeout(a, d)});
return d >= 0 ? new Promise(mypromise).then(t) : t();

how does it handle functions that require parameters?

@infinity-matrix You are always expected to use a local wrap for the original function logic:

async function createOriginalAsyncRequest(a1: any, a2: any, a3: any) {

    // wrap request logic into a local function:
    const createAsyncRequest = async (s: RetryStatus) => {
        // all request logic here, using a1, a2, a3
    }

    return retryAsync(createAsyncRequest, {delay: 100, retry: 5});
}

...with the exception of when the original function takes no arguments, then you can use it directly, if you want, though a wrap always provides a cleaner approach, because then you can just call the original function name, while hiding the retry logic inside it.

Note that this works fine even when the wrapper function is not async.

Also note that if the original function does not throw any exception but returns null or blank string or some specific value in case it cannot get the desired result, then you need to add a check for that condition inside the wrapper method after the call to original function and then throw an error in case the value is not valid - else this won't work. Example below shows how this could be done:

export function createOriginalAsyncRequest(a1: any, a2: any, a3: any) {

    // wrap request logic into a local function:
    const createAsyncRequest = async (s: RetryStatus) => {
        // all request logic here, using a1, a2, a3
      let tmp: string = selectEnvConfig(a1, a2, a3);
      if (isStrEmpty(tmp)) {
        throw new Error(`##### Request Failed Try: ${s.index + 1})`);
      }
      return tmp;
    }

    return retryAsync(createAsyncRequest, {delay: 100, retry: 5});
}

In case anyone else comes across this and also wants to print details of each retry, the return line of the retryAsycnc function (line 59) can be changed to the following:

Why would you want to change it that way when you already have error handler that provides the details?

Note that this works fine even when the wrapper function is not async.

No, it doesn't, retryAsync requires that the wrapper always returns a promise.

Also note that if the original function does not throw any exception but returns null or blank string or some specific

That is not applicable here! The original function is always async, this is what it is all about, retrying an async function, so it always returns a Promise. No need throwing an exception, though you can, if you want.

In all, I sense a lot of confusion around how retryAsync works, even though the source code is all up in front. Somehow you managed to make all the wrong assumptions here :) Perhaps you haven't worked with promises and async code in JavaScript enough.

@infinity-matrix if you have questions, open a discission in the repo, it will be easier there, as gists are too limited.