Node.js design pattern: Asynchronous serial iteration over a collection using callbacks

README.md

This is an example of a very common Node.js design pattern: a serial asynchronous iteration over a collection (array) using callbacks.

If you are interested in learning more about common (and even less common!) Node.js design patterns, you should check out the book Node.js Design Patterns by Mario Casciaro and Luciano Mammino (spoiler alert: that's me :P):

https://www.nodejsdesignpatterns.com/

---

To run the example:

• Download the two files (test.js and iterateSeries.js) into a folder
• From your command line run node test

iterateSeries.js

/**
 * A callback based function that allows to iterate over a collection
 * and perform asynchronous actions on every element. The processing is
 * done is series, so an element is started only when the previous one
 * has been completed. In case of error the whole processing is interrupted
 * early and the error is propagated to the finalCallback to let the caller
 * decide how to handle it.
 *
 *
 * @type function
 * @param {array} collection - The generic array of elements to iterate over
 * @param {function} iteratorCallback - The callback based function that will
 *        be used to process the current element.
 *        It receives the current element as first paramenter and a callback
 *        as second parameter. The callback needs to be invoked to indicate
 *        the end of the asynchronous processing. It can be called passing an
 *        error as first parameter to propagate an error)
 * @param {function} finalCallback - A function that is called when all the
 *        items have been processed or when an error occurred. If an error
 *        occurred the function will be invoked with a single argument
 *        representing the error.
 */
const iterateSeries = (collection, iteratorCallback, finalCallback) => {
  const stoppingPoint = collection.length

  function iterate (index) {
    if (index === stoppingPoint) {
      return finalCallback()
    }
    const current = collection[index]
    iteratorCallback(current, err => {
      if (err) {
        return finalCallback(err)
      }
      return iterate(index + 1)
    })
  }

  iterate(0)
}

module.exports = iterateSeries

test.js

/**
 * In this example we have a collection of object.
 * Every object contains a file path and a content and.
 * We want to use the information in every object to actually
 * create the file in the file system with the given content.
 */

const fs = require('fs')
const iterateSeries = require('./iterateSeries')

const collection = [
  { filePath: 'file1.txt', content: 'hello from file 1' },
  { filePath: 'file2.txt', content: 'hello from file 2' },
  // uncomment the following line to see test the error propagation
  // { filePath: '/this/path/does/not/exist/file3.txt', content: '😱' },
  { filePath: 'file3.txt', content: 'hello from file 3' }
]

const createFileWithContent = (currentElement, cb) => {
  console.log(`Creating ${currentElement.filePath}`)
  fs.writeFile(currentElement.filePath, currentElement.content, 'utf8', cb)
}

iterateSeries(collection, createFileWithContent, err => {
  if (err) {
    console.error(err)
    process.exit(1)
  }

  console.log('All files written correctly')
})

improved with generic typescript support, auto detect callback (without callback support) https://www.webmanajemen.com/2023/05/asynchronous-serial-iteration-over-an-array-collection.html

Nice one. I am no TS expert but I'd be curious to know if you also tried to do that callback vs promise inference using infer...

In this realization if your collection doesn't contain anything (length = 0) program will create Zalgo problem. Because we expected that iterateSeries is an async function, but in case when collection length = 0 it behaves like a sync function. So I would add this statement

if (!collection.length) return process.nextTick(finalCallback)