Skip to content

Instantly share code, notes, and snippets.

@chikamichi
Last active February 11, 2020 04:36
Show Gist options
  • Save chikamichi/0c635ed175f8d7d2680b56d947d68fc9 to your computer and use it in GitHub Desktop.
Save chikamichi/0c635ed175f8d7d2680b56d947d68fc9 to your computer and use it in GitHub Desktop.
About Lodash, how it works (forEach example)

Hi!

About Lodash's forEach function, and Lodash in general…

I told you that it "abstracts away from you the chore (and complexity) of looping", so that you can focus on what really matters for your application: the collection you want to iterate through, and the piece of logic you wish to be applied for each item.

You use forEach like this:

// You first define "the collection you want to iterate through":
var myArray = ["lazy cat", "wild dog", "hidden Lorbo", "Alf", "POTUS"];

// You then define "the piece of logic you wish to be applied for each item":
var myFunction = function(element) {
  // do something with the element
};

// And finally, you leverage Lodash#forEach:
_(myArray).forEach(myFunction); // looping chore/complexity abstracted away!

myFunction above is just a definition, you are never calling it. At least not explicitly: you merely pass its reference (its name) to the forEach function which will take care, behind the scenes, of looping through the elements of myArray and of calling myFunction on each step of the looping, passing in the current item as element.

How Lodash makes it work

Now, how does forEach really work behind the scenes? If you go to https://lodash.com/docs#forEach you'll see three icons at the top: # S N. Clicking on S will show you the source code on GitHub (you could also find it in your local lodash.core.js file, but it's just easier clicking on a button ;)).

It looks like this:

function forEach(collection, iteratee) {
  var func = isArray(collection) ? arrayEach : baseEach;
  return func(collection, getIteratee(iteratee, 3));
}

A quick note: _(myArray).forEach(myFunction), the way we used it, internally gets transformed into _.forEach(myArray, myFunction), the way it's being defined above with two arguments. myArray is known as the "collection", and myFunction as the "iteratee", aka. "the piece of logic you wish to be applied for each item".

A little bit of context: forEach may process either arrays or hashes (objects), and lodash leverages a different function in each case (for efficiency reasons); therefore it first checks what type of collection it has been provided with. myArray is an array indeed, so isArray(myArray) returns true and forEach stores a reference to the convenient arrayEach function as func. At this point, the second line "looks" like this at runtime:

return arrayEach(myArray, getIteratee(myFunction, 3));

You would rather expect something simpler, like arrayEach(myArray, myFunction);, so what's getIteratee all about? Well, that one's a bit complex, so long story short: lodash wants people to have full control over that "iteratee", so it's somewhat customizable. getIteratee will take care of checking whether any customization has been applied globally, and also is able to fallback on a default iteratee shall no valid iteratee be provided. In our specific case, we're not doing anything fancy, so it eventually does call:

arrayEach(myArray, myFunction); // which is pretty close to the initial code, forEach(myArray, myFunction)

arrayEach? You know it already!

What's left for us is unveiling what exactly arrayEach does. It's defined at https://github.com/lodash/lodash/blob/4.12.0/lodash.js#L459-L478 (to quickly find it, I looked for "function arrayEach" in my browser, and not just "arrayEach"):

function arrayEach(array, iteratee) {
  var index = -1,
      length = array.length;

  while (++index < length) {
    if (iteratee(array[index], index, array) === false) {
      break;
    }
  }
  return array;
}

A quick glance at it, and you'll notice the while loop. They're using some (nice) tricks (for instance, combining the incrementation with the exit condition for instance (++index < length)), but that sets aside, it looks like a good old while loop! arrayEach takes care of calling myFunction (remember, it's known as iteratee at runtime).

Some may refer to iteratee/myFunction as a "callback". As the developer, you simply provided forEach with a reference to the function using _(myArray).forEach(myFunction), and internally, forEach/arrayEach is "calling myFunction back" for you when the time is right.

What's super interesting is the way they are calling the iteratee:

iteratee(array[index], index, array)

In our example, at runtime, that evaluates to myFunction(myArray[index], index, myArray). So myFunction is called with three arguments, on each step of the looping:

  • the first one is the item at position index within myArray
  • the second one is the index position itself, in case you need it
  • the third and last one is the entire array itself, in case you need it

By "in case you need it", I meant that you may define myFunction one of those three different ways:

var myFunction = function(item) {  }			           // #1
var myFunction = function(item, index) {  }		       // #2
var myFunction = function(item, index, collection) {  } // #3

We're using #1 right now, expecting a single argument that's the current item:

var myFunction = function(element) {
  // do something with the element
};

But depending on what "do something with the element" means, we might need more information that just the current element, so Lodash is nice and pushes that through arguments as well in case we need the intel.

Using forEach to build more complex stuff

Another interesting aspect of arrayEach is the fact that whatever myFunction returns gets compared to false:

if (iteratee(array[index], index, array) === false) {
  break;
}

It means that, if you implement myFunction in such a way that it returns false at some point (for a specific item), it will have the side effect of stopping the looping entirely (break;). It's handy when you are, say, looking for something in an array:

// The following function takes care of searching for a specific element
// within an array. It will return true as soon as it finds it, false otherwise.
function search(array, element) {
  var found = false; // The element we've not found just yet.

  _(array).forEach(function(item) {
    if (item == element) {
      // We found the element!
      // Let's acknowledge that, then break off the looping.
      found = true;
      return false;
    }
  });

  return found;
}

Using it is as simple as this:

var myArray = ["lazy cat", "wild dog", "hidden Lorbo", "Alf", "POTUS"];
search(myArray, "hidden Jeetbo"); // returns false
search(myArray, "hidden Lorbo"); // returns true

Excellent!

Lodash already has something like search: https://lodash.com/docs#includes

Conclusion

So you can see how you'll be able to leverage helpers such as forEach or includes: either directly, or within your own functions. My search example happens to be already covered by Lodash's includes, so you'd rather use that helper, but as a general-purpose library, Lodash can only do so much. It's then quite likely that at some point, you would either:

  • use Lodash's chaining feature to compose Lodash helpers: https://lodash.com/docs#chain
  • extend Lodash with your own little helpers: https://lodash.com/docs#mixin (it is a slightly more advanced use-case, so no need for you to spend time studying it, but just remember that Lodash provides you with a default set of helpers, that you may extend)

Let me know if some of that stuff troubles you :)

Building a complex strategy with lodash

New challenge: we'd like to craft a findAndReplaceAll function that would replace all occurences of an item within an array:

// Use-case:
var myArray = ["lazy cat", "wild dog", "hidden Lorbo", "Alf", "POTUS", "hidden Lorbo", "God"];
searchAndReplaceAll(myArray, "hidden Lorbo", "GOT YOU");
// => should edit and return the array as ["lazy cat", "wild dog", "GOT YOU", "Alf", "POTUS", "GOT YOU", "God"];

Finding and replacing all occurences sounds like a complex task, so we'd better break it down into simpler, more manageable steps:

  • finding: we already have a search function — but it stops at the first occurence
  • replacing: we don't have any logic about that just yet, but "replacing in an array" shouldn't be too hard, it's a matter of running array[key] = new_value

It seems like the core complexity here lies with the need for handling all occurences. So let's scale the whole thing down and focus on replacing the first occurence… first.

Replacing first

We'll define a findAndReplace (implied: first occurence, as with search) function that does just that:

function searchAndReplace(array, element, foundMsg) {
  var foundAt = search(array, element);
  if (foundAt > -1) {
    array[foundAt] = foundMsg;
  }
  return foundAt;
}

Simple enough: it tries to find an element within an array; if it does find one, the element gets replaced. The function then returns the position the found/replaced element is at, or -1 if no element was found.

This implementation needs search to return the position an element is found at (or -1 if there's no match). So far, our search function returns a boolean, but having it return an integer is pretty simple:

 function search(array, element) {
-   var found = false;
+   var foundAt = -1;
 
-  _(array).forEach(function(item) {
+  _(array).forEach(function(item, index) {
     if (item == element) {
-      found = true;
+      foundAt = index;
       return false;
     }
   });

-  return found;
+  return foundAt;
 }

You can try it out:

var myArray = ["lazy cat", "wild dog", "hidden Lorbo", "Alf", "POTUS", "hidden Lorbo", "God"];
searchAndReplace(myArray, "hidden Lorbo", "GOT YOU");
// => ["lazy cat", "wild dog", "GOT YOU", "Alf", "POTUS", "hidden Lorbo", "God"];

Replacing all

Switching from replacing one to replacing all essentially means we should keep on searching for the element even after we've found it once (or twice, or…), until we're sure there's no more room for it to hide within the array. A while loop is a perfect candidate here: "do something while you should".

More precisely, we want to keep on doing searchAndReplace until it reports -1 (ie. while it does not). We'll use a do… while structure because it better demonstrates that we want to attempt searching & replacing at least once:

function searchAndReplaceAll(array, element, foundMsg) {
  var foundAt = -1;
  do {
    foundAt = searchAndReplace(array, element, foundMsg);
  } while (foundAt > -1);
  return array;
}

var myArray = ["lazy cat", "wild dog", "hidden Lorbo", "Alf", "POTUS", "hidden Lorbo", "God"];
searchAndReplaceAll(myArray, "hidden Lorbo", "GOT YOU");
// ["lazy cat", "wild dog", "GOT YOU", "Alf", "POTUS", "GOT YOU", "God"];

Notes

While the implementation above works, it's highly inefficient. Even if the array contains only one occurence of the element, the do… while loop has it parsed until the element is found, and then parsed again a second time, until the element is not found — which means 'till the end. It only gets worse when there are multiple occurences.

The behaviour is good, but the implementation is bad. It calls for a refactoring!

Refactoring means modifying (improving!) the implementation while keeping the exact same behaviour. Tests come in handy in doing so: if properly written, they control the behaviour, not the innards.

We could have used a different strategy, that's going to fix the efficiency issue at its core by avoiding useless looping. Let's define a searchAll function that returns an array of positions (at which the element was found), and have replaceAll simply edit each reported position:

// Returns an array of integers. The array is empty if the element is not present.
function searchAll(array, element) {
  var positions = [];
  for (var i = 0; i < array.length; i++) {
    if (array[i] == element) {
      positions.push(i);
    }
  };
  return positions;
}

// Returns nothing, just gets the job done.
function replaceAll(array, positions, replaceWith) {
  for (position in positions) {
    array[position] = replaceWith;
  };
}

// Will return false if nothing got replaced, true otherwise.
function findAndReplaceAll(array, element, replaceWith) {
  var positions = searchAll(array, element);
  if (positions.length == 0) { return false; }
  replaceAll(array, positions, replaceWith);
  return true;
}

The overall code is simpler, easier to understand and to maintain. The core difference with the previous implementation is the fact that we've design a specialized function, searchAll, to pave the way for a clean and simple replacing strategy that involves no looping. The only place we're looping is within searchAll, and we're looping through the array only once.

Supporting arbitrary replacing strategies

Our boss just fired his last brillant idea: when a hidden Lorbo's been caught, he should receive an email.

Now, "replacing" means two things:

  1. array[key] = new_value — that's actual "replacing", right?
  2. sending an email to the boss – well, I guess that's important too, but rather specific to our use-case

Because the first task is "core" and the second is somewhat "custom", we'll keep them apart. At this stage, it's a good idea getting a clear understanding of what code exactly we wish to write — something like that:

searchAndReplaceAll(myArray, "hidden Lorbo", "GOT YOU", notifyBoss);
//                  \             core              /   \ custom /
//                   \_____________________________/     \______/

notifyBoss is the name of a function that will encapsulate the custom logic we wish to run when an element gets replaced within the array (in that case, when "hidden Lorbo" is found and replaced within myArray). To be able to pass along custom instructions like that, we need to change searchAndReplaceAll to accept a fourth argument, a callback:

-function searchAndReplaceAll(array, element, foundMsg) {
+function searchAndReplaceAll(array, element, foundMsg, callback) {
   var foundAt = -1;
   do {
-    foundAt = searchAndReplace(array, element, foundMsg);
+    foundAt = searchAndReplace(array, element, foundMsg, callback);
   } while (foundAt > -1);
   return array;
 }

The callback argument exist so the developer may pass any function name in. There's no structural change involved as far as searchAndReplaceAll is concerned: the callback is simply passed further down into searchAndReplace. That function thus needs to accept a callback as well — and more importantly, it will actually call it:

- function searchAndReplace(array, element, foundMsg) {
+ function searchAndReplace(array, element, foundMsg, callback) {
   var foundAt = search(array, element);
   if (foundAt > -1) {
     array[foundAt] = foundMsg;
+    callback(array, foundAt);
   }
   return foundAt;
 }

The callback is called in such a way that it gets fed with useful information, the array (the element was found within) and the position (the element was found at in the array). That is hard-coded convention, which means that although the callback may be any custom function, such a function has to somewhat abide by this signature (expecting two arguments, the first one being an array and the second one an integer, with the semantic described above).

Let's define a couple functions responsible for notifying the boss:

// Our very callback:
function notifyBoss(array, position) {
  sendEmail("[email protected]", "Ninja uncovered", "We got him, boss!\nHe was at " + position + ".\nCheers.");
}

// A standalone helper solely responsible for sending emails, clueless as to what the context might be.
function sendEmail(email, title, body) {
  // TODO: actually send the email…
}

And here it is, the following code just works:

searchAndReplaceAll(myArray, "hidden Lorbo", "GOT YOU", notifyBoss);

And so does this one:

searchAndReplaceAll(myArray, "hidden Lorbo", "SHHH", function() {
  sendEmail("[email protected]", "Save another ninja!", "A Lorbo hidden in plain sight I've just saved.\nTake great care of him.\nYours truly, L.");
});
/**
* Final shape of the code.
*
* Remember this is suboptimal implementation.
* Exercise: try to support callbacks using the other strategy (searchAll).
*/
function search(array, element) {
var foundAt = -1;
_(array).forEach(function(item, index) {
if (item == element) {
foundAt = index;
return false;
}
});
return foundAt;
}
function searchAndReplace(array, element, foundMsg, callback) {
var foundAt = search(array, element);
if (foundAt > -1) {
array[foundAt] = foundMsg;
callback(array, foundAt);
}
return foundAt;
}
function sendEmail(email, title, body) {
// TODO: actually send the email…
}
function notifyBoss(array, position) {
sendEmail("[email protected]", "Ninja uncovered", "We got him, boss!\nHe was at " + position + ".\nCheers.");
}
function searchAndReplaceAll(array, element, foundMsg, callback) {
var foundAt = -1;
do {
foundAt = searchAndReplace(array, element, foundMsg, callback);
} while (foundAt > -1);
return array;
}
var myArray = ["lazy cat", "wild dog", "hidden Lorbo", "Alf", "POTUS", "hidden Lorbo", "God"];
searchAndReplaceAll(myArray, "hidden Lorbo", "GOT YOU", notifyBoss);
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment