JS - Measuring Performance

gistfile1.txt

JS - Measuring JavaScript Functions’ Performance

Performance has always played a crucial part in software. On the web, performance is even more important as our users can easily change website and visit one of our competitors if we offer them slow pages. As professional web developers, we have to take this issue into account. A lot of old web performance optimization best practices, such as minimizing requests, using a CDN and not writing rendering blocking code, still apply today. However, as more and more web apps are using JavaScript, it’s important to verify that our code is fast.
Suppose that you have a working function but you suspect it’s not as fast as it could be, and you have a plan to improve it. How do you prove this assumption? What’s the best practice for testing the performance of JavaScript functions today? Generally, the best way to achieve this task is to use the built-in performance.now() function and measure the time before and after your function executes.
In this article we’ll discuss how to measure code execution time and techniques to avoid some common pitfalls.
Performance.now()
The High Resolution Time API offers a function, named now() that returns a DOMHighResTimeStamp object. It’s a floating point number that reflects the current time in milliseconds “accurate to a thousandth of a millisecond”. Individually, the number doesn’t add much value to your analysis, but a difference between two such numbers gives an accurate description of how much time has passed.
In addition to the fact that it is more accurate than the built-in Date object, it’s also “monotonic”. That means, in simple terms, that it’s not affected by the system (e.g. your laptop OS) periodically correcting the system time. In even simpler terms, defining two instances of Date and calculating the difference isn’t representative of the time that has passed.
The mathematical definition of “monotonic” is “(of a function or quantity) varying in such a way that it either never decreases or never increases”.
Another way of explaining it, is by trying to imagine using it around the times of the year when the clocks go forward or go back. For example, when the clocks in your country all agree to skip an hour for the sake of maximizing daytime sunshine. If you were to make a Date instance before clocks go back an hour, and another Date instance afterwards, looking at the difference it would say something like “1 hour and 3 seconds and 123 milliseconds”. With two instances of performance.now() the difference would be “3 seconds 123 milliseconds and 456789 thousands of a millisecond”.
In this section, I won’t cover this API in detail. So if you want to learn more about it and see some example of its use, I suggest you to read the article Discovering the High Resolution Time API.
Now that you know what the High Resolution Time API is and how to use it, let’s delve into some potential pitfalls. But before doing so, let’s define a function called makeHash() that we’ll use for the remainder of the article.
function makeHash(source) {
  var hash = 0;
  if (source.length === 0) return hash;
  for (var i = 0; i < source.length; i++) {
    var char = source.charCodeAt(i);
    hash = ((hash<<5)-hash)+char;
    hash = hash & hash; // Convert to 32bit integer
  }
  return hash;
}
The execution of such function can be measured as shown below:
var t0 = performance.now();
var result = makeHash('Peter');
var t1 = performance.now();
console.log('Took', (t1 - t0).toFixed(4), 'milliseconds to generate:', result);
If you run this code in a browser, you should see something like this:
Took 0.2730 milliseconds to generate: 77005292
live demo of this code is shown below:
* 		JS
* 		Result 
EDIT ON
function makeHash(source) {
  var hash = 0;
  if (source.length === 0) return hash;
  for (var i = 0; i < source.length; i++) {
    var char = source.charCodeAt(i);
    hash = ((hash<<5)-hash)+char;
    hash = hash & hash; // Convert to 32bit integer
  }
  return hash;
}

var t0 = performance.now();
var result = makeHash('Peter');
var t1 = performance.now();
console.log('Took', (t1 - t0).toFixed(4), 'milliseconds to generate:', result);
Resources 
EXTERNAL CSS
This Pen doesn't use any external CSS resources. 
EXTERNAL JAVASCRIPT
This Pen doesn't use any external JavaScript resources. 



With this example in mind, let’s start our discussion.
Pitfall #1 – Accidentally Measuring Unimportant Things
In the example above, you can note that the only thing that we do between one performance.now() and the other is calling the function makeHash() and assigning its value to a variable result. This gives us the time it takes to execute that function and nothing else. This measurement could also be made as detailed below:
var t0 = performance.now();
console.log(makeHash('Peter'));  // bad idea!
var t1 = performance.now();
console.log('Took', (t1 - t0).toFixed(4), 'milliseconds');
A live demo of this snippet is shown below:
* 		JS
* 		Result 
EDIT ON
function makeHash(source) {
  var hash = 0;
  if (source.length === 0) return hash;
  for (var i = 0; i < source.length; i++) {
    var char = source.charCodeAt(i);
    hash = ((hash<<5)-hash)+char;
    hash = hash & hash; // Convert to 32bit integer
  }
  return hash;
}

var t0 = performance.now();
console.log(makeHash('Peter'));  // bad idea!
var t1 = performance.now();
console.log('Took', (t1 - t0).toFixed(4), 'milliseconds');
Resources 
EXTERNAL CSS
This Pen doesn't use any external CSS resources. 
EXTERNAL JAVASCRIPT
This Pen doesn't use any external JavaScript resources. 

But in this case, we would be measuring the time it takes to call the function makeHash('Peter') and how long it takes to send and print that output on the console. We don’t know how long each of those two operations took. You only know the combined time. Also, the time it takes to send and print the output will vary greatly depending on the browser and even on what’s going on in it at that time.
Perhaps you’re perfectly aware that console.log is unpredictably slow. But it would be equally wrong to execute more than one function, even if each function does not involve any I/O. For example:
var t0 = performance.now();
var name = 'Peter';
var result = makeHash(name.toLowerCase()).toString();
var t1 = performance.now();
console.log('Took', (t1 - t0).toFixed(4), 'milliseconds to generate:', result);
Again, we won’t know how the execution time was distributed. Was it the variable assignment, the toLowerCase() call, or the toString() call?
Pitfall #2 – Measuring only Once
Another common mistake is to make just one measurement, summarize the time taken and draw conclusions based on that. It’s likely to be totally different at different times. The execution time greatly depends on various factors:
* 		Time for the compiler to warm up (e.g. time to compile the code into byte code)
* 		The main thread being busy doing other things we didn’t realize were going on
* 		Your computer’s CPU(s) being busy with something that slows down your whole browser
An incremental improvement is to execute the function repeatedly, like this:
var t0 = performance.now();
for (var i = 0; i < 10; i++) {
  makeHash('Peter');
}
var t1 = performance.now();
console.log('Took', ((t1 - t0) / 10).toFixed(4), 'milliseconds to generate');
A live demo of this example is shown below:
* 		JS
* 		Result 
EDIT ON
function makeHash(source) {
  var hash = 0;
  if (source.length === 0) return hash;
  for (var i = 0; i < source.length; i++) {
    var char = source.charCodeAt(i);
    hash = ((hash<<5)-hash)+char;
    hash = hash & hash; // Convert to 32bit integer
  }
  return hash;
}

for (var i=0; i < 10; i++) {
  var t0 = performance.now();
  var result = makeHash('Peter');
  var t1 = performance.now();
  console.log('Took', (t1 - t0).toFixed(4), 'milliseconds to generate:', result);
}
Resources 
EXTERNAL CSS
This Pen doesn't use any external CSS resources. 
EXTERNAL JAVASCRIPT
This Pen doesn't use any external JavaScript resources. 



The risk with this approach is that our browser’s JavaScript engine might make sub-optimizations which means that the second time the function is called with the same input, it can benefit from remembering the first output and simply use that again. To solve this issue, you can use many different input strings instead of repeatedly sending in the same input string (e.g. 'Peter'). Obviously, the problem with testing with different inputs is that naturally the function we’re measuring takes different amounts of time. Perhaps some of the inputs cause longer execution time than others.
Pitfall #3 – Relying too Much on the Average
In the last section we learned that it’s a good practice to run something repeatedly, ideally with different inputs. However, we have to remember that the problem with different inputs is that the execution might take much longer than all the other inputs. So let’s take a step back and send in the same input. Suppose that we send in the same input ten times and for each, print out how long that took. The output might look something like this:
Took 0.2730 milliseconds to generate: 77005292
Took 0.0234 milliseconds to generate: 77005292
Took 0.0200 milliseconds to generate: 77005292
Took 0.0281 milliseconds to generate: 77005292
Took 0.0162 milliseconds to generate: 77005292
Took 0.0245 milliseconds to generate: 77005292
Took 0.0677 milliseconds to generate: 77005292
Took 0.0289 milliseconds to generate: 77005292
Took 0.0240 milliseconds to generate: 77005292
Took 0.0311 milliseconds to generate: 77005292
Note how the very first time, the number is totally different from the other nine times. Most likely, that’s because the JavaScript engine in our browser makes some sub-optimizations and needs some warm-up. There’s little we can do to avoid that but there are some good remedies we can consider to prevent a faulty conclusion.
One way is to calculate the average of the last nine times. Another more practical way is to collect all results and calculate a median. Basically, it’s all the results lined up, sorted in order and picking the middle one. This is where performance.now() is so useful, because you get a number you can do whatever with.
Let’s try again but this time we’ll use a median function:
var numbers = [];
for (var i=0; i < 10; i++) {
  var t0 = performance.now();
  makeHash('Peter');
  var t1 = performance.now();
  numbers.push(t1 - t0);
}

function median(sequence) {
  sequence.sort();  // note that direction doesn't matter
  return sequence[Math.ceil(sequence.length / 2)];
}

console.log('Median time', median(numbers).toFixed(4), 'milliseconds');
Pitfall #4 – Comparing Functions in a Predictable Order
We’ve understood that it’s always a good idea to measure something many times and take the average. Moreover, the last example taught us that it’s preferable to use the median instead of the average.
Now, realistically, a good use of measuring function execution time is to learn which of several functions is faster. Suppose we have two functions that take the same type of input and yield the same result but internally they work differently.
Let’s say we want to have a function that returns true or false if a certain string is in an array of other strings, but does this case insensitively. In other words we can’t use Array.prototype.indexOf because it’s not case insensitive. Here’s one such implementation:
function isIn(haystack, needle) {
  var found = false;
  haystack.forEach(function(element) {
    if (element.toLowerCase() === needle.toLowerCase()) {
      found = true;
    }
  });
  return found;
}

console.log(isIn(['a','b','c'], 'B'));  // true
console.log(isIn(['a','b','c'], 'd'));  // false
Immediately we notice that this can be improved because the haystack.forEach loop always goes through all the elements even if we have an early match. Let’s try to write a better version using a good old for loop.
function isIn(haystack, needle) {
  for (var i = 0, len = haystack.length; i < len; i++) {
    if (haystack[i].toLowerCase() === needle.toLowerCase()) {
      return true;
    }
  }
  return false;
}

console.log(isIn(['a','b','c'], 'B'));  // true
console.log(isIn(['a','b','c'], 'd'));  // false
Now let’s see which one is the fastest. We do this by running each function 10 times and collecting all the measurements:
function isIn1(haystack, needle) {
  var found = false;
  haystack.forEach(function(element) {
    if (element.toLowerCase() === needle.toLowerCase()) {
      found = true;
    }
  });
  return found;
}

function isIn2(haystack, needle) {
  for (var i = 0, len = haystack.length; i < len; i++) {
    if (haystack[i].toLowerCase() === needle.toLowerCase()) {
      return true;
    }
  }
  return false;
}

console.log(isIn1(['a','b','c'], 'B'));  // true
console.log(isIn1(['a','b','c'], 'd'));  // false
console.log(isIn2(['a','b','c'], 'B'));  // true
console.log(isIn2(['a','b','c'], 'd'));  // false

function median(sequence) {
  sequence.sort();  // note that direction doesn't matter
  return sequence[Math.ceil(sequence.length / 2)];
}

function measureFunction(func) {
  var letters = 'a,b,c,d,e,f,g,h,i,j,k,l,m,n,o,p,q,r,s,t,u,v,w,x,y,z'.split(',');
  var numbers = [];
  for (var i = 0; i < letters.length; i++) {
    var t0 = performance.now();
    func(letters, letters[i]);
    var t1 = performance.now();
    numbers.push(t1 - t0);
  }
  console.log(func.name, 'took', median(numbers).toFixed(4));
}

measureFunction(isIn1);
measureFunction(isIn2);
We run that and get following output:
true
false
true
false
isIn1 took 0.0050
isIn2 took 0.0150
A live demo of this example is shown below:
* 		JS
* 		Result 
EDIT ON
function isIn1(haystack, needle) {
  var found = false;
  haystack.forEach(function(element) {
    if (element.toLowerCase() === needle.toLowerCase()) {
      found = true;
    }
  });
  return found;
}

function isIn2(haystack, needle) {
  for (var i = 0, len = haystack.length; i < len; i++) {
    if (haystack[i].toLowerCase() === needle.toLowerCase()) {
      return true;
    }
  }
  return false;
}


console.log(isIn1(['a','b','c'], 'B'));  // true
console.log(isIn1(['a','b','c'], 'd'));  // false
console.log(isIn2(['a','b','c'], 'B'));  // true
console.log(isIn2(['a','b','c'], 'd'));  // false

function median(sequence) {
  sequence.sort();  // note that direction doesn't matter
  return sequence[Math.ceil(sequence.length / 2)];
}

function measureFunction(f) {
  var letters = 'a,b,c,d,e,f,g,h,i,j,k,l,m,n,o,p,q,r,s,t,u,v,w,x,y,z'.split(',');
  var numbers = [];
  for (var i = 0; i < letters.length; i++) {
    var t0 = performance.now();
    f(letters, letters[i]);
    var t1 = performance.now();
    numbers.push(t1 - t0);
  }
  console.log(f.name, 'took', median(numbers).toFixed(4));
}
measureFunction(isIn1);
measureFunction(isIn2);
Resources 
EXTERNAL CSS
This Pen doesn't use any external CSS resources. 
EXTERNAL JAVASCRIPT
This Pen doesn't use any external JavaScript resources. 


What the heck has just happened? The first function was three times faster. That was not supposed to happen!
The explanation is simple but subtle. The first function which uses haystack.forEachbenefits from some low-level optimizations in the browser’s JavaScript engine that we don’t get when we use an array index technique. It proves our point: you never know until you measure it!
Conclusions
In our attempt to demonstrate how to use performance.now() to get an accurate execution time in JavaScript, we stumbled across a benchmarking scenario where our intuition turned out to be quite the opposite of what our empirical results conclude. The point is that, if you want to write faster web apps your JavaScript code needs to be optimized. Because computers are (almost) living breathing things, they are unpredictable and surprising. The most reliable way to know that our code improvements yield faster execution, is to measure and compare.
The other reason we never know which code is faster, if we have multiple ways of doing the same thing, is because context matters. In the previous section we perform a case insensitive string search looking for one string among 26 other strings. It’s likely that the conclusion would be totally different if we instead had to look for one string among 100,000 other strings.
The list above isn’t exhaustive as there are more pitfalls to be aware of. For example, measuring unrealistic scenarios or only measuring on one JavaScript engine. But the sure thing is that a great asset for JavaScript developers who want to write faster and better web apps is performance.now(). Last but not least, remember that measuring execution time only yields one dimension of “better code”. There’s also memory and code complexity considerations to bear in mind.
What about you? Have you ever used this function to test your code’s performance? If not, how do you proceed in this stage? Please share your thoughts in the comments below. Let’s start a discussion!

Meet the author
Peter Bengtsson 
Peter is a full-stack Web Developer at Mozilla where he works in the Web Engineering team. His passion is building fun web apps that are fast as greased lightning and has published open source software for over 15 years. He blogs and experiments on www.peterbe.com.