The following lesson will introduce you to the basics of unit testing. More importantly, it will show you how to think like a tester.
We will be using Mocha and Chai. Mocha is the test runner; a framework for taking your test code and running the tests automatically. Chai is the assertion library; the syntax that you will use to write your tests.
Chai comes in three flavors: assert, should, and expect. For this lesson we will be using expect; an expressive and super readible style.
Mocha provides the describe block where we will write our tests.
describe('Example tests', function() {
// tests go here
});
Running this test file won't work though. Next we must add an it block.
describe('Example tests', function() {
it('should demonstrate a basic example', function() {
// test code goes here
});
});
We now have our first passing test...that makes no assertions and therefore will pass every time. This is where we mix in some Chai assertions:
const expect = require('chai').expect
describe('Example tests', function() {
it('should demonstrate a basic example', function() {
expect(2 + 1).to.equal(3)
});
});
When this test is run, it will produce passing results:
Example tests
✓ should demonstrate a basic example
This is still not a very useful test. Let's switch to a legitimate test. For the remainder of this lesson, we will be testing the ingredient parsing function for a hypothetical recipe parsing app.
There are countless recipes available online. If you have ever cooked or baked yourself, you have probably referenced one or more of these recipes. Recipes are relatively standard: description + ingredient list + steps to reproduce. Most humans can read these recipes with ease. However, for computers, there be recipe monsters lurking the internet waiting to destroy the good intentions of recipe parsers everywhere.
The problem: human input.
Humans provide a wide spectrum of acceptable written recipes for other humans. Microformats such as schema.org/recipe have been adopted by many websites in order to make recipes easier to parse and display. However, these microformats are at the level of description, ingredient list, and steps. Our recipe app needs to understand recipes at a deeper level. More on monsters later.
For our example tests, we are looking specifically at the parts of a recipe. Our model will evolve over the course of testing, but let's start with this simple breakdown:
[
{
raw: <string>,
quantity: <number>,
unit: <string>,
name: <string>,
}
]
Our ingredient parsing function (generically named parse() for ease of typing in this lesson), accepts a single ingredient string as input, and returns an array of ingredient objects. Why an array of objects? Don't we have just one recipe ingredient to parse at a time? We're preparing for recipe monsters; more on that later.
If our parse() is working as expected, this test should pass.
describe('Ingredient Parsing Tests', function(){
it('should parse basic ingredients', function(){
let input = '1 cup flour'
let expected = [{
raw: '1 cup flour',
quantity: 1,
unit: "cup",
name: "flour",
}]
expect(parse(input)).to.deep.equal(expected);
});
});
If you're curious as to why we are using deep.equal instead of just equal, you can read more about it in the Chai documentation. The simple explanation is that if you are comparing objects, you will almost always want to deep compare.
This is the only needed assertion for this test. We're comparing the expected result with the actual result returned from the parse() function. But since we're in lesson mode, let's add a few more.
First, we can ensure that parse() is returning an array.
expect(parse(input)).to.be.an('array')
In Chai assertion style, we can chain assertions together. Here, we are able to check that the result is an array and that it has only one index.
expect(parse(input)).to.be.an('array').and.to.have.lengthOf(1);
We can also check for specific keys in the ingredient object.
expect(parse(input)[0]).to.have.all.keys('raw', 'quantity', 'unit', 'name')
And to foreshadow how our model will need to expand in the future as we look and more and more complicated recipe ingredients, we can make certain that this ingredient string does not result in comments.
expect(parse(input)[0]).to.not.have.keys('comment')
We can check the value types of specific keys.
expect(parse(input)[0].quantity).to.be.a('number')
expect(parse(input)[0].unit).to.be.a('string')
expect(parse(input)[0].name).to.be.a('string')
We can go so far as to check that specific key/value pairs exist.
expect(parse(input)[0]).to.deep.include({unit: 'cup'});
Or, alternatively, simply check the value of a specific key.
expect(parse(input)[0].quantity).to.equal(1);
We can check that the array includes the expected object.
expect(parse(input)).to.deep.include(expected[0]);
This last assertion might be helpful in the future as our model evolves. We can check the order of the objects in the array.
expect(parse(input)).to.have.deep.ordered.members(expected)
But honestly, the only assertion we really need for this entire lesson, one that we can repeat over and over again, is this one:
expect(parse(input)).to.deep.equal(expected);
And this is the secret to testing. Most test assertions are simple. You can easily refer to the Chai documentation when you need a different kind of assertion; not something that needs to be hammered into your brain through lessons. Most of the complication of writing tests comes in the form of test setup and infrastructer; something that we are only going to hint at throughout this lesson.
But the number one winning strategy for test writing is to think like a tester. And to think like a tester means to live in the world of edge cases. It means being able to walk in hundreds or thousands of different users' shoes. It means undestanding that your relatively limited viewpoint, and the way that you approach life, may blind you to many of the myriad ways that users can and will interact with your apps. Thinking like a tester means thinking beyond yourself.
It also usually involves a desire to break things. The desire to watch an app crumble. Not out of the satisfaction of destruction; but in how, with the tester's mindset, it is possible to break things, and more importantly: fix things. All before a user has to experience it themselves. Testing is an extension of customer (user) service. The world will remain inevitably buggy; but testing can at least make a small dent in those bugs. If you care about your users, then please also care about your tests.
We should also be ready to handle unexpected input. Let's start with the simplest of all: an empty string.
it('should handle empty input', function(){
let input = ''
let expected = undefined
expect(parse(input)).to.be.undefined
});
We want our parsing function to return undefined if we receive input that we cannot parse.
it('should handle garbage input', function(){
let input = '35235lksjdf #@#$%#$%'
let expected = undefined
expect(parse(input)).to.be.undefined
});
We could assume that the parsing function will never be passed such garbage. We should have a function elsewhere in the app that is cleaning this up before passing it along to parse(), shouldn't we? Yes, but what if that function has its own bug? But if it does have a bug, shouldn't that be caught by that function's tests?! Yes, but what if it doesn't? Unit tests are cheap; as in, they run really fast. So a few extra safety net type tests provide extra insurance. Worst case they never fail. But you'll be very happy if it ever fails in the future.
Outside of the scope of most unit testing, at the integration level (testing multiple fuctions together) or the UI level (testing like a user), you may find benefit in at least manually testing more obscure garbage input such as...
it('should handle lengthy input', function(){
let input = [insert the entire text of War & Peace]
let expected = undefined
expect(parse(input)).to.be.undefined
});
We want to make sure that the tests only return singular forms of the units. That means teaspoon instead of teaspoons for consistencies sake.
it('should return singular units', function(){
let input = '20 cups sugar'
let expected = [{
raw: '20 cups flour',
quantity: 20,
unit: "cup",
name: "sugar",
}]
expect(parse(input)).to.deep.equal(expected);
});
What should we do about comments? In this context, comments are those little snippets of text used to further describe the state that an ingredient must be in at the beginning of the recipe. In many cases, these are mini pre-recipe steps. chopped, diced, and toasted are a few examples. However, comments could also refer to preferred brand names for ingredients or some other note from the recipe author.
it('should parse comments', function(){
let input = '5 cups flour, sifted'
let expected = [{
raw: '5 cups flour, sifted',
quantity: 5,
unit: "cup",
name: "flour",
comment: "sifted"
}]
expect(parse(input)).to.deep.equal(expected);
});
This is the first recipe monster we're going to encounter. Start reading enough recipes with computer based parsing in mind and you'll quickly see why machine learning may ultimately be a better way to parse recipes written by humans. We don't have that luxury with our parse() function though. Instead, we need to think about these edge case monsters lurking on the internet, waiting for our app to attempt to digest them. Our goal is to keep these monsters under control.
How many iterations of comments can you think of? Will parse() properly parse all of them at this stage in the app's development?
TODO: Insert system for user submitted tests here. Similar to mutation testing, a specific number of comment based entries could be run through a real recipe parsing function (not yet written) along with the student's tests. The parsing function would purposely be written to fail on unexpected comments. Then ensure that the number of failed student tests match the number of expected failures. Then regex the test expectations to see if they match the same kinds of comments on the list. Then provide hints for any missing tests.
Thus far, we have focused on whole numbers. But a large majority of non-metric based recipes include fractions. Let's make sure that these fractions are converted into decimals.
it('should parse ingredients with fractions', function(){
let input = '1/2 cup flour'
let expected = [{
raw: '1/2 cup flour',
quantity: 0.5,
unit: "cup",
name: "flour",
}]
expect(parse(input)).to.deep.equal(expected);
});
Try writing a few fraction based tests of your own. Make sure to think of all ways that fractions can be expressed online.
Fractions themselves aren't much of a monster. But vulgar fractions are. They may look like regular fractions to the human eye, but they are each their own unicode character. Vulgar monsters indeed; don't let them surprise you!
it('should parse ingredients with unicode vulgar fractions', function(){
let input = '½ cup flour'
let expected = [{
raw: '½ cup flour',
quantity: 0.5,
unit: "cup",
name: "flour",
}]
expect(parse(input)).to.deep.equal(expected);
});
There are only 18 vulagr fractions. But that's not the only way a recipe will corrupt output in our parse() function if we aren't prepared. Let's make sure we have test coverage around fractions based on fractions using Unicode's fraction slash, superscript, and subscript numerals:
it('should parse ingredients with unicode fractions', function(){
let input = '₇⁄₈ cup flour'
let expected = [{
raw: '₇⁄₈ cup flour',
quantity: 0.875,
unit: "cup",
name: "flour",
}]
expect(parse(input)).to.deep.equal(expected);
});
Don't forget packaged ingredients. These will often times hava a count and a size for the package.
it('should parse canned ingredients', function(){
let input = '2 15-ounce cans black beans'
let expected = [{
raw: '2 15-ounce cans black beans',
quantity: 2,
unit: "15-ounce can",
name: "black beans",
}]
expect(parse(input)).to.deep.equal(expected);
});
Can you think of any other packages that would differ from the previous one?
How about cans with a range? In this instance, our app is going to be opinionated when parsing. If it works with 15- or 16-ounce cans, let's just always use the lower end of the range. We need a test to verify that doesn't change in the future.
it('should parse can size range', function(){
let input = '1 15- to 16-ounce can black beans'
let expected = [{
raw: '1 15- to 16-ounce can black beans',
quantity: 2,
unit: "15-ounce can",
name: "black beans",
}]
expect(parse(input)).to.deep.equal(expected);
});
Under some recipe style guides, the one scenario where spelling out a number is acceptable is if two separate numbers are next to each other. Like this can example. Monsters; even in an official style guide for a food magazine! Of course, at least that is an hard opinion. Any food blogger will do whatever they please anyway. So be prepared for spelled out numbers.
it('should parse word numbers', function(){
let input = 'two 15-ounce cans black beans'
let expected = [{
raw: 'two 15-ounce cans black beans',
quantity: 2,
unit: "15-ounce can",
name: "black beans",
}]
expect(parse(input)).to.deep.equal(expected);
});
What about recipe ingredient strings with more than one ingredient? This can happen, for instance, when one ingredient must be dissolved in another ingredient. This really should be two separate ingredients with the disolving step in the directions. However, we can't stop recipe authors from creating more monsters.
You were warned that the results being in array was to combat monsters. Here is one reason why. Like said above, these really are two ingredients despite needing to be kept together in our app. Hence the need for two ingredient objects bound to each other in an array.
it('should parse \"dissolved in\"', function(){
let input = '2-3 tablespoons unflavored gelatin, dissolved in 1/2 cup water'
let expected = [{
raw: '2-3 tablespoons unflavored gelatin, dissolved in 1/2 cup water',
quantity: 2,
quantityEndRange: 3,
unit: "tablespoon",
name: "unflavored gelatin",
}, {
raw: 'dissolved in 1/2 cup water',
prefix: "dissolved in",
quantity: 0.5,
unit: "cup",
name: "water",
}]
expect(parse(input)).to.deep.equal(expected);
});
Our parsing function chose to refer to "dissolved in" as a prefix to the ingredient. The same can be said of alternative measurements inside of parenthesis. These alternative measurements and units could be recorded differently, but we already have a need for multiple ingredient objects, so we can use that the same here.
it('should parse alternative measurements in parenthesis', function(){
let input = '250 grams (1 cup) cassava flour'
let expected = [{
raw: '250 grams (1 cup) cassava flour',
quantity: 250,
unit: "gram",
name: "cassava flour",
}, {
raw: '1 cup cassava flour',
prefix: "or",
quantity: 1,
unit: "cup",
name: "cassava flour",
}]
expect(parse(input)).to.deep.equal(expected);
});
What about ranges of measurements? And multiple ranges in different units of measurement? We need to test for these scenarios too.
it('should parse range in parenthesis', function(){
let input = '2 to 3 tablespoons (20 to 30 grams) cinnamon'
let expected = [{
raw: '2 to 3 tablespoons (20 to 30 grams) cinnamon',
quantity: 2,
quantityRangeEnd: 3,
unit: "tablespoon",
name: "cassava flour",
}, {
raw: '20 to 30 grams cinnamon',
prefix: "or",
quantity: 20,
quantityRangeEnd: 30,
unit: "gram",
name: "cassava flour",
}]
expect(parse(input)).to.deep.equal(expected);
});
TODO: Add project to create more tests; still need to figure out a way to test all of the additional tests though.