All right, everybody! Welcome to my talk. ES2016, the evolution of JavaScript. First a little bit about me. My name is Jafar Husain. I'm a tech lead at Netflix, I work for Falcor, an upcoming open data platform, which we intend to release pretty soon, and I'm also one of Netflix's representatives on TC-39, which is JavaScript's standard's committee. This talk used to be called ES7, the evolution of JavaScript, but something happened a couple of committee meetings ago. We decided to change ES6 to ES2015 and ES7 to ES2016. I want to explain this name change. We as a committee want to start shipping JavaScript every year. Just the way you would ship software in an agile way, we want to add features and ship them quickly. That's why we have this new name. The language formerly known as ES6 is ES2015 and the upcoming language version is ES2016. ES2016 features are already starting to roll on the browsers. For example, there's object.observe in Chrome right now. And it's important to know that ES2015 and ES2016 were developed concurrently. So although there was this big -- many, many years between ES5 and ES2015, what the committee is doing is they're developing new features. In the latest version of the language, at the same time that they're developing features in the next version of the language. And that's exciting, because it actually helps us plan the future of JavaScript. We don't have to do everything in one release. We can add features today to ES2015 and we can build on them, with the hopes of building on them later in ES2016. So the big story about ES2016 is: how do we make async programming easier? I think there's probably some folks out there who think this is kind of a hard problem. Right? Any pain out there, about async programming? Quite a few people, I'm guessing. So starting with ES5, we introduced something to make your life a little bit easier, which is arrow functions. I'm not going to dwell on this too long, but as of ES2015, you're now able to write this way. A lot of people are familiar with arrow functions. I'm not going to spend too long on this. This definitely helps when you have a lot of callback. Arrow functions are an improvement, but can we do better? What if you can write async programs without any callbacks at all? How does that sound? Right? Yeah?
So we all know that blocking is easy. Right? Blocking is no problem. Blocking -- let's imagine for a moment this function which gets a stock price, calls, gets stock symbol, which makes a blocking XHR request, one of those things we're never supposed to do. Never make a blocking request. But it's easy, because we can wait for the request to come back, block when it does, and pass the symbol for that stock and get the price for it. We know blocking is easy but it produces a terrible User Experience. Because user might be sitting there and our UI is not responding to mouse clicks while we're making our request. So that's something you're not supposed to do in JavaScript. But when I say blocking, I want you to think pulling. When we call a function and receive the output in the return position of that functioning, we're pulling the value of that function. To the left hand side, pulling the value out. So when I say blocking, I mean pulling, and that means data is delivered in the return position of the function. That's how most functions are in JavaScript. Most functions return their data in the return position.
But what if we decide that we want to wait instead of block? We don't want to block the UI, waiting for the output and not being able to respond to user input. We want to be able to just wait. Well, in order to wait in JavaScript, waiting means pushing. What it means is you hand the callback to that function and that function pushes the value, the result of that function, in the argument position of your callback. So this is a thing you would see in node. You hand the function to callback and it pushes the value, the result, in the argument position. So as soon as we try to do that, as soon as we try to wait in JavaScript, we very rapidly find ourselves in the pyramid of doom. If you've done a lot of node.js programming, you probably know what I mean. Your code takes on this pyramid-like structure that you see there on the left hand side.
So on the left hand side here, with get stock price, we're doing a blocking function. Just the same function you saw earlier. But as soon as you want to move to waiting on the right hand side, take a look at how much more complicated our function becomes. Why is there so much more code here? For one thing, it's because we're doing the job that JavaScript usually does for us. If an error is encountered anywhere in this function, now as soon as we're using callbacks to forward on those errors, we're responsible. For making sure that those errors get forwarded up. Whereas when we're doing blocking code, we have try/catch to rely on. So as soon as -- the problem really, what's going on here, is since JavaScript doesn't expect functions to wait, in JavaScript, they sort of expect functions to block -- there's no support for reporting errors this way, and the language can't provide you with any sort of support, whether it's loops or try/catch, so you're kind of on your own.
But what if waiting were just as easy as blocking? What if JavaScript understood this concept of functions that waited, and it could provide you syntactical support just like try/catch, and you could use loops, for example, to repeat asynchronous functions? Well, in ES2015, we got a little closer to this vision, with promises. Now, the way I think most of you are probably familiar with promises -- most of you have been exposed to them at this point. But the way to think about a promise is that it's an object that represents the eventual value of an asynchronous operation. So if you want to get the value from a promise, you call then and pass to callbacks. One is for receiving data if the asynchronous operation succeeded. That's on the left hand side there. And the other callback is for receiving the error if the asynchronous operation failed. So it's sort of like a promise is like a present that you unwrap, and great -- if everything worked out, you get a value. But if things go bad, you can get an error. So now let's take a look at that same previous example, waiting with callbacks, and that pyramid of doom structure you see there, and let's see how we can now do the same thing with promises.
So what's happened here is promises actually take care -- they catch any errors that occur, and forward them up until they're actually caught, where you pass an error handler, notice on get stock price, we pass an error handler. We don't have to catch errors at every possible step in the function. We just catch errors in one place and the promise type will catch errors and forward it up until -- there's a function provided to handle that error. So what's actually happening is that type is doing the job that try/catch does for you in the language. So that's how we got back down to that nice little piece of code. That's an improvement. No more pyramid of doom. It's not bad. But why can't we just write code that looks pretty much the same, regardless of whether it's waiting or blocking, right? What if we could just write our algorithm and make it easy to switch that algorithm between waiting and blocking? That promise code you saw on the previous slide looks different from the synchronous blocking code you saw earlier. As of ES2015, you will now be able to wait like this.
So using generator functions and the yield keyword, which is something I'm going to tell but later, you can actually write code that looks pretty much exactly like the blocking code, but instead waits. It doesn't block. And so you can allow handling of user input, but you can write your code top to bottom. More, you can take advantage of looping constructs to block flow in an asynchronous way. So on the right hand side I can use loops and I can even use try/catch. How do we make this work in JavaScript? In order to explain that to you, I have to explain generators. But the metapoint I want you to understand here is that pulling and pushing are symmetrical. You can do the exact same thing -- write your code precisely the same way, whether it's pulling or pushing. It's kind of a detail. Somehow be able to write your logic and after the fact decide whether you want that function to push or you want it to be a pulling function. Anything that you can push, you can pull out of a function. I'm going to demonstrate that in just a moment.
So in order to understand how that works, I'm going to explain you to the most powerful and most misunderstood feature in ES2015, and that is generator functions. Hands up -- quick, how many people have heard of generator functions? Great. Quite a few people. How many people understand them? Right. Okay. So that's -- hopefully I want to rectify that today. If you get one thing out of this talk, it'll at least be a slightly better understanding of what generator functions are, because as I said, they're really powerful. At a high level, this is how I want you to think about generator functions. It's a function that can return multiple values. Well, you can already make a function that returns multiple values. Throw a few values in an array. But this allows you to do that more efficiently. This is what generator functions look like in ES2015. This yield keyword is like an intermediary return. It runs in the middle and you can return more values. So if you're consuming the data from a generator function, this is what it looks like. In order to get the data out, you request an iterator. By calling that function, what comes out is an iterator. An iterator is what you can call next on, and then what happens is the function evaluates to the first yield, and then stops and returns that value. And then it pops out of its little tuple which contains the value that was returned and a Boolean telling you whether or not there are more values. So if done is False, we can call next again, and execution is going to resume and move to the next yield and then pause and then return you another tuple, saying that value -- we know we have more to do, so I'm going to keep calling next, until finally we get return and we get our last value, and we know we're done, because done is True. If I want you to write a get numbers function today in ES2015, you would probably roll a sync machine. Calls next and I get the next value. Let's look at this Fibonacci sequence function here. And I'm going to show you what it desugars to, from ES2015 to ES5. So you can see I've got this stink variable on the left hand side and that's keeping track on of where I am. And I'm returning the object with this method, that's the iterator you saw earlier. Now, what this function actually is doing -- you can see here what generator functions do is they figure out every possible state that that function can be in and they build a state machine. So this get Fibonacci sequence can be in three different states when you call next. Where it's supposed to return zero at the beginning, where it's supposed to return one, and finally whatever it returned last time, whatever it returned before that, add them together, and return them. In JavaScript, in ES2015, this generator function is actually smart enough to figure that out and build this state machine for you. And what it's really doing is it's turning what looks like a push. Notice on the left hand side here -- yield kind of looks like a push. Imagine yield was a callback. Right? And there was brackets around that yield. The code looks like we're pushing the data. But the compiler turns it into code that pulls by building a state machine on the left hand side. So that's effectively what generator functions do. They let you write code that looks like you're pushing information when in fact it turns it inside out and then it uses pull for the control flow.
So for each one of these yields, we have that return which returns the tuple, the value, and the done, and increments the state variables, as it goes along, and that's how we move through the states. This allows us to use iteration. The whole process of pulling a value out of it until you're done is called iteration. And it involves a consumer and a producer. The consumer requests an iterator from the producer, and then it pulls the value out by calling next. So the producer emits a value. Right? And why do I say it's pull? Well, notice I'm getting the value out in the return position of next. So I'm pulling functions out. I'm pulling values out. I keep pulling values out, until finally I pull out a value and the producer says you're done. So that's how iteration works. You keep pulling values out until the producer says I'm done. No more data.
So if generator functions return iterators, why are they called generator functions? Why don't we just call them iterator functions? There's more than meets the eye when it comes to generator functions. A generator is actually built out of two different interfaces. The iterator you saw earlier, where you can call next and get out the little tuple saying what the value is and whether you're done -- it also has an entirely other side, a split personality, which is an observer interface, which gets push values. So a generator is both a source of data, which you can pull data out of, but it's also a sync which you can push data into. So these two sources combined create a generator. A generator is an iterator. You can pull a value out. If you attempt to pull a value out, you can get an error, because it might throw if you call next. And finally you can pull a value out and get that done:true in that little tuple. So there's three types of notifications that a producer can tell you during iteration. I've got some data. An error. And here's your final value. You can see all of those notifications in reverse on the observer side. So when you call next on a generator, you can also push a value in by passing a value in the argument position of next.
You can also send an error in, by calling throw, and finally, you can send in a final value by calling return. So why does an observer have this sort of split personality of both being a source of data and sync? It's to solve a little problem. And that's that iteration only allows data to flow in one direction. Whereas with generators, two functions can effectively have a conversation. Have a long running conversation. I put a value out. I push a value back in. I pull a value out, I push a value back in. Why would I want to do that? What does that buy me? It allows us to use a very powerful pattern that I think a lot of people in the room are going to be using in the next few years called asynchronous iteration. So we have our generator function and the first thing I want you to notice is that I told you to use a mental model for understanding yield. The model I gave you was -- it's sort of like an intermediary return. But in this particular piece of code, it's used like an expression. We're siding to the result of a yield. So what's that about? Here we have a producer function.
And what it's actually going to produce if you notice what's on the right hand side of those yields -- it's going to be an iterator of promises. So we're going to pull promises out of this iterator, and then what we're going to do is we're going to resolve those promises and then push the value back in. And so we need a consumer to go along with this producer. The consumer's job is just going to be to pull out, by calling next, pull out promises, resolve those promises, and push the value back in. And what you're going to see later is when you push a value back into the generator, the whole yield expression gets replaced with the value that you put back in. And this is the key. This is the recipe to allow you guys to write code that looks -- that looks like it blocks, but actually waits. So let's take a look at how this works. We have this spawn function, which I'm going to define in just a moment. Think about as the consumer -- it's pulling promises out of this iterator promise. So if I run this, what spawn is going to do is it's going to produce a promise which will be the eventual result of get stock price, after we resolve the get stock symbol promise and the get stock symbol price promise. So when I run this code eventually it's going to run the price of an individual stock. In this case, Pfizer stock. Well, asynchronous iteration works this way. You've got a consumer and a producer. The consumer requests a generator from the producer. And then calls next, pulls out the first value, but notice -- execution suspends at the first yield. And what we're yielding back is a promise. It's a promise that will eventually resolve to the symbol for Pfizer. And so that promise is handed to the consumer, and the consumer and the producer have an agreement. The consumer says -- every promise I pull out, I'm going to resolve, and then I'm going to push the value back into the producer.
So the consumer calls then on the promise, waits until it's resolved, and then calls next again, on the generator, from the producer, except this time -- notice we pass PFE in the argument position of next. We're effectively pushing a value in, at the same time as we're pulling a value out. So we send PFE back in, and now I want you to look at what happens to the yield expression. It effectively gets replaced with a value that we get pushed in, and then execution resumes, and pauses at the next yield. So now the consumer's pushed in a value, and at the same time, they're pulling out another promise. This time for the price of that particular stock. So agreement between the consumer and the producer -- the consumer is going to resolve that promise. And when it gets a value, it's going to call next again, and this time, push 2783 back into the producer. That replaces yield. And we continue and stop at the final return value. So now we've actually returned -- it's not a promise. It's just a plain old value. Back to the consumer. And we said -- hey, we're done. We're not going to give you any more data. At this point, the consumer takes the promise that it returned, the spawn function, and it resolves it to the final value, 2783. So pictures are great. But let's take a look at some code.
So here I have get stock price. And then I have this spawn function, which consumes it. So notice at the bottom I'm calling spawn, but I'm passing in the generator that comes out of get stock price into spawn. And what's going to come out of that is a promise that's eventually going to resolve to the final price of that stock. So I run this. And what happens is we immediately create a promise inside of the spawn function. And execution runs to the very bottom and hits this onresult function. And this function is basically my way of asynchronously looping and continuing to resolve promises. You're going to see me recursively call this function again and again, every single time I get a new value out of a promise. So we go up here, first time around, the last result is undefined, haven't gotten any yet, as soon as I call next, notice up there at the top, execution stops at the first yield and we return a promise that symbolizes the eventual stock symbol. And that gets returned to the consumer. If it's a promise, the consumer calls then on it. Notice here we're passing on the onresult function we started with. If but this time when the on result function is called it's going to get the result of that promise. So as soon as I do this, we're going to hop up back here, and the last promise result is going to be whatever came out of that promise. JNJ. So now I got the result of the promise as a producer, but I want to push it back -- excuse me, the consumer. I want to push it book back to the producer. So I call next. I pass back in JNJ as the argument, you look at the top there, notice what happens to the yield symbol promise expression -- it gets replaced with JNJ and execution resumes and stops at the next yield point. So get symbol price is a promise that eventually resolves to the price. That's what comes out to the consumer. Another promise. We call then on it again. Pass in on result, that recursively gets resolved to the next promise, which is the price. We call next, yield gets replaced with that price, and then we continue on to the very end of the generator function, at which point it comes out and says -- you know what? Done is true. We're not going to get any more values. Consumer takes that, doesn't need to resolve it, it's not a promise, and then resolves the overall promise that was returned from spawn to that final value.
And that's how asynchronous iteration works. And that's how you can write code at the top, right? That looks like it blocks, but it actually waits. So there is this spawn function actually defined in a library that you can go and try, which is called task.js. If you're using a transpiler that allows you to use yield in your code or you're using a browser that is ES2015 capable, lucky you, you can do this today. You can download the spawn function, which is in task.js. So that's how asynchronous functions work and how generators interact with promises to produce this pretty impressive code. But why should we have to download a library to do this? This is going to be a common expression in JavaScript. Why do we need a library? Why can't we have direct support in the JavaScript language for this? And that is our first feature today from ES2016, and its async functions. And now you guys know everything you need to know about how async functions need to work. They are like sugar over what you just saw. This combination of spawn and yield -- soon in the next version of JavaScript you'll be able to write code like this, on the left hand side. For those of you who are familiar with C#, this feature is there as well. You can just put the word async in front of any function and then inside of there, any time you want to pause until a promise has been resolved, you can throw a wait in there. So that's what's coming in the next version of JavaScript. So we're getting closer to this vision of symmetrical support for blocking and waiting in JavaScript.
Right? If I want to wait, I throw an async in front of that function. I throw a few waits in there. If I want to block, I take them out. In order to talk about any features in the next version of JavaScript, responsibly, we have talk about the ES feature maturity stages. So as we develop new protocols in JavaScript committee, we have to give time to move them out. So I'm going to tell you about which stage it is in maturity. So async functions are in the draft stage. That means that the committee expects the feature to be developed and included in the standard. I think it's very, very likely we're going to see async functions in the next version of JavaScript. It's definitely something you can play with today if you use Babel or a transpiler, like Tracer. Right now. If you want to. Go ahead, try it out. That's why it's there. And give us your feedback. You can try it in Babel, if you want to try it in Regenerator as well -- I think they're in the main branch. Babel is probably the easiest and most likely way by which you're going to try this feature. The await keyboard makes it easy to await one asynchronous value. What if you want to wait on multiple values? We await multiple values all the time. Web sockets, DOM events. Lots of streams of information that are being pushed at us, and there's no real language support for traversing those streams of information.
Now, in ES6, all collections became iterable. And it's a contract. If you walk up to the array, for example, it means you can ask that array for an iterator, and gradually consume its items one at a time by pulling them out. Here's an example. I can call this symbol.iterator method and get out an iterator and keep looping and calling next. That's annoying, a lot of boilerplate, which is why in ES2016 we introduced this for of loop. So there's a couple of new collections coming in ES6 as well, like maps, and you can use this -- to consume that data. Progressively. So that's what the new for of loop is for. It just desugars to what you see on the left hand side. Now, if we can wait for values that we can pull out, for streams of values that we can pull out, why can't we wait for streams of values that are being pushed at us? Why can't we have an equal support, like a loop for consuming data that comes out of a web socket, or comes out of async IO, for example? That's the logic behind the proposed for on loop for the next version of JavaScript. If you can have a for of, why can't you have a for on? So I'm creating an async function and inside I'm consuming the prices from a web socket. And the very first price differential that's over a certain threshold -- I'm going to resolve the promise and grab that particular diff between the previous price and the next price.
No callbacks required. I'm just looping over a stream of information being pushed at me. So this would be great to do. Right? Now when I run this, it resolves to a promise to give you the next price spike in this stream of stock prices. But there's a little problem. There's a reason we can't have nice things just yet. Right? The problem is that the web has no standard observable interface. We have iterable. As of ES6, this contract you saw earlier, what you call symbol.iterator. But today we have this proliferation of different APIs that push us streams of data. That push us data in a callback in a streamed way. DOM events, web sockets, node streams, XHTML requests -- can all push you values. But they don't implement one common interface. So one thing proposed for ES2016 is the observable contract. So here we have the iterable contract, introduced in 2015. How are we going to get the observable contract? It's hidden inside of this type. If we just swap the arguments and the return type of the iterable what pops out is an observable. An observable accepts a generator, and then uses the push side of the generator, and pushes multiple values in it until it finally calls return to signal that no more values are coming. If you think about it, iteration and observation are both about the same thing. Both about a producer giving a consumer multiple values. The difference is in one circumstance with DOM events, the data is being pushed to you.
But when you're using an iterator to pull values out of an array, you're pulling values. Right? In iteration, the consumer is in control, and in observation, the producer is in control. The web socket decides when it calls you. So it's sort of like they're it rating you. The producer is iterating you by calling your callback. So how does observation work? The consumer and producer and the relationship with iteration -- it's the exact same process but kind of the inverse. So here the producer, instead of the consumer requesting a generator from the producer, the producer -- or the consumer hands a generator to the producer. And in this context, just think about a generator as like three callbacks. It's got the next, the throw, and the return callbacks. Just like you're handing an API three callbacks and expecting it to push information to you by calling your callbacks. So the producer produces a value and calls the next method on your generator, the one you provided to it, and that's how it pushes 42 to you, and pushes another value and at its leisure, decides to push 39 to you, until finally it says you know what? There's no more data coming. So it calls return to indicate to you no more data will arrive.
So observation and iteration are actually deeply linked. As we saw, we can turn one inside out and get the other. So this is what it would look like to consume a web socket or a DOM event that implemented this contract. You could just walk up to any of these data sources and hand it a generator, which is these three callbacks, and then it will just push streams of data at you. Until it tells you it's done. So we can add sugar, just like we added sugar for iteration, for that whole process of calling symbol.iterator, and that while loop, and checking the done property, just like we can add sugar for that, we can add sugar for observation. A 4H method, for example, to an observer that returned a promise when it resolved, or, as I showed you guys earlier, now that we have a well defined method for observation, which we do, a for on loop. So the push stream can be done entirely without callbacks. So the hope would be if we introduce observable, all the push APIs on the web can implement this contract and all of a sudden we can just add language support magically for all of these different push data sources.
So if I wanted to consume an observable in ES2016, at least as the proposal stands, I could do something like this. And this is just going to keep printing out new sign-ups at Netflix. Just going to keep going. So... If an async function returns a promise, and a function star returns an iterator, we as language designers have a question to answer. What does an async function star return? You can't just go adding features to languages and not describing what happens when you put them together, when they interact. Well, there's a couple of different opinions about this. And there's actually a couple different ways the committee could go. I'm here today to tell you about one particular option. So if we look at this table, we've got synchronous functions, which return a value, synchronous generators, which return multiple values, you can pull multiple values out of -- but we also have asynchronous functions, which return a promise, and a promise pushes you one value. Right? You give it a callback and it pushes the value on that callback. But what goes here?
Well, if a function star returns multiple values, and an async function pushes one value, maybe an async function star pushes multiple values. And what type pushes us multiple values? An observable. So we can envision, in the next version of JavaScript, an async function star, which pushes -- which actually returns you an observable. So you could build an asynchronous generator function, which consumes data, stock data, for example, and then translates it and produces new stock data. So now, instead of just getting the next price spike from a web socket, I want to get all the price spikes from a web socket. I'm going to use the for on loop and yield inside of async function star, and this whole thing is going to produce an observable that pushes me all the price spikes, all the stock differences, when one stock comes along and another stock comes along, and that price difference is over a certain threshold, just going to push it on through to me. And if I want to consume this, I can just use for on, inside of an async function. And print to the console. So what does that mean? It means symmetrical support for functions that return multiple values. Whether they're push or they're pull.
You guys can stop worrying about the machinery and the callbacks and just focus on your code. So... This particular proposal -- I just want to call out -- is at the strawman phase. The earliest phase of the process. So we're definitely going to be thinking about this more, and this may not make it through as a final feature. Just want to make that clear. It's very early in the feature stages. We're looking for feedback on this. So we want to hear from you about whether this works for you and solves your problems. So thanks very much, everybody. Here are some resources for you, and that's the end of my talk.