In case you haven’t been following the progress being made on JavaScript 1.8 here’s a quick overview of what’s already landed in the latest Firefox 3 nightlies.
So far, three main features have landed, with a few more to come. In general, this release will be relatively ‘light’, mostly attempting to bring the current JavaScript implementation closer to the desired JavaScript 2 spec.
Expression Closures
This just landed in trunk and I’m sure it’ll tickle those of you who enjoy functional programming. Effectively, this addition is nothing more than a shorthand for writing simple functions, giving the language something similar to a typical Lambda notation.
For example, look at the Lambda notation that’s available in a couple languages, in comparision to JavaScript 1.8.
Python:
lambda x: x * x
Smalltalk:
[ :x | x * x ]
JavaScript 1.8:
function(x) x * x
(For reference) JavaScript 1.7 and older:
function(x) { return x * x; }
I think, probably, my favorite use for this shorthand will be in binding event listeners, like so:
document.addEventListener("click", function() false, true);
Or combining this notation with some of the array functions from JavaScript 1.6:
elems.some(function(elem) elem.type == "text");
This will give you some JS/DOM code that looks downright elegant.
Generator Expressions
This is another one that was just committed to trunk. It’s a little bit more involved than the previous addition, as this one encompasses a number of concepts. Specifically, it requires knowledge of most of the features within JavaScript 1.7, specifically Iterators, Generators, and Array comprehension. This particular feature is based off of the generator expressions that exist in Python.
In the ticket tracking this feature, Brendan posted an elegant, functional, Sudoku solver written using the new syntax that this addition affords us. This demo is based off of a similar one written in Python used to demonstrate its generator expressions.
To better understand what this feature means, lets look at a single line of JavaScript 1.8 code, taken from the Sudoku solver.
dict([s, [u for (u in unitlist) if (u.contains(s))]] for (s in squares))
This line relies upon the dict() function, which takes a 2xN sized matrix, and converts it into a key/value pair object. The code of which can be found here:
function dict(A) { let d = {} for (let e in A) d[e[0]] = e[1] return d }
Let’s go through that line of code, part-by-part, to better understand what exactly is going on.
[u for (u in unitlist) if (u.contains(s))]
The first part of the line is an example of array comprehension coming from JavaScript 1.7. Specifically, we’re iterating over ‘unitlist’ and building it out into an array of keys (excluding which keys don’t contain ‘s’).
[s, ...] for (s in squares)
The second part of this line is another example of array comprehension. At first glance, it appears as if it’s another feature from JavaScript 1.7, a destructuring assignment, although, that isn’t the case. A normal destructuring assignment only occurs when you’re assigning a value, however in this situation we’re just creating an array value using array comprehension. These new 2-unit arrays will then plug back into the dict function.
dict([s, ...] for (s in squares))
This is where the magic happens. In JavaScript 1.7 we could’ve called the dict() function like so:
dict([[s, ...] for (s in squares)])
Note the extra, explicit, use of array comprehension. The issue with that extra comprehension is that it has to be completely run when it’s first encountered, in order to build out the full array (which will then be turned into a ‘dictionary’). However, the lack of extra […] is what makes this a generator expression. That makes that line in JavaScript 1.8 equivalent to the following in JavaScript 1.7:
dict((function(){ for (s in squares) yield [s, ...] ; })())
As you’ll see, it the generator expression lazy-builds the resulting array, handling it as a generator – meaning that the specific values won’t have to be generated until they are explicitly needed by the dict() function (resulting in less loops and better overall performance).
Here’s another example of a generator expression
// Create a generator that loops over an object values function val_iter(obj) { return (obj[x] for (x in obj)); } // Iterate through an objects keys for ( let key in obj ) { ... } // Iterate through an objects values for ( let value in val_iter(obj) ) { ... }
Of course, the val_iter() function could be built with JavaScript 1.7 right now, using yield:
function val_iter(obj) { for (let x in obj) yield obj[x]; }
Most likely, though, generator expressions will see the most use in memory/cpu hungry code (like the Sudoku solver), since solutions will now be able to get results when they need them, as opposed to loading them all up front.
Fun with iterators
On a side note, I’ve been playing with iterators/generators a lot more lately and one particular feature that I’ve wanted was the ability to easily iterate over a set of numbers. (e.g. 0 – 9) With a little bit of dancing, we can add that functionality to the language, like so:
// Add an iterator to all numbers Number.prototype.__iterator__ = function() { for ( let i = 0; i < this; i++ ) yield i; }; // Spit out three alerts for ( let i in 3 ) alert( i ); // Create a 100-unit array, filled with zeros [ 0 for ( i in 100 ) ] // Create a 10-by-10 identity matrix [[ i == j ? 1 : 0 for ( i in 10 ) ] for ( j in 10 )][/js] This may be old hat for some, but I've gotten a real kick out of it.</blockquote> <h2>Array Reduce</h2> The last feature to <a href="https://bugzilla.mozilla.org/show_bug.cgi?id=363040">sneak in</a> recently was the addition of the missing Array.reduce/Array.prototype.reduce function, from the <a href="http://developer.mozilla.org/en/docs/New_in_JavaScript_1.6#Array_extras">JavaScript 1.6 Array Extras</a>. You would call reduce on an array, something like this: [js]someArray.reduce( fn [, initial] );with the function being called like so:
someArray.reduce(function(lastValue, curValue){ return lastValue + curValue; });The “lastValue” argument is the return value from the previous call to the reduce callback function. The first time the callback is called, it’ll have a “lastValue” of the first item in the array (or an initial value, if you’re provided it in the call to the reduce function) and a “curValue” of the second item in the array.
Thus, if you wanted to sum up the numbers 0-99, you could do it like so (using JavaScript 1.8, and the number iterator from above):
[x for ( x in 100 )].reduce(function(a,b) a+b);Pretty slick!
You could also use the reduce function to do things like “merge sets of DOM nodes into a single array”, like so (thanks Andrew!):
nodes.reduce(function(a,b) a.concat(b.childNodes), []);Try it yourself
Everything that I’ve mentioned above is live today in the latest nightly builds of Firefox 3, thus if you want to try some of the above for yourself, just do the following:
- Download a Nightly of Firefox 3
- Create a page that has the following script tag in it (which was just committed):
<script type="application/javascript;version=1.8"> ... your code ... </script>And that should be all that you need – enjoy!
Neil Mix (May 30, 2007 at 8:39 pm)
Great examples, John. Neat stuff.
Jordan Sissel (May 30, 2007 at 9:40 pm)
iirc lambda and reduce are both going away in Python 3. I think the reasoning was that lambdas were used almost exclusively for map/reduce/filter and friends. They’re removing map/filter in favor of array comprehensions and generator expressions. reduce(), according to Guido, is almost always used with a + b or a * b, etc, so reduce is going away in favor of sum() and product(). Seems reasonable to me.
The reasonings are logical and I agree with them. Strange to see them pop up in javascript now.
Guido on map/reduce/filter goinga way in Py3k: http://www.artima.com/weblogs/viewpost.jsp?thread=98196
John Resig (May 30, 2007 at 10:10 pm)
@Jordan: I, also, am not a huge fan of .reduce(); although that may be because I haven’t seen an immediate need for it as of yet. But yeah, I agree with the conclusion that array comprehensions can mostly replace the need for functions like map/filter/reduce.
The lambda bit is really nothing more than a shorthand for typical JavaScript functions, so I’m less concerned with that. Really, since functions are such a critical part of JavaScript, in general, it makes sense to have their syntax be shorter.
Andrew Dupont (May 31, 2007 at 12:03 am)
Prototype uses “inject” (our version of reduce) internally for things like DOM node accumulation. Imagine something like getting all of a collection’s child nodes:
function getChildren(nodes) {
return nodes.inject([], function(total, node) total.concat(node.childNodes));
}
The first argument to inject is the starting value (the thing that gets passed to the first callback. So you can slide an empty array into a collection and allow each item to toss some stuff in.
Slava Pestov (May 31, 2007 at 12:28 am)
> lambdas were used almost exclusively for map/reduce/filter and friends
Absurd.
> reduce(), according to Guido, is almost always used with a + b or a * b, etc, so reduce is going away in favor of sum() and product()
I use reduce all the time, but almost never with + or *.
> The reasonings are logical and I agree with them. Strange to see them pop up in javascript now.
Python is a procedural language with OOP bolted on top; JavaScript is multi-paradigm.
Yuh-Ruey Chen (May 31, 2007 at 12:48 am)
Er, correct me if I’m wrong, but that example doesn’t use destructuring assignment at all. That example was just building an array. Destructuring assignment only occurs in assignment exprs or within for-in statements/exprs, e.g. [x,y] = z; for ([x,y] in z); [x for ([x,y] in z)].
Jordan Sissel (May 31, 2007 at 12:48 am)
@Slava – don’t shoot the messenger, I was mostly summarizing the article I linked to except where places I said “I agree”. Curious, what else do you use reduce for that can’t be expressed in list comprehensions and friends? max and min? I’m sure there are others. Anything neater than simple summation or maximzation type problems?
@John – I agree with the shortening of functions for closures; I was always partial to perl’s codeblocks myself ;)
Anders (May 31, 2007 at 1:45 am)
> giving the language something similar to a typical Lambda notation.
Sure, if “f” or “fn” was made a synonym for the verbose “function”.
Not mentioned in this post, but a question regarding the type annotation of variables, does anybody know why it was chosen to put the types after the vars as in pascal instead of infront (and instead of the var keyword) as in c, c++, java (e.i. more recent/current/popular languages) and very much like the var keyword in c# (that was kind of introduced the other way around, but the result would be similar and the idea seems to be borrowed by c/c++ if not the actual word).
shabunc (May 31, 2007 at 1:47 am)
I strongly believe that such syntax:
function(x) x * x
can be very dangerous in complicated cases.
let us see what we’ve already have in js.
we have function(x){return x*x}
what problems do we actually have with such notation?
maybe, it is too long?
well let us write smth like |x|{return x*x}
оr even |x|{x*x} – in case of perl-like return;
but i defiinitely believae that curly braces shouldn’t be removed/
Brendan Eich (May 31, 2007 at 1:51 am)
Here is an example of reduceRight, based on a posting to LtU (sorry, I’ve lost the reference so can’t give proper credit atm), that does not fold trivially:
BTW, the real win in adding generator expressions for the JS version of Peter Norvig’s Sudoku solver was in the argument to some() called from search().
/be
najtje (May 31, 2007 at 3:10 am)
The lambda stuff does not appeal to me. I think what really is needed is a kind of function literal similar to { } for object and [] for arrays. What’s a big deal even if curly brackets are omitted ? You still have the keyword in front of the whole statement.
Laurentj (May 31, 2007 at 3:11 am)
> dict([s, [u for (u in unitlist) if (u.contains(s))]] for (s in squares))
Sorry, but for me, things like that are UNREADABLE. All new stuffs since JS1.7 make javascript code less and less readable and comprehensible by a “normal” developer. I think this stuffs won’t be adopted by most of developers…
Francois (May 31, 2007 at 4:04 am)
I totally agree with Laurentj. I make programing since 25 years (I began with cobol – what a shame no?) and web since 10 years, using of course JavaScript (many, many, many thanks to Brendan Eich).
I think I’m a little dumb, too old (and french too), but now I don’t understand what you’re doing and I don’t understand where you are taking us.
I think the “new style” of coding is hard to read, understand and debug.
I don’t understand why every framework are managing objects differently, trying to mimic Java.
Why functional programming ? It’s mathematics and how many “normal” developer will use it ?
Why a Generator Expression (after “like Java” it’s “like Python”) ?
I read and heard many and many posts and videos about the “so bad object model of javascript” (personnaly, I’m happy with prototyping inheritance). If this model is so bad why don’t make it better instead of this [[ i == j ? 1 : 0 for ( i in 10 ) ] for ( j in 10 ) ]
Masklinn (May 31, 2007 at 4:23 am)
> iirc lambda and reduce are both going away in Python 3.
Wrong, reduce goes away (which is retarded) but lambda definitely doesn’t.
Please refer to PEP 3100 for the actual, planned changes for Python 3.
> reduce is going away in favor of sum() and product(). Seems reasonable to me.
No, it’s retarded, ‘+’ and ‘*’ are only used as example of folds because they’re easy to understand. As Slava already explained people who actually use folds almost never use them with ‘+’ or ‘*’, what you’re saying is akin to saying “well since people only use recursion to compute fibonacci suites let’s make fib a built-in function and the need for recursion goes away”.
Just because the fib is the most common introduction to recursion doesn’t mean it’s anything more, and the addition/multiplication folds are exactly the same.
> Why functional programming ?
Because it’s a fairly simple and extremely powerful abstraction.
> It’s mathematics
no.
> and how many “normal†developer will use it ?
Those who already program in functional style right now.
> Why a Generator Expression
Because generator expressions yield an easy way to generate lazy evaluation, and lazy evaluation is cool.
> If this model is so bad why don’t make it better instead of this [[ i == j ? 1 : 0 for ( i in 10 ) ] for ( j in 10 ) ]
Because changing the object model is a breaking change in the language (which is planned for JS2, afaik) while adding features is an incremental change which doesn’t break the language in any way.
Additionally, these features right now are mainly for the Firefox extensions developers as no browser other than Firefox implements JS1.6 (*), let alone 1.7
*) Not completely true, I know that the current Webkit nightlies have at least a partial implementation of the JS 1.6 Array Extras.
Laurentj (May 31, 2007 at 6:24 am)
>>It’s mathematics
>no
I think you should read the definition of functional programming http://en.wikipedia.org/wiki/Functional_programming
>Those who already program in functional style right now.
stop insincerity ! Most of web/XUL developpers in the world don’t know functional programming.
Keep javascript simple !
>Because generator expressions yield an easy way to generate lazy evaluation, and lazy evaluation is cool.
Generators are cool in theory, but here, the syntax sucks ! Because for a maintainer, the only way for him to know if a function is a generator or not, is to find a little keywork (yield). Sorry but in a complex function, it is not obvious !
It’s cool for nerds, not for maintainers or “normal” developers.
A better solution to implement Generators, would be to use for example “generator” instead of “function” to implements a generator (ex, in one of the example, writing “generator val_iter(obj) {” instead of “function val_iter(obj) {“). So a developer knows immediately that it is a generator.
Mike McNally (May 31, 2007 at 7:29 am)
If that’s really the Javascript 2 semantics of “reduce”, I’ll be sticking with Prototype’s “inject.” Having the accumulator argument not be something the calling argument provides makes the operation almost useless.
João Marcus (May 31, 2007 at 7:31 am)
The new features do *not* make the code harder to debug. The *abuse* of the new features to build absurdly large one-liners surely makes code look like shit, but it’s up to the developer to find the balance.
List comprehension is *very* useful, as well as generators. It allows for some very beautiful code. You only need a couple of hours to get used to such features.
John Resig (May 31, 2007 at 9:45 am)
@Andrew: Good call, the reduce function does, in fact, take an initial value (I missed it the first time around). I’ve updated the blog post to reflect this.
@Yuh-Ruey: Good catch, I fixed the explanation.
@Mike: Note the revised post. I misinterpreted the code the first time around – it does, in fact, take an initial value (much like Prototype’s inject).
Jason Orendorff (May 31, 2007 at 11:01 am)
reduce()
is really just afor each
loop.Taking Brendan’s
reduceRight()
example:You could write it like this instead:
You might even like this better. There are more variables, but they have better names: n for numerator, d for denominator.
Likewise, Andrew Dupont’s example:
could be written like this:
But in this case what you really want is:
Code using reduce() is harder to modify and often harder to read than code using a for loop. In some cases it’s exactly what you want to say; for me that is quite rare.
Hans (May 31, 2007 at 12:50 pm)
At first I liked the lambda style shorthand for creating functions above as I pass anon functions as args to other funcs or object methods all the time. but on second thought the shorthand may end up making custom code harder to read and maintain by others in a production environment – especially when passed in with other args. Without the return statement and curly braces its definately tougher to see at a glance where one arg ends and the other begins… plus it pre-supposes some knowledge of functions as first class objects in JS.
It’s kind of a moot point for awhile anyway (unless you do XUL stuff) until it’s commonly supported by more of the major browsers.
Brendan Eich (May 31, 2007 at 1:57 pm)
najtje and others, please notice that return is also omitted along with braces. The function keyword is still there, and we’ve argued endlessly over the value of a short-hand for it, and no one has made the case (Unicode Greek lambda, \u03bb or javascript:String.fromCharCode(0x3bb) is a legal JS identifier). C# syntax of the (a,b,c,d) => e is not easy for top-down parsers *or humans* to read (no introductory keyword).
It’s important to note that generator expressions are purely syntactic sugar for generator function expressions that are called immediately. You don’t have to use them if you prefer to write out the functions the long way and call them by name. After working a bit that way, you will probably prefer to use the sugar, though.
JS is mature enough that it already admits of several styles or moods of programming. It would be a mistake to cut it on anyone’s particular Procrustean bed to fit one style only.
/be
Brendan Eich (May 31, 2007 at 1:57 pm)
Jason: loops are not functionally compositional (functions are ;-). So there’s room for a reduce (and reduceRight) function in the Array extras even with for-in loops and comprehensions.
/be
voracity (May 31, 2007 at 11:34 pm)
Can you use array methods on generators and expression closures?
i.e. Instead of:
you use:
voracity (June 1, 2007 at 1:54 am)
er, s/expression closures/generator expressions/. And the array methods would only be those that can take iterators, of course.
Cynebeald (June 1, 2007 at 3:40 am)
Why is everyone equating lambda expressions to closures? The ability to write a function as an expression is NOT THE SAME as having having the ability to acess variables from an enclosing scope inside that function.
So basically, this is “just” a lambda:
But using a local variable from the enclosing scope is a closure:
Jason (June 1, 2007 at 10:16 am)
Brendan: What does functional composition buy you here though?
Don’t get me wrong, I don’t mind having Array.reduce and reduceRight.
Jeff Walden (June 9, 2007 at 2:07 am)
Masklinn:
The WebKit nightlies, to the best of my abilities, completely, correctly, and compatibly implement the array extras of JavaScript 1.6. All seven methods were implemented in WebKit, correctly but for a couple edge cases, as of a few months ago. I noticed a few mistakes in the implementations when reading their code, and I submitted a patch to make the methods completely compatible with Mozilla’s implementation; it’s been in WebKit trunk for a month or so. With that patch there should be no difference between WebKit’s array extras and Mozilla’s array extras, sans reduce(Right) for now at least.
Jared Nuzzolillo (June 11, 2007 at 10:22 pm)
I haven’t completely mulled these over, but I am quite excited!
I just have one question — if you are introducing the shorter lambda expression syntax, do you think it would hurt to have an alias for function “fn”?
So we go from:
function(x) { return x * x; }
to:
fn(x) x * x
instead of:
function(x) x * x
echaozh (August 1, 2007 at 6:48 am)
Jason:
Writing for each every time you do the iteration is a lot of redundant code. Using map helps you to be free from all of the for’s and in’s, and the variable name: what if one day you decided to rename those variables, say for they were too short? Using reduce/inject/fold functions helps you remove even more var declarations. When the piece of code is to be reused, you don’t need to wrap it up in a separate function, because it’s already one function call.
Higher order functions are another level of abstraction. OO is for abstraction. If you use C instead of C++, with no polymorphism, some big project may cause you real trouble. Higher order functions provide another way of looking at the world. Take the some/filter/collect/remove-if-not function for example,. If you have to filter the item list before you do everything else, display them, put them into the cart, etc, what should you do? Write the for each loop every time? Refactor the code and make up a function to do just that? When you write a function just to filter the items by color, what if next time you have to filter them by size, price, or any other properties? And other lists, say the friend list of a registered user. You have so many chances to do filtering in you app, and all the functions you write for code reuse look really alike. Why not abstract the structure away from them, and make that a higher order function.
Yaroukh (August 4, 2007 at 5:40 pm)
I also agree with Laurentj on the generators.
I don’t like the idea of changing a function into a generator just by sticking a “yield” keyword somewhere in its body.. (I wonder what discussion was behind the decision of not introducing the “generator” keyword for generators.)
ad lambdas:
In my early stages of coding and programming I did tend to compress my code as much as it was possible.. Then I moved to “more readable and transparent code” policy.
IMO lambdas will make reading of JS-code harder.
Brendan Eich (August 9, 2007 at 6:08 pm)
Yaroukh: the yield-in-function-makes-generator follows Python. We don’t see a greater value in being different here, modulo indent-based block structure and all the existing differences. Convergence, even a little, is good if the languages and their problem domains are “close”.
Our experience (see the SML code downlodable from http://ecmascript.org/) with expression closures has been both readable and pleasant. But we welcome feedback based on actual use.
/be
Hector Santos (August 10, 2007 at 1:12 am)
I’ve been programming for nearly 30 years with atleast 20+ languages under my belt and when I see this new JS 1.8 work and claims are made that this is FUNCTIONAL PROGRAMMING, I can’t help but scratch my head! Did I lose it? I wonder. What I am missing? For goodness sake, if you want APL, to deal with vectors, arrays amd matrics, then do it right with logical mental translations like it is done in REAL FUNCTIONAL PROGRAMMING and SYMBOLIC languages, such as APL. But to tell us these new syntaxs in JS 1.8 is, how did John put it, “This will give you some JS/DOM code that looks downright elegant.” is simply surreal! What kind of Pot are you guys smoking? I can undestand the possible need to improve performance, but use the COMPUTER POWER and possible alittle AI to internally process the current JS 1.7 logic for better performance. In other words, don’t change the language! But if you must, CALL it SOMETHING ELSE – not Javascript!
My two pennies.
Hector Santos (August 10, 2007 at 1:31 am)
“more readable and transparent code†policy.
Yaroukh, I agree with you 100%, I tend to believe all young idealist, include maybe you and I, fall into this code redunction syndrome and most will eventually, with time and experience, fall into the category writing code for clarity, transparency as well as propersity.
I just don’t get it. They are changing the language, for what reason? To create a (strange) syntax that a few individuals believes matches a highly subjective mental translation with the goal of improving performance?
Geez, typically you don’t change the language, just improve the compiler, the pcode RTE or what have you. But in reality, as it historically proven over and over again with nearly all heavy handled symbolic functional language through the computer age, there was always a much cheaper and guarantee way to improve code processing performance – the next generation faster CPU/hardware!
Improving performance should not be a main reason for CHANGING the JavaScript language syntax, especially one that is catching on across the board as a viable interpretive language not only for the client, but for servers as well.
—
HLS
David Toso (August 20, 2007 at 12:37 am)
I really don’t get why the “experienced” imperative/OO programmers are complaining so much… if you don’t use (or ban by policy) the new features, then Javascript (
David Toso (August 20, 2007 at 12:42 am)
Damn – lost it. Oh well… you imperative programmers are spared my novella of wrath ;)
Michael Sync (October 1, 2007 at 7:19 am)
Very interesting.. I found this link yesterday when i was reading about js ver 1.7 and 1.8…
the way of writing the program and the concept are very highly improved day by day.. On the other hand, the code written in latest version of programming language become very strange…..
Thank you so much for your article..
Jack Diederich (October 5, 2007 at 12:11 pm)
> Python is a procedural language with OOP bolted on top; JavaScript is multi-paradigm.
Python has been OO since day one (1991). The myth that Python added classes sometime later seems to be popular in the Ruby fanboy community (as opposed to just regular Ruby users). Python/C++/Java/Ruby/Smalltalk/Blurb all do OO differently because they are different languages. There is no “One True Way” to be OO.