I have another handy method, that I recently developed, that allows you to simply remove an item – or a group of items – from an array. Like with my implementation of JavaScript Method Overloading I wanted something that was concise, elegant, speedy, and highly effective.
So here’s the method that I came up with:
// Array Remove - By John Resig (MIT Licensed) Array.prototype.remove = function(from, to) { var rest = this.slice((to || from) + 1 || this.length); this.length = from < 0 ? this.length + from : from; return this.push.apply(this, rest); };[/js] and here's some examples of how it could be used: [js]// Remove the second item from the array array.remove(1); // Remove the second-to-last item from the array array.remove(-2); // Remove the second and third items from the array array.remove(1,2); // Remove the last and second-to-last items from the array array.remove(-2,-1);[/js] I extend that native Array prototype, if you don't want to extend a global object, you can do something like the following, instead: [js]// Array Remove - By John Resig (MIT Licensed) Array.remove = function(array, from, to) { var rest = array.slice((to || from) + 1 || array.length); array.length = from < 0 ? array.length + from : from; return array.push.apply(array, rest); };[/js] Here's a couple goals that I had for the method: <ul> <li>It had to add an extra method to an array object that would allow me to remove an item by index (e.g. array.remove(1) to remove the second item).</li> <li>I wanted to be able to remove items by negative index (e.g. array.remove(-1) to remove the last item in the array).</li> <li>I wanted to be able to remove a group of items by index, and negative index (e.g. array.remove(0,2) to remove the first three items and array.remove(-2,-1) to remove the last two items).</li> <li>It had to be destructive (modifying the original array).</li> <li>It had to behave like other destructive array methods (returning the new array length - like how push and unshift work).</li> </ul> The above code may appear to be simple, due to its trivial length, but once again, in order to make it happen, I made good use of some lesser-known JavaScript features that really need to be explained. To start with, most "remove" methods that you'll find on the Internet end up making use of two slice operations (and a concat) in order to compose the final result, like so: [js]array = array.slice(0,i).concat( array.slice(i+1) );
(The above composes all of the items before the item that’s to be removed together with all the items after it into a single array.) This can end up becoming quite costly as slice and concat operations aren’t terribly fast. We can circumvent this by doing a single slice operation and doing a push instead. This is where the trickiness comes in.
Now, we know that the front of the array is never going to change (even if the “front” contains no elements, there’s no need for us to do an additional slice operation). Thus, making that assumption, we can do this trick:
array.length = i - 1; array.push.apply( array, slicedResults );
Here’s what that does:
- Modifying the length of an array effectively removes all results from the end of it. If length or elegance isn’t an issue for you, you could modify the above method to check for calls like .remove(-2) and optimize it to be a single operation: this.length += from;
- Thus, since we’ve already removed all the dangling items from the array (along with the items that we wanted to remove, to begin with) we need to merge the sliced items back onto the original array.
- An array’s push method is capable of taking any number of arguments (meaning statements like this are valid: array.push(1,2,3,4)). Thus, if we call that method using apply and put in the sliced array all of the resulting items will be “concatenated” onto the end of the array, in one, single super-fast operation.
Fun Fact: I use this technique to construct the array-like results of DOM elements in a jQuery object.
The only other tricky bit of the function is the following snippet:
(to || from) + 1 || this.length
“(to || from) + 1” is, effectively, saying “start the slice just after where the remove operation ended”. Thus, if you did .remove(1,2) we’d need to start the slice at the index of 3. However, if we just did .remove(1) we’d start the slice at 2. We get to cheat a little bit since a to of 0 is irrelevant and can be ignored.
Now, the second part, “|| this.length” is just a tricky way of saying “if the end index was -1, make sure that we don’t start the slice operation at 0, and instead start it after the end of the array”. This issue arrives because doing .slice(0) returns the full array. Thus, in order to simulate our intended result (an empty array) we just pass in an index that’ll always return an empty result (this.length).
This is one thing that I really like about JavaScript: You can write a three line function for a trivial operation and still need 800 words to explain its true nature. Comments and feedback are appreciated.
Mike Fitzgerald (December 3, 2007 at 3:14 am)
What is it about concat operations that make them slower than push.apply operations? Is there an additional feature of a concat that makes push.apply an unacceptable substitute in other cases? Is it purely that it’s a non-destructive operation, and the creation of a new array slows it down?
Just curious.
Brendan Eich (December 3, 2007 at 3:28 am)
Hey John, I guess you passed on splice because it always creates a new array for the “spliced out” elements, which it returns. For ES4, I had a place-holder method, set_slice, but abandoned mutating slice syntax (i.e., a[from:to:step] on the left-hand side of assignment); set_slice would have taken slice-like args, as your remove does.
/be
John Resig (December 3, 2007 at 3:29 am)
@Mike: That’s, pretty much, exactly it. The cost of creating a new array is tremendous (in some crude tests that I just ran, .concat() was at least as twice as slow as using .push() – probably more). It’s interesting because we couldn’t even have used concat to create a destructive version of our .remove() method, even if we wanted to – we would have to use some form or .push() anyway. (And, based upon more code that I see online, that would involve looping through all the results and pushing each individual item onto the existing array.)
@Brendan: Interesting – is that syntax part of Python’s slicing stuff or was this defined independently? As far as slice vs. splice – slice just seemed like the easier method to be interacting with, and after running some quick speed tests, it seemed to be, more consistently faster.
Boris (December 3, 2007 at 3:31 am)
Mike, concat has to do the following:
1) Create a new array
2) Copy all the items from the first array into the new array
3) Copy all the items from the second array into the new array
Copying items involves defining the corresponding properties ([k] for various k), so it’s O(N) in the number of items being copied.
In other words, if concat() is used then the remove operation described would be O(N) in the length of the array even if you remove somewhere near the end. That’s not counting the object creation cost itself, which it not that trivial.
Using push() means you only need to copy the tail end, and don’t need the new array.
You can compare the actual implementations of push() and concat() used in Gecko in this case:
http://bonsai.mozilla.org/cvsblame.cgi?file=mozilla/js/src/jsarray.c&rev=3.131&mark=1332-1350#1332
and
http://bonsai.mozilla.org/cvsblame.cgi?file=mozilla/js/src/jsarray.c&rev=3.131&mark=1625-1704#1625
Jeff Walden (December 3, 2007 at 6:35 am)
Zeroth, do you know that copying your code sucks because it’s a numbered list and gets prefixed with “#” in Firefox? Makes it annoying to copy the samples.
First, I’m going to have to quibble philosophy over “concise, elegant, speedy, and highly effective”. Readability plus conciseness is the win — shouldn’t sacrifice the former. Elegance is nice for the version you show off in a blog post ;-) but irrelevant to what you actually use. Speedy, for something operating on probably-small arrays, isn’t huge as long as it’s linear with constant factor 1 (so avoid extra copying). “Highly effective” I’ll interpret as being correct. (I must, however, strongly disagree with an inclusive upper bound — it took me a long time to notice that was a requirement, and I think it’s completely non-intuitive. :-) )
Second, slice and push versus splice: the two optimize different things. The former copies the tail of the array, the latter copies the elided part of the array — you can pick benchmarks where either wins. In practice, though, I suspect you’re not usually removing more elements than are in the tail, so I think splice would win in general use. Now cost to execute the operations comes into play, so two function calls versus one, max three property-gets versus five, no property-sets versus max one, no local-sets versus one, plus number of bytecodes or whatever executed. Yours is 70 instructions latest SpiderMonkey for me, the following is 64 for me. No idea which of us wins on instruction count — we both short-circuit some. Mine wins on local slot count but probably at the cost of a larger stack depth. I think mine’s clearer (once you grok the different-signs check for dealing with the more difficult arguments cases) because there’s only one function’s arguments to remember. Elegance, speed, I think I might win on the number of gets/sets/calls here. Correct, yes (but with the inclusive-to botch, natch).
Array.prototype.remove = function(from, to){
this.splice(from,
!to ||
1 + to - from + (!(to < 0 ^ from >= 0) && (to < 0 || -1) * this.length));
return this.length;
};
In the end I’m not going to bother actually benchmarking the two; it’ll be as much a matter of workload as anything, I bet, with mostly irrelevant differences for comparisons where tail-size and elided-size in the two are set appropriately.
For anyone else who feels like playing the numbers game or writing another implementation, here’s a test function:
function jtest(){
var c = [0, 1, 2, 3, 4, 5, 6, 7, 8, 9];
var ex = "1,2,3,4,5,6,7,8,9";
if (c.remove(0) != 9)
throw "FAIL 1: expect " + ex + " got " + c;
ex = "3,4,5,6,7,8,9";
if (c.remove(0, 1) != 7)
throw "FAIL 2: expect " + ex + " got " + c;
ex = "3,4,5,6,7";
if (c.remove(-2, -1) != 5)
throw "FAIL 3: expect " + ex + " got " + c;
ex = "3,4,5";
if (c.remove(-2, 4) != 3)
throw "FAIL 4: expect " + ex + " got " + c;
ex = "3";
if (c.remove(1, -1) != 1)
throw "FAIL 5: expect " + ex + " got " + c;
print("PASSED");
}
Also, since a precedent’s been set, yadda yadda, public domain, do whatever you want with either function in this comment but be nice and attribute somehow, etc.
Stuart (December 3, 2007 at 8:15 am)
Why did you choose to make “to” inclusive rather than exclusive? I’ve used Java more than JavaScript but I thought they used the same conventions – “from” inclusive, “to” exclusive, so that “to – from” is the length of the affected segment. That always seemed to make the most sense to me (and I appreciate it even more since working in C# which *doesn’t* use it). Does JavaScript use a different convention?
John Resig (December 3, 2007 at 11:31 am)
@Jeff: As far as the # before lines of code goes – I think that’s a by-product of the code formatting extension that I use. I’ll try tinkering around with it to see if I can fix it up.
But yeah, I see now what Brendan was alluding to – your code is definitely faster (in the basic tests that I’ve run) and decidedly shorter. Now I’m trying to remember back and figure out exactly why I wanted to not use splice in the first place and I really can’t remember. Oh well; thanks for making it, I’ll probably start using it myself.
@Jeff & Stuart: My reasoning for making it inclusive was so that you could actually remove items from the end of the array without having to reference its length. For example: a.remove(-2,-1) if it was exclusive you’d have to do: a.remove(-1,a.length) which hardly seems nice. Although, convention may rule the day in this aspect. Thankfully, it’s not that hard of a change.
Brendan Eich (December 3, 2007 at 1:13 pm)
Stuart makes a good point I passed over: Pythonic ranges are half-open intervals: [start, end) in mathematical notation. This wins because you can tile the integer line with them: [a, b) abuts [b, c) for a < b < c. The “end” index is optional, so it defaults to s.length when slicing: s[i:] is short for s[i:s.length].
If you use fully closed ranges a lot, you find yourself adding 1 too much, or forgetting and suffering off-by-one (“fencepost) errors. Python gets this right and JS2 is following it.
/be
Dustin Diaz (December 4, 2007 at 1:19 am)
Clever John. I think the most interesting part is the comment that says it’s under MIT license. I haven’t really seen that done before given that it’s just a remove function. Nevertheless, I like these little tid-bits of JavaScript goodies. It’s actually surprising that something like this isn’t included in Prototype (or better yet, jQuery). Although, yeah, this does some more like a Prototype thing.
Karellen (December 4, 2007 at 3:14 am)
I’m with Stuart & be, range should be [start, end)
You could solve your problem by allowing a.remove(-1, 0) to remove the last item from the array, or a.remove(-2, 0) to remove the last 2 items. And, the difference between “start” and “end” is 2, so you know you’re removing 2 items.
Walter Vargas (December 4, 2007 at 3:25 am)
Good idea, I wrote a mortal code, because I am new with Javascript, that eliminates the array elements:
var len = serialesArray.push(1 );
for (var i = 0; i
Neil (December 4, 2007 at 7:36 am)
So, when can we expect the insert function? (Currently I write x = x.splice(0, n).concat(y, x) but that’s obviously a specific pattern.)
Jeff Walden (December 4, 2007 at 5:26 pm)
Neil: insert is just a special case of splice, whose signature is:
function(index, count, elements...)
Supply the index, make count 0, and then give your elements. Sadly if you have a variable number you have to duplicate the array of elements, but you can’t do much about that given the current signature for
Function.prototype.apply
.Andrea Giammarchi (December 5, 2007 at 1:46 pm)
Hi John, nice stuff here!
I wonder if this one should be even faster and more “memory safe”:
Array.prototype.remove = function(from, to) {
this.splice(from, from < 0 ? this.length + (to || from || 1) : to || from || 1);
return this.length;
};
I just through your examples and at least these seems to work perfectly.
The goal? Try to obtain the same result without forcing array length.
What do you think?
John Resig (December 5, 2007 at 3:15 pm)
@Andrea: That looks good, looks like a simpler version of Jeff’s code. Making a minor tweak would get you:
Array.prototype.remove = function(from, to) {
this.splice(from, (to || from || 1) + (from < 0 ? this.length : 0));
return this.length;
};
Which is a little bit simpler.
Jeff Walden (December 7, 2007 at 4:40 am)
Andrea: your code doesn’t pass the FAIL 2 case in the
jtest
function I gave above (same goes for John’s tweak, of course).Andrea Giammarchi (December 14, 2007 at 6:36 am)
You’re right Jeff, the strange thing is that this one seems to be ok with your jstest but FireFox shows dialog to print something :D
Array.prototype.remove = function(from, to){
this.splice(from, (to=[0,from||1,++to-from][arguments.length])<0?this.length+to:to);
return this.length;
};
Quite hilarius, any problem with Opera and Safari … but I’m not testing so much :-)
stauren (January 9, 2008 at 4:26 am)
This is a Chinese translation introduction of your elegant function, so smart!
http://stauren.net/log/dscrqea81.html
Machete (February 18, 2008 at 4:13 pm)
Hello, found this while googling for how to remove items from Arrays. Very good discussion. Two things:
– In testing Andrea’s method, it appeared to remove x items if I neglected to provide the ‘to’ argument. x being the value of the ‘from’ argument. The classic efficiency vs. flexibility debate of course. Speaking as a real world developer, I’d prefer the ‘to’ argument being optional but the argument can be made that it lends to ambiguity. I therefore made two methods, one for removing ranges the other for removing single entries. removeRange, removeAt.
– Thanks for the snippets and the discussion. Very informative.