First, some background: I highly recommend that you read the following two blog posts: by Ashe Dryden: The Ethics of Unpaid Labor and the OSS Community and by James Coglan: Why Github is not your CV. They make some fantastic points and communicate the issues surrounding “Using Github as your CV”.
Both of these were largely provoked by this tweet by Chris Anderson:
Advice received on hiring software devs: reject anyone who doesn't have a @GitHub profile (the more active the better). Agree?
— Chris Anderson (@chr1sa) October 30, 2013
I had a similar, extremely popular, tweet from 2011:
When it comes to hiring, I'll take a Github commit log over a resume any day.
— John Resig (@jeresig) February 5, 2011
I wanted to take this chance to clarify my position on using Github during the interview process a bit.
To start, the position being posited by Chris is absolutely wrong. No one should ever require a Github profile in favor of a resume (unless you’re applying for a job at Github, perhaps?). Nor should anyone prefer candidates who provide a Github profile over those who don’t.
(And to clarify, some people seem to be getting caught up in the terminology – I see many people using ‘Github’ as a placeholder for ‘any OSS activity’, I hope no one is being that literal.)
When you’re applying for a job it’s generally advised that the more high-quality information that you provide, the better. Anything that emphasizes your work habits, the quality of work that you will perform, and your interests is ideal. Thus the more targeted information you’re able to provide, the better. Github can provide this for some but almost certainly can’t be the final arbiter of this information for all.
We’ve had dev applicants at Khan Academy who’ve applied with traditional resumes, resume + Github profile, resume + personal web site, resume + links to projects they’ve worked on, resume + a custom project made special for the interview.
Out of all of these submissions the ones that’ve had the most impact are the ones where the candidates have gone above-and-beyond and done some custom coding, made a custom web site, or contributed to one of our Open Source projects as part of the application process. A popular project is one where candidates make a custom exercise using our exercise framework.
There are a few reasons why this particular approach is so effective:
- It communicates just how interested the candidate is in the job: That they’re willing to go above-and-beyond to demonstrate their interest.
- They show that the sort of things they’re interested in working on are largely aligned with what we’re trying to do (especially if they make an exercise with our exercise framework).
- If it’s a coding project you can get a sense for what their code quality is like and how they wish to portray themselves.
Naturally this isn’t completely ideal as it requires a candidate to put in considerably more time into their application than just generating a single-sheet resume. That being said the level of information that is conveyed is so much higher that it almost isn’t comparable.
The traditional interview process is broken. It’s high-stress and a poor representation of what you’ll actually experience working for the employer, or working with the candidate.
I extracted a tangible, contained, project that I had assigned to one of my interns and turned it into a project. Thus far it’s worked extremely well as, as can be attested to by Pamela Fox (who completed the project with flying colors). I’m very pleased with this approach as it gets all the benefit of the candidate creating a custom project on their own but it has the added benefit of communicating to the candidate the exact type of projects that they’ll be working on here at Khan Academy.
I’d love to open up the project at some point to show how it’s done but it’s still an effective tool in our interviewing toolbox and I’m not sure I want to lose that just yet!
However, you can be privy to the project-based interview process if you wish to apply to Khan Academy! Specifically if you’re interested in working with Pamela Fox and me to build Khan Academy’s Computer Science platform (more details on the launch, a technical talk I gave on it after launch).
The biggest drawback to this approach is that we’re demanding some of your free time to gather this information. We’re not sure what the best way is to offset this, as of yet. However by doing the project you’ll likely be able to skip a number of stages in our interview process and hopefully get started on changing the world with us at Khan Academy all the sooner.
David Parker (November 21, 2013 at 1:42 pm)
Great post John.
We also do project-based interviews. We do this in lieu of both Github and more importantly, technical interviews. I’d rather see that you can get the job done than theoretical answers to questions that don’t matter.
As to compensation, we offer to pay our interviewees for their time. I’m sure that Khan Academy has enough money, and few enough interviewees that make it to the project phase you could offer to pay a little money? Or am I wildly off on the number of interviewees?
John Resig (November 21, 2013 at 1:45 pm)
@David: Good point, we’ve considered paying the interviewees as well although we haven’t implemented that, yet. We do get a lot of interviewees so it would be a substantial amount of money. It’s tricky as we’re mostly trying to figure out how to scale this up in a reasonable way (especially since it has the potential to simplify the interview process for us to a large degree).
Timmy Willison (November 21, 2013 at 2:21 pm)
This is how I was hired for my first developer job. I don’t think I would have been able to demonstrate my commitment or capabilities had the company not asked me to build something. I am very thankful.
exim (November 21, 2013 at 2:56 pm)
Why the “Authorization to work in the US” requirement?
You might be afraid of word-wide remote employment (so far), but are you really unable to sponsor a work permit? Besides H1B’s, there are other workaround visas.
Michael Pryor (November 21, 2013 at 3:22 pm)
“The biggest drawback to this approach is that we’re demanding some of your free time to gather this information”
You’d be demanding some of their free time to come to an in-person interview (or coding screen) as well, and it would be a lot less obvious to you if it was a good fit. You shouldn’t look at this as a drawback. This is an efficiency.
Paul Betts (November 21, 2013 at 3:36 pm)
Even GitHub would (and has!) hired someone who doesn’t have a GitHub profile (or has one, but it’s not particularly active), though it certainly is a point in your favor if you have code we can read.
Nathan (November 21, 2013 at 3:40 pm)
I thought I’d mention that I’ve been using a take-home project-based interview approach for years now. I discovered that (especially around here) – there’s a lot of people that think they know a lot more than they do.
Giving them a small project really lets me evaluate someone on their actual skills, and also see what areas they may need support in if they get the job. Generally, we weed people out with a face-to-face chat (just to get a feel for them) before giving them the project – and we pay them for their time too. Having a chat first really helps to weed out a lot of time-wasters. There’s no fixed interview questions – we just get a feel for them and talk about stuff they’ve done. It tends to make the list of people that we give the project to quite small and keeps the process quite affordable.
In general, I find the small fee for a few hours of contractor work is more than fair when you look at it in the context of what it can cost to find a good employee.
Mike (November 21, 2013 at 4:02 pm)
I just went through months of interviews with development companies. I finally accepted an offer, but the months-long trials gave me some insight.
The most difficult problems I encountered were:
1) No personal communication at all. The application process required that I re-design my resume for some custom system – sometimes it was a third-party platform, sometimes custom – either way it was a complete waste of my time because it would take hours a day to really do a good job. Most of the time, I’d get an email back about my qualifications not matching, even in cases where my experience is easily far beyond what they’re looking for.
2) Too many companies refused to look at my GitHub or other work. There’s suddenly a trend where everyone refuses to look at my portfolio because they’ve been “scammed” too many times. They refuse to look at my code on github because they don’t trust I wrote it. I was blown away by these people…
4) Some companies have *very* poorly designed tests. One company had a live code review session with me but they were grading me down on things that were personal preference – asking me what’s wrong with a piece of code and the answer turned out to be the spacing, tabs vs spaces, etc. I’ve even seen questions designed to trick the applicant, looking for people who know some inane technicality that most people get caught up on.
5) It’s fine to do a project as part of the application. However, when you’re applying to hundreds of jobs in race to support your family, you can’t possibly spend a day doing unpaid work for every company that asks. I had to turn down dozens of crazy free-work requests, it’s just spam.
The company I finally accepted an offer from had a single phone call after my application, and then a simple mockup project (with the requirement that no existing libraries be used). That actually sounded fun to do. I spent a day working on it and impressed everyone with the work. They also looked through all of my portfolio and GitHub work.
Maybe 5% of all companies I applied to handled the application process in a way I’d call “humane”.
Engineer (November 21, 2013 at 4:23 pm)
This is an excellent idea.
I’ve got a project that I’ll have you implement as well, so I have an idea of what kind of team you’ve got going.
This way it’s mutual too- you give up as much time as I give up.
Much better than the limited 2 minutes of questions candidates get in traditional interviews (always answered vaguely.)
Show me a company that’s as interested in jumping thru hoops as they are in making me jump thru hoops, and I’ll show you a company that’s not desperate for great engineers.
Part of how the process is broken is how completely one sided it is.
Nathaniel Eliot (November 21, 2013 at 6:43 pm)
Infochimps did a small project as the last portion of the interview process. It was roughly a week or two worth of real (but low priority) work, and was paid accordingly. One of the best interview techniques I’ve ever encountered, and not just because I got a job through it.
William Estoque (November 22, 2013 at 7:07 pm)
Excellent post! Just basing a candidate on his Github profile for a job interview is really one-sided. I’ve seen/worked with good developers that have pretty much not done any open source at all.
On how I do things, I usually give candidates a take home project that usually contains 2 exercises with it a local git repo so I can see progress on how they eventually came up with the solution. It also mimics the “real world” process of the job so you get most of the things you need to know.
Rich D (November 22, 2013 at 8:58 pm)
In this scenario you have many engineers are competing for a position. Sometimes the situation is reversed, and companies compete for an engineer. In that situation the companies will be the ‘interviewees’ and spend time competing for the engineer’s limited time.
Similar to sports, some athletes need to tryout to make the team, and some athletes have teams fighting over them. It’s not always desperation, just the nature of competition.
Annmaria (November 23, 2013 at 3:06 pm)
Some people spend years working on proprietary or classified projects. For me, putting my side projects on github isn’t a priority although it’s been on my to-do list for a long time. When I hire people, I ask for some code samples and for them to explain to me why they made the coding decisions they did.
Nick (November 26, 2013 at 1:37 am)
Just curious – is this option also open to students applying for internships?
Bill White (November 27, 2013 at 10:09 am)
I’ve been given take home projects for interviews. I’ve not found them every very satisfying. The interviewers can’t reasonably expect the interviewee to spend more than a day on the project, and it’s hard to show anything substantive on a project that small. The ones I’ve been asked to do were essentially trivial. I’ve found that asking some simple programming question separates the sheep from the goats pretty well. Asking: “Write a C definition of a linked list, and a function to print one backwards.” seems to be very appropriate. Asking “Write a C function which inspects each member of a sized array of integers and returns true if all the integers are equal to 6.” seems to separate about 50% of the people I’ve interviewed in the past.
I have never worked for an employer who would let me work on any open source project. All the companys I have worked for have used open source pretty heavily, but I have always been enjoined from participating in any projects by non-compete clauses and proprietary information confidentiality clauses. I suspect most people are like that.
Anon Coder (December 4, 2013 at 9:26 am)
One question about project work: what about non-competes.
Since it’s not uncommon to move to a competitor of a current employer, you would be essentially working for two competing companies, which is likely a breach of an employment contract, as well as a very bad idea in general.