Recently my friend Jared Spool posted on twitter about why it’s wrong to give interview candidates design exercises:
Asking a candidate to perform a design exercise so you can “see how they handle the pressure” is unethical.
#DanceMonkeyDance @jmspool (#)
This split me right down the line.
First, I think most interviews are bullshit. There’s little evidence interviewing identifies better candidates than other methods do, it’s just one we are all familiar with, and it’s easy, so we do it. Any questioning of job interview practices makes me happy, since we put great faith in something we’ve never examined that is filled with bias. Some interviewers are sadists who never have to justify their cruelty since it’s never questioned.
The ideal way to hire is to use referrals and references of people you trust in your line of work to arrive at candidates. And then, to hire a candidate to do a small project for you where they can demonstrate their ability. That is the best interview in the universe since it is based on what they can actually do for you on real work, rather than an absurdly artificial interview loop of mostly cliche conversations. To do this requires more work and is less efficient, but I’m convinced the results are dramatically better.
However, assuming I’m required to interview candidates, there’s value in asking people to show their ability to do tasks I’m going to hire them to do. If they were a painter, I’d ask them to give their opinion of the paintings in my office. If they were a carpenter, I’d ask them to evaluate the furniture in the room. I’d pick something real, in the present, where their abilities and knowledge should apply, and get out of their way. “How would you make this better?” and hand them a whiteboard pen and step back. I’d want to see what questions they ask and what they know about what they need to know before they can design something. I can do this in a friendly way, and at natural points politely disagree with them, just as I would if we worked together.
The idea of “see how they handle the pressure” is a tricky one. Most work environments are not designed around designers. To be successful a designer needs to be willing and able to persuade, convince and persist in selling or defending ideas. While I’m not certain there is a reliable way to test if a person is capable of doing this during an interview, it’s my job as an interviewer to try. I have to find places to challenge and ask tough, but fair, questions, to let them demonstrate their conviction in their ideas. As well as their ability to change their minds and learn from new information.
Or put another way, if I were hiring an Air Traffic Controller, I’d need to know they can handle the stress of being an Air Traffic Controller. It wouldn’t be unethical for me to expect them to demonstrate something of what I need to see to know they are credible. While the means I use to get this information could be unethical, my interest in getting that data isn’t – it can’t be. It’d be unethical to all the passengers of all those airplanes if I hired someone without even attempting to learn about their abilities for a key part of the job.
All of this is obviously more abstract and collaborative than having them build a chair, or paint or a portrait, while I yelled at them through a megaphone while timing them with a stopwatch. That’s a game show, not a job interview. But if done right a design exercise is a chance for them to lead the conversation through a problem. And if leading the way through a problem is something expected of them in the job, it’s reasonable to provide an opportunity for them to do it during the interview process.
Related: my essay on how to use an hour to interview creatives for a job
“see how they handle the pressure”
Am I wrong in saying that if you are pressuring someone in an interview then maybe you are hiring the wrong person? If an interviewer tries to pressure me in an interview I don’t care, as they need me more then I need them. If you are hiring someone good that is usually the case because they are in demand.
Jon: The pressure isn’t “to get you to say yes to the job”. Instead it’s difficult interview questions asked under pressure. e.g. Solve this Rubic’s cube in 25 seconds while blindfolded, underwater and with rabid mice climbing up your back.
Scott, this interview trend seems to be the norm today as I’ve been asked and drilled on some odd subjects and personal subjects that seem to have no bearing on the tasks of the position at hand. I wish could remember the specifics of the questions, but I cannot.
There was a state government web marketing and social media management job that requested we write answers to timed essay questions in longhand and in pencil on unruled paper. Obviously an attempt to judge penmanship under pressure for a position that is 100% web related.
That sounds horrible.
I do think the interview process is works both ways. The process itself tells candidates a great deal about what the company is like.
There’s not even a need to increase the pressure: job interviews are already stressful. Plus, any realistic exercise is going to already be somewhat confusing when presented all-at-once.
Been doing a lot of interviews the last few months, and I always feel bad when the interviewee’s hand starts shaking. Of course, I’ve had to watch plenty of “senior developers” struggling to figure out how to iterate through a list/array… so you *have* to throw some stuff at them that will help you see where their skill level is
I feel like “handle the pressure” feels like a code phrase for an array of more specific things, mostly related to communication. It seems reasonable to want to know what’s going to happen the first time someone challenges the interviewee’s ideas in a meeting. Can he explain himself coherently? Can he adapt to feedback? or does he freeze up? Maybe get defensive? Tell us what an idiot someone would be to do something else?
I’ve seen all these behaviors at one time or another, and I admit, I’m totally willing to filter based on them. In my experience, most of these design/situational questions are less about the solution provided and more about getting a sense of how this person thinks.
That said, man, characterizing it as seeing how they handle the pressure definitely just seems mean spirited. One step removed from “We’re going to throw rocks at you while you code up a twitter client. We feel it’s the best way to simulate the work environment here.”
(Of course, that’s probably self-fulfilling. If that’s the kind of interview a place runs, that’s also probably the kind of workplace environment you’ll find there.)
It’s lovely when the job candidates really come from the recommendation or referral pool, but I’ve worked too many place where a fat layer of HR policies guarantees a lot of keyword driven random candidates. You need to do _something_ to pick out the fakers and jerks, and I’m not sure putting them through the paces is a terrible way to try.
(CAVEAT: All this assumes the interviewers are actually directly related to the job this guys coming in for – team members and such. An HR interview presenting such exercises would get nightmarishly bad quite quickly.)
If the work environment involves rocks being thrown at you, then it seems a good interview should include some rock throwing. Or at least a rock throwing simulation, or a question meant to surface your feelings about rocks and projectiles in general.
I’m not saying hostile work environments are good things – the opposite actually. I wouldn;t work anywhere that threw rocks.
But the goal of any interview has to be to attempt to measure how suitable a candidate is for work in a specific job at a specific company. If there is a lot of rock throwing, and I say “HIRE” without evaluating your ability to dodge rocks, I’m setting you, and me, up to fail once you are hired.
Hmm. That’s logical, but I would be startled at any workplace that has sufficiently clarity of vision into their own failings to approach it that way*. if that was really the motivator, then I’d expect more interviews to revolve around very stupid, self-contradictory or outright impossible requests or maybe justifying a bosses bad decisions. :)
* Well, ok, not startled. I’m sure they exist. But probably not over a certain size.
I would love to be able to work with someone on a small project for a limited period of time as a person who currently interviews candidates. As someone who has interviewed for a job, though, I would not want to leave my current job without some offer of job security at my new job. It usually takes three to six months for a new person to become familiar with our internal tools and processes to become fully effective–a single small project may not be a good measure of that person’s skills. Finally, working with a person on a small project is quite time consuming and high risk. An hour long interview with an hour long writeup is not time consuming and low risk.
In the four years at my current job, I’ve done ~100 interviews and reviewed with a group ~500-1000 candidates. I would love to hear about alternatives to structured interviews testing coding, technical skills, and communication.
Let’s think about this. The whole point of going through interviews, testing, etc, is to get an idea of how well a person will actually work in an organisation. If accurate, we reduce risk because we can predict if a person is (grossly) productive or not.
I agree that most interviews are pants, particularly within UX (even more so when the interviewer made fundamental errors) but using exercises as an alternative instead: well I think the notion is that if a person does well in an exercise then they’ll do well in real life.
But most exercises are not structured like in real life. Most exercises leave the interviewee to do it all themselves. I know we have to think for ourselves in UX but seriously, the best practitioners are always getting consensus before committing and they prefer to work with others anyway. It’s good fun to design with a stakeholder / developer hovering on my shoulder. Most projects are long and the stamina to see them through is critical. Exercises cannot test that.
My guess is that exercises are slightly better predictors but still not good predictors. As for what is – who knows?!
The added constraint that someone is watching your moves and evaluating you can make otherwise capable people crumble. Some rare interviewers do it well, they try to look into the person, they defer evaluation, staying open-minded. Other interviewers actively look for cues and therefore fail to see the person in front of them. Interviews that feel like someone is looking in a horse’s mouth are not only unethical, but inhumane. I for one have no problem walking out on an unreasonable interviewer.
I totally agree that it is important to make an interview situation as “real” as possible. that being said, it is not reality to design spontaneously within X minutes with a bunch of people in a room. I do my best work by first thinking by myself first then coming prepared into the group discussion. Unfortunately this means that I fail in interviews – which are designed to benefit those with quick snappy answers. Those whose work style is more contemplative will be eliminated with these sorts of exercises.