Are design exercises in interviews unethical?
Recently my friend Jared Spool posted on twitter about why it’s wrong to give interview candidates design exercises:
Asking a candidate to perform a design exercise so you can “see how they handle the pressure” is unethical.
#DanceMonkeyDance @jmspool (#)
This split me right down the line.
First, I think most interviews are bullshit. There’s little evidence interviewing identifies better candidates than other methods do, it’s just one we are all familiar with, and it’s easy, so we do it. Any questioning of job interview practices makes me happy, since we put great faith in something we’ve never examined that is filled with bias. Some interviewers are sadists who never have to justify their cruelty since it’s never questioned.
The ideal way to hire is to use referrals and references of people you trust in your line of work to arrive at candidates. And then, to hire a candidate to do a small project for you where they can demonstrate their ability. That is the best interview in the universe since it is based on what they can actually do for you on real work, rather than an absurdly artificial interview loop of mostly cliche conversations. To do this requires more work and is less efficient, but I’m convinced the results are dramatically better.
However, assuming I’m required to interview candidates, there’s value in asking people to show their ability to do tasks I’m going to hire them to do. If they were a painter, I’d ask them to give their opinion of the paintings in my office. If they were a carpenter, I’d ask them to evaluate the furniture in the room. I’d pick something real, in the present, where their abilities and knowledge should apply, and get out of their way. “How would you make this better?” and hand them a whiteboard pen and step back. I’d want to see what questions they ask and what they know about what they need to know before they can design something. I can do this in a friendly way, and at natural points politely disagree with them, just as I would if we worked together.
The idea of “see how they handle the pressure” is a tricky one. Most work environments are not designed around designers. To be successful a designer needs to be willing and able to persuade, convince and persist in selling or defending ideas. While I’m not certain there is a reliable way to test if a person is capable of doing this during an interview, it’s my job as an interviewer to try. I have to find places to challenge and ask tough, but fair, questions, to let them demonstrate their conviction in their ideas. As well as their ability to change their minds and learn from new information.
Or put another way, if I were hiring an Air Traffic Controller, I’d need to know they can handle the stress of being an Air Traffic Controller. It wouldn’t be unethical for me to expect them to demonstrate something of what I need to see to know they are credible. While the means I use to get this information could be unethical, my interest in getting that data isn’t – it can’t be. It’d be unethical to all the passengers of all those airplanes if I hired someone without even attempting to learn about their abilities for a key part of the job.
All of this is obviously more abstract and collaborative than having them build a chair, or paint or a portrait, while I yelled at them through a megaphone while timing them with a stopwatch. That’s a game show, not a job interview. But if done right a design exercise is a chance for them to lead the conversation through a problem. And if leading the way through a problem is something expected of them in the job, it’s reasonable to provide an opportunity for them to do it during the interview process.
Related: my essay on how to use an hour to interview creatives for a job
“see how they handle the pressure”
Am I wrong in saying that if you are pressuring someone in an interview then maybe you are hiring the wrong person? If an interviewer tries to pressure me in an interview I don’t care, as they need me more then I need them. If you are hiring someone good that is usually the case because they are in demand.
Jon: The pressure isn’t “to get you to say yes to the job”. Instead it’s difficult interview questions asked under pressure. e.g. Solve this Rubic’s cube in 25 seconds while blindfolded, underwater and with rabid mice climbing up your back.
Scott, this interview trend seems to be the norm today as I’ve been asked and drilled on some odd subjects and personal subjects that seem to have no bearing on the tasks of the position at hand. I wish could remember the specifics of the questions, but I cannot.
There was a state government web marketing and social media management job that requested we write answers to timed essay questions in longhand and in pencil on unruled paper. Obviously an attempt to judge penmanship under pressure for a position that is 100% web related.
That sounds horrible.
I do think the interview process is works both ways. The process itself tells candidates a great deal about what the company is like.
There’s not even a need to increase the pressure: job interviews are already stressful. Plus, any realistic exercise is going to already be somewhat confusing when presented all-at-once.
Been doing a lot of interviews the last few months, and I always feel bad when the interviewee’s hand starts shaking. Of course, I’ve had to watch plenty of “senior developers” struggling to figure out how to iterate through a list/array… so you *have* to throw some stuff at them that will help you see where their skill level is
I feel like “handle the pressure” feels like a code phrase for an array of more specific things, mostly related to communication. It seems reasonable to want to know what’s going to happen the first time someone challenges the interviewee’s ideas in a meeting. Can he explain himself coherently? Can he adapt to feedback? or does he freeze up? Maybe get defensive? Tell us what an idiot someone would be to do something else?
I’ve seen all these behaviors at one time or another, and I admit, I’m totally willing to filter based on them. In my experience, most of these design/situational questions are less about the solution provided and more about getting a sense of how this person thinks.
That said, man, characterizing it as seeing how they handle the pressure definitely just seems mean spirited. One step removed from “We’re going to throw rocks at you while you code up a twitter client. We feel it’s the best way to simulate the work environment here.”
(Of course, that’s probably self-fulfilling. If that’s the kind of interview a place runs, that’s also probably the kind of workplace environment you’ll find there.)
It’s lovely when the job candidates really come from the recommendation or referral pool, but I’ve worked too many place where a fat layer of HR policies guarantees a lot of keyword driven random candidates. You need to do _something_ to pick out the fakers and jerks, and I’m not sure putting them through the paces is a terrible way to try.
(CAVEAT: All this assumes the interviewers are actually directly related to the job this guys coming in for – team members and such. An HR interview presenting such exercises would get nightmarishly bad quite quickly.)
If the work environment involves rocks being thrown at you, then it seems a good interview should include some rock throwing. Or at least a rock throwing simulation, or a question meant to surface your feelings about rocks and projectiles in general.
I’m not saying hostile work environments are good things – the opposite actually. I wouldn;t work anywhere that threw rocks.
But the goal of any interview has to be to attempt to measure how suitable a candidate is for work in a specific job at a specific company. If there is a lot of rock throwing, and I say “HIRE” without evaluating your ability to dodge rocks, I’m setting you, and me, up to fail once you are hired.
Hmm. That’s logical, but I would be startled at any workplace that has sufficiently clarity of vision into their own failings to approach it that way*. if that was really the motivator, then I’d expect more interviews to revolve around very stupid, self-contradictory or outright impossible requests or maybe justifying a bosses bad decisions. :)
* Well, ok, not startled. I’m sure they exist. But probably not over a certain size.
I would love to be able to work with someone on a small project for a limited period of time as a person who currently interviews candidates. As someone who has interviewed for a job, though, I would not want to leave my current job without some offer of job security at my new job. It usually takes three to six months for a new person to become familiar with our internal tools and processes to become fully effective–a single small project may not be a good measure of that person’s skills. Finally, working with a person on a small project is quite time consuming and high risk. An hour long interview with an hour long writeup is not time consuming and low risk.
In the four years at my current job, I’ve done ~100 interviews and reviewed with a group ~500-1000 candidates. I would love to hear about alternatives to structured interviews testing coding, technical skills, and communication.
Let’s think about this. The whole point of going through interviews, testing, etc, is to get an idea of how well a person will actually work in an organisation. If accurate, we reduce risk because we can predict if a person is (grossly) productive or not.
I agree that most interviews are pants, particularly within UX (even more so when the interviewer made fundamental errors) but using exercises as an alternative instead: well I think the notion is that if a person does well in an exercise then they’ll do well in real life.
But most exercises are not structured like in real life. Most exercises leave the interviewee to do it all themselves. I know we have to think for ourselves in UX but seriously, the best practitioners are always getting consensus before committing and they prefer to work with others anyway. It’s good fun to design with a stakeholder / developer hovering on my shoulder. Most projects are long and the stamina to see them through is critical. Exercises cannot test that.
My guess is that exercises are slightly better predictors but still not good predictors. As for what is – who knows?!
The added constraint that someone is watching your moves and evaluating you can make otherwise capable people crumble. Some rare interviewers do it well, they try to look into the person, they defer evaluation, staying open-minded. Other interviewers actively look for cues and therefore fail to see the person in front of them. Interviews that feel like someone is looking in a horse’s mouth are not only unethical, but inhumane. I for one have no problem walking out on an unreasonable interviewer.
I totally agree that it is important to make an interview situation as “real” as possible. that being said, it is not reality to design spontaneously within X minutes with a bunch of people in a room. I do my best work by first thinking by myself first then coming prepared into the group discussion. Unfortunately this means that I fail in interviews – which are designed to benefit those with quick snappy answers. Those whose work style is more contemplative will be eliminated with these sorts of exercises.
I have been working in senior level positions for about 5 years at startups and have recently started interviewing at very large and visible companies in SF and Silicon Valley. Most of these companies put you through a pretty heavy design test which eats up your spare time for about a week on average, not to mention multiple interviews. I’m surprised that no one has brought up the fact that these giant companies don’t even offer to pay you anything for your time. This seems unethical. Also, my friends and I who are in the same position can’t get any critical feedback to help our future efforts. In my past I nailed almost every interview I had ever done, so I know I am at a crux. I am taking away from the experiences, but to ask all of this from individuals is a bit brutal without any compensation or feedback.
I’m a designer here in the Bay and have had the same issues! Now I ask the recruiters upfront if they will be requiring some type of design brief or test that would need hours of my own work. If they do require it, I request an alternative as I’m usually employed full-time and don’t have hours to do free design work…I’m open to working side-by-side with a design director or senior staff to show my design process and work flow. If they still say no, I politely tell them that I have ethical objections to their interview process and move on to the next company.
We gotta stand together to reject this unethical exploitative practice of talent!
There seems to be a growing trend of companies requiring written exercises as part of their interview process. While in theory I don’t have a problem with asking people to show you they can do the work, the practice seems questionable at best, unethical at worst. At the final stage, being asked to do a 1- or 2-hour exercise seems fine. I’ve been asked in the past to do exercises that took anywhere from 8 to 12 hours to complete, and in both cases the company didn’t make an offer to anyone but said they “decided to move in a different direction”.
I’ve heard from friends that companies are now in the habit of posting ghost job descriptions to see what ideas they can get from candidates without ever intending to hire someone. Others have recounted their own stories of putting in the time only to find that the company used their work.
It seems to me that people have taken a reasonable interview technique and turned it into something they can profit from. So in that case, what can job hunters do to protect themselves?
Design exercises in interviews are completely unethical for several reasons.
Issue #1: Who owns the IP if you’re not hired? I can say for a fact I interviewed at a well known US retailer and was asked to design some furniture pieces. Didn’t get the job, but saw something pretty darn close to my design in stores a year later. I didn’t have any legal recourse at that point.
Issue #2: Conflict of interest and potential breach of contract with a current employer. All of my jobs have had specific non-compete and no moonlighting legal documents I had to sign when I was hired. By doing a design project “on spec” I am now willingly violating those agreements. How is that ethical?
Issue #3: Design is about taking the time to not only understand the design problem, but also understand the context in which the solution will fit. How good will a design solution really be if it does not take into consideration the business needs of the employer? That is not something easily done as a true outsider with less visibility than a contractor or consultant. Basically all that is left is judging the work on subjective things like visual design and general design process.
Issue #4: Not a level playing field. When I still worked as an Industrial Designer and not UX, I could not personally afford the high end 3D modeling and photorealistic rendering tools I used on the job. If the criteria by which a candidate is evaluated is on how the final design solution is presented, I would have been at a disadvantage compared to a candidate with the financial means to buy that software or the less virtuous candidate who simply pirated it.
Issue #5: Compensation (or lack thereof). Being asked to spend 20-40 hours on a project without compensation is downright in bad form.
By your standard it’s very hard to effectively hire someone for any kind of creative position. I’d agree only that there are definitely opportunities for abuse of trust, but that’s true in the beginning of any relationship, business or otherwise.
For #1 – Some companies define this in the NDA they ask you to sign before interviewing. Even if not – how can a creative person looking for a creative job expect to convince someone of their ability without demonstrating it in some fashion? They can of course refuse if they fear being taken advantage of, but it’s not fair to the hiring manager to take everything a candidate says on faith.
For #2 – That would be an ethical problem with the current employer, not the job interview / new employer.
For #3 – This isn’t an ethical issue – it’s that most interview processes are flawed :) It’s also the burden any hiring manager in any field has: how can I use my limited time with this candidate to evaluate their fit for this position? Arguably a good candidate will have done their homework on the employer’s context before the interview begins.
For #4: Hard to claim this is an ethical or solvable problem. The world isn’t a level playing field by-design by the universe! It’s not fair to require a college degree or a thousand other things that not everyone has equal access to. Sure, a company can do some things to make their candidate pool more fair (high school internships, scholarships, etc.) but even *those* things won’t be entirely fair either.
#5 – Sure. I suggested candidates get paid (I said ‘hire for contract’ above). But I’d think 20-40 is far more than necessary to hit the sweet spot in discovering someones abilities for most positions.
“Even if not – how can a creative person looking for a creative job expect to convince someone of their ability without demonstrating it in some fashion?”
So would you ask a surgeon to perform a surgery free of charge as part of the hiring process? A lawyer to represent you in court?
Basically I take your position as not valuing the time and effort it takes to become a design professional and only being able to gauge their talent and competency by minimizing your effort required in the process to the point of exploiting them.
As my second post said, that is what a design portfolio is for. As a design professional with 16 years of experience, it is very easy to spot the “posers” by asking probing questions about their design portfolio. You can quickly weed out the people who are only good at making pretty pictures and taking credit for other people’s work on large projects with many team members.
Regarding your answer to #2, I would respect someone who declined to participate for the reasons I laid out than to take your attitude. Doesn’t speak highly of their overall moral compass.
Several of my past employers (who were design leaders in their respective industries) took the stance of implementing contract to hire rather than assigning design projects as part of the interview process. That way you minimize the “not a good fit” risk for both parties without minimizing the value of the design field as a whole.
I respect anyone’s right not to interview in a place where they feel exploited. So while we disagree on what exploitation means, I do agree that the way a company interviews says a great deal about their ethics and culture. I think we agree that if the interview makes you uncomfortable, you shouldn’t take the job even if offered one.
A surgeon benefits in credibility from having a license and federally regulated training. Designers don’t. It’s not the best example.
An example that is also unfair, but serves my argument :) – is a chef: would you hire someone to cook at your wedding without tasting something they’d made? Samples are a large part of many creative professions. A resume, or a portfolio of food they’ve made for other people wouldn’t quite be sufficient, would it?
Clearly we disagree, which is fine. The question I’d ask is: if you were a CEO hiring a designer, how would you decide who to hire, and in an ethical fashion? That’s really the hope of the post. There are two competing needs, the needs of the candidate and the need of the hiring manager.
How do you serve both challenges?
A portfolio is, from experience, insufficient. I can’t know how much you actually did or didn’t do, or what more representative, but negative, examples you’ve chosen to omit from what you share.
More precisely a portfolio is an outcome of work. I’d need to hire someone I can work with, and collaborate with, and the interview process has to expose some of what that dynamic is like.
Thanks for the dialog. I’m enjoying it – hope you are too.
Scott,
Enjoying the conversation too. If you haven’t noticed, I care deeply about the evolution of my profession.
Don’t take this the wrong way, but you seem to come off as someone who doesn’t want to put the effort required into hiring the right people. Working relationships are just that – relationships. Wouldn’t you put more effort into finding a partner/spouse?
Regarding the chef example, I get the impression you minimize the importance of design to innovation and product development. A design solution divorced of a larger understanding of the problem and its context is honestly just a pretty picture. By assigning a project under pressure, you’re just seeing how they perform under pressure, not if they are a good designer.
To answer your question. If I were a CEO, I would rely on my network of connections as additional data points when hiring a designer and do the due diligence required to paint a good picture of a candidate. If I didn’t know what the right questions to ask during a portfolio review, I would make sure I talked with a trusted advisor to prep myself or even have them be present for the interview to give a second opinion.
The design community is not that big, this isn’t hard to do, especially if it is within a distinct sub-community such as UX or ID.
I don’t think CEO’s should hire designers. I think CEO’s should hire design leaders/directors who have worked in the industry long enough to know what type of designers they need to hire.
The issue is a lot of non-design leader types thinking they know design well enough to give ridiculous design tasks and hire people they think would be the best fit for their product. Like you said, Scott, most interviews are bullshit and a task that should be left to professionals within that specific field of practice!
Just to add a final point, this is why designers have portfolios. That is how you get a sense of how they think and work beyond reading a resume.
A good designer will explain his process and how he approaches design in his portfolio.
If a designer interviewed with me and didn’t talk about his process and just the final solution, he would not be under consideration.
I do have a problem with being set a design task in interviews.
I have had it rear it’s head twice now and both times i have been burned. My interviews go great, they like my portfolio and they admire my experience. Then they throw you out on the back of a wooly brief they’ve set and a piece of throwaway work you spent a few hours on.
A recent example for me was a brief given which was exactly this – ‘redesign one of our homepages, you can do anything you like’. So i did. And unfortunately for me i didn’t meet whatever expectations where in their mind. A total waste of time for me, for them, they dont get a candidate and i dont get the role and worst of all they don’t get to see me for what i can bring to the table in a real work environment. Lose – lose all the way.
> My interviews go great, they like my portfolio and they admire my experience
Part of the problem with all interviews is that the candidate can only self-evaluate how well they’re doing. Of course I can’t say this is true in your situation since I don’t know you and weren’t there, but it’s possible you weren’t doing as well as you thought, and therefore the design task didn’t weigh as heavily as you think.
> And unfortunately for me i didn’t meet whatever expectations
> where in their mind
This is partially their fault – they should be clear about what their expectations are for most things they have candidates do. It’s in their interest not to make it a guessing game, and least not all the time.
Where we might agree is that live design tasks, like many parts of interviews, are very easy to for the interviewer to run poorly (or even unfairly).
Scott, thanks so much for sharing your insights with this post! I’m a little late to comment, but what you’ve mentioned resonates with me and is a reflection of a positive experience I’ve had with the paid project approach.
Hiring a candidate to do a small, scoped project is a fair alternative to the typical design exercise. I had a prospective employer ask me to submit my hourly rate ahead of doing a short design sprint with their product development team to evaluate my fit for a full-time role. They accepted my rate with a freelance contract, briefed me (under NDA) on what we’d be working on (a small project in their backlog), introduced me to a product manager, developer and fellow designer to work with on the project (+ answer any questions or requests for information), and away we went.
Without a doubt, it was one of the most insightful and thoughtful interview experience I’ve encountered. As you’ve articulated, it’s more time consuming and less efficient, but they gained a better understanding of how I thought and worked (and vice versa) and came away with a possible solution to an actual problem and I was compensated for my time and effort.
I took a personal stand against these. I’ve been in the design field for over 20 years and it feels demeaning to me when interviewing to see if I can pass a little design quiz put together by some designers 2-3 years out of school.
If you love them, more power to you, but if you want to know who I am as a designer – talk to former colleagues, look at the work, ask me really specific questions, etc. no problem. But a design quiz? uh-huh. No more.
I am a human factors and UX designer in the greater Boston area and I have been interviewing to find a new position. It is really getting under my skin how many “design challenges” and “process demonstration” exercises I have been doing for companies to try to land the job. I had to put together a huge presentation for a one hour review and discussion to a team recently and they had me locked in a room with no food break for 6 hours as the whole organization came in and asked specific questions about issues they are having in this or that section of the product and how I would improve things for them. I also had 4 one hour phone screens prior to the invitation to come in to the company face to face. I kept track of the time I spent pulling the presentation materials and work together and it was over 30 hours of work. Of course they got a lot of my time and answers to their issues but I have heard nothing from them since the interview.
Another company in the Seaport District of Boston just outright stole my “design challenge” mock-ups and high fidelity concepts to enhance their product recently. They specified that I had to show them all of my steps: ideation, whiteboard work, wire frame and high res comps in the submitter for the “design challenge”.
I am at a loss, I need a job but I am really unhappy about being used in this manner to try to secure a new position. What to do…?
Not good news, but somewhat comforting to hear I have company. I’m primarily an art director and brand designer, but ventured into the UX niche for a more fruitful career (read: it pays much better).
I recently had to complete a design exercise for a medical software company, redesigning their existing interface as well as compiling it into a presentation. This is in addition to submitting the usual portfolio, an initial phone interview, and undergoing a three-hour proctored skills assessment test. All in all, it was an investment of approximately 40 hours.
Needless to say, some of these companies are getting a hell of a lot of free ideas. Let’s hope it pays off!
I was just asked to “complete a design test as part of the first step in the interview process” for an in-house Digital Art Director role for a company represented by Creative Circle. Here’s the project scope:
The relaunch of *Company name* is being kicked off with 3 new product releases (see attached documents for details). Using the attached mood board and materials, design the following conceptual assets for the brand’s relaunch:
1.) Website Homepage
⁃ Include the following elements: Navigation, Shopping Cart, Promotions, Articles and/or videos, Social, Email List Signup, etc…
2.) Website Product Detail Page for Plant Protein
3.) Direct to Consumer Email – Announcing Nature’s Best Plant Protein product
4.) 4 Display Banners Ads For Plant Protein (728×90, 300×250, 300×600, 160×600)
They asked if a week was enough time to complete the project. I told them I didn’t feel comfortable doing spec work without any sort of compensation. I’m also very willing to accept a freelance to perm situation if they’d like to test me out that way, or even if they wanted to throw me a quick freelance project.
I have 20 years of Design experience including 10+ years of digital. I find it absurd that they can’t make a decision to interview me based on my body of work, resume and LinkedIn profile. I’m not sure what taking this test is supposed to accomplish. What’s to stop me from having someone else to do the project for me?