[First published January, 2003]

Unlike a brainstorming meeting, where the goal is to come up with new ideas, a critique meeting is focused on evaluating a set of existing ideas, and possibly identify future directions or changes. Instead of hoping informal discussions will resolve hard issues, its worth setting up  a specific critique to drive a design forward.

Goals of a design critique

A design critique involves a small group of 3-7 to discuss a set of sketches or prototypes. You could focus purely on branding elements, ease of use concepts, or even engineering feasibility, it’s up to whoever leads. The important thing is someone does lead the discussion, define what questions should be discussed and facilitate the conversation.

If there are 3 or 4 specific questions you want answered, define them. Without goals everyone will work from different assumptions, and it will be more of a brainstorm meeting than a critique.

If you are early in a project, critiques  should emphasize the higher level user, customer and business goals, and minimize the focus on specific engineering constraints or aesthetic choices. Its worth flagging ideas that engineers or business managers have large concerns about, but hold off on completely eliminating them from the discussion. There may be opportunities to ask for more resources or make other adjustments to a project, if a stellar design concept or idea is championed successfully  (e.g. perhaps a design idea exposes a new business plan that has more opportunities than the current one, and would justify a change in the project goals).

But as the project timeline progresses the tone of critique discussions should change. There should be pressure to have solutions to issues, and the bar for considering new ideas should get higher. If managed well the scope of critique discussions should peak during planning and continually decrease.

Shepherding the creative phase of a project is a significant challenge, and it’s rare to find someone than can manage it as well as the more production oriented implementation and release phases. Often there is a key leadership role for designers to play to fill this gap. Overall, the tone, content and quality of critique meetings is one indicator to how well the creative process is being managed.

Typical goals for critique meetings might include:

1. Obtain specific kinds of feedback from those in the room about a set of different design approaches for one feature or area of a website.
2. Compare how several different components of the same product are designed. (Are there elements that should be reused more? Do things that look similar behavior similarly? Etc.)
3. Discuss the user flow through a design, by examining each screen in the sequence that users would go through to complete a task. (Similar to a cognitive walkthrough).
4. Explore the designs of competing products, or designs of other products that have elements or qualities that you want to achieve.
5. Allow teammates with different job functions to provide feedback from their expertise. (QA might raising testing issues, development might ask feasibility questions, marketing might ask questions about advertising or partnerships, etc.)

These goals listed are mostly mutually exclusive. You might be able to manage two of these at the same time, if you’re a great meeting facilitator, but I wouldn’t recommend it.

Secondary goals often include:

1. Provide some structure to the creative process of a project.
2. Improve your team’s ability to think about and discuss design ideas.
3. Teach non-designers about the design critique technique, so they can apply it to other kinds of problem solving situations.

Independent of the specific critique goals: If there are questions from your teammates about your design that don’t fit your intent for the meeting, make sure you come up with some way to address them outside of the meeting. During the meeting, write them down on a whiteboard or notepad, and take them with you when you leave. The more inclusive your design thinking is, the more influence and authority you’ll have over how project decisions are made. Even if the issues you are confronted with arise from decisions out of your control (a demand from the marketing team, or a new constraint from engineering) you want your designs, and your design process, to work with these issues, not around them. (Unless you feel confident that your superior design and skills of persuasion will convince someone with authority to change their mind.)

Who is in the room

A critique should allow a small group of people to review and discuss many ideas quickly and informally. You can’t be informal and intimate about ideas with more than 5 or 6 people in the room. Instead, you must narrow down your invite list to the people most critical to the design process. Try to forget about job titles or hierarchy, and instead, focus on the people who are most likely to understand the creative process, and give useful and meaningful feedback, both positive and negative.

Depending on the personalities of your teammates, make adjustments as necessary. For anyone on your team that isn’t invited to the meeting, allow them to look at any handouts or pictures, and give you their feedback.  Or even better, make sure to forward them any of the notes you send out following the meeting. In most cases, they’ll see the quality of the dialog and kinds of discussions points that were raised, and ease up on their complaints about not being in the room. And even in the absolute worst case, make yourself available to listen to their feedback independent of the critique session. You can diffuse difficult teammates, appeasing them without derailing the critique meeting, and the creative momentum of the team.

One alternative for designers in larger organizations: you might be able to do design critiques with the other designers in your organization, even if each of you works on different projects. This can be a great way to build a sense of design community in your organization, and give you the benefit of other well trained design eyes, that are fresh to the problems you’re trying to solve. The downside to this is that you miss on the opportunity to build better design relationships with the non-designers on your team. In the best possible world, you might have time to do both kinds of critiques, at different times in your project.

Materials and rooms

Depending on the kinds of designs you’re working with, and the goals you have, you might arrange the room differently, and bring different kinds of materials.

In the simplest kind of critique, where you have several alternatives to the same design problem, make it easy for everyone to see each design approach. There are several ways to do this:

1. Print a stapled handout of the 5 or 6 pictures and give each person a copy. This works fine, unless you have prototypes for each design approach – the printouts won’t capture that :) It might be fine to have rough hand drawn sketches, if the folks in the room are capable of working with rough representations, or you might need to have more complete visual presentations of the design ideas. (I once made the mistake of showing some high powered marketing folks some hand drawn sketches: it was a disaster. Unfamiliar with design work, or design process, they naturally confused the low fidelity of the sketch, with the quality of the ideas. Learn from this :)
2. Use wall space in the room to display each of the designs. This is by far the best way to examine branding or compare/contrast different areas of the same website. If you’re new to a project, and want to illustrate how inconsistent certain design elements are, there is nothing better than putting them all up on the wall together, and asking everyone in the critique to walk the room and examine them. If you’ve never done this before, I guarantee you’ll hear a few people gasp.
3. If the room has a television, use your laptop to show designs. If you’ve made prototypes, demo them. Personally, I find this is the most convenient way to work. It usually requires little preparation beyond the prototypes themselves, and if I’m facilitating the meeting, it gives me additional control over what we discuss and how the discussion goes. Typically, what’s on the monitor is what we’re going to talk about.
4. Pick the right kind of room. I recommend small conference rooms that would accommodate 4-8 people. You want a room with lots of whiteboard space for new ideas, or for taping up printouts. Ideal is to also have a television or monitor so that everyone can easily look at the same designs from a laptop or computer.

If you are working on a long project, there is value in reusing the same room for critiques. You may be able to leave certain screenshots up on the walls ,or in the hallway outside. Plus you have the psychological advantage of identifying a single physical place with the kind of thinking and dialog you want for a critique.

The Rules of order for good critiques

Without some basics set of rules or guidelines, discussions about ideas can go in any direction. Many creative people (writers, filmmakers, artists, etc.) recognize this, and have certain shared guidelines or assumptions for how critiques should be run. Instead of starting with opinions and points of view, participants in the critique work to clarify the creators intent with the work, and only then, respond to how well the work achieves or does not achieve that intention. (e.g. – If the film director wanted you to feel angry when watching the opening scene, and you don’t feel angry when you look at it, this is useful. But telling the filmmaker you don’t like movies that make you feel angry, might not be as useful.) So when it comes to software or web design, it’s important to clarify assumptions before offering a criticism or challenging an assumption.

The general rules of order are:

1. Start with clarifying questions. Clarify any assumptions about what the presented design is intended to do, or what kind of experience it is intended to create. Hopefully, this intent is derived from the overall project goals, which is already agreed upon.
2. Listen before speaking. Many times in work environments, we confuse conversations, which should be exchanges of ideas, with opportunities to inflict our opinions on others. If you take a moment to listen and understand before voicing an opinion, you’re open to hear something new that might challenge your old thinking. So don’t just wait for other people to finish, actively try to understand what’s being said, and reflect it back to the speaker.
3. Lead into explorations of alternatives. Ask questions that surface other choices the designer might not have recognized. Postpone judgments, unless there are obvious gaps between the designers intent, and the designs you are critiquing.
4. If it fits with the goals of the critique, point out situations, sequences, or elements within the design that may be problematic given what you know about your customers, the scenarios involved, or the project goals.
5. Avoid statements that refer to absolutes. Instead, make points referent to the goals of the design. Bad example: This sucks and it’s ugly”. Good: “Well, if the goal is to make this feel friendly, black and flaming red doesn’t convey that to me.” Bad: “How could anyone figure that out?” Good: “I think there’s something missing between step 3 and 4. It’s not clear to me what the sequence of operations is.  How do you expect people to know where to click?”
6. Speak in context of your point of view. It’s fine to have a personal opinion, expressing your own preferences. But don’t confuse this with your perception of what your customers need or want. So make sure to specify which kind of opinion you’re offering. Hopefully there is data and research to help everyone agree on the likely customer perspective on different ideas.

Running the meeting

Someone should be responsible for leading and driving the meeting. This is more about facilitation than dictation. For a critique to work, everyone has to feel open about voicing their opinions and discussing ideas. It requires a different style of leadership than a status or accounting type meeting. The meeting leader, or facilitator, has to be comfortable asking quiet people to speak up, or loud or obnoxious people to quiet down.

I recommend that the creator of the designs lead the meeting. They should be confident and mature enough with the creative process to lead other people through it.

The biggest challenge to this is their ego. If the designer is leading the meeting, and controlling the discussion, there is every opportunity to push for feedback that makes their pet ideas shine, and exclude everything else. It requires maturity on the part of the designer to walk in the room with the attitude that the value of the meeting is to hear new thoughts and opinions, rather than simply to defend the ideas they already have. On a healthy team, the designer should be rewarded for the quality of their output, regardless of who may have made what suggestion, or gave birth to what initial idea, so there is little real conflict of interest. However, in the end, who runs the meeting is less important than the quality of discussion, and the overall progress of the design effort.

When a critique meeting is going well, it’s fun. It should feel like an informal conversation between people with the same goals, all trying to explore and surface good thinking. The person running the meeting has the responsibility of setting the right tone for this, preferably by example, and doing everything in their power to maintain that attitude and spirit in the room throughout the meeting. On a good team, this responsibility should come easily.

Pre and post meeting work

You may want to do some up front work to ensure that the critique goes smoothly. If you can define the goals when you call the meeting, include them in the meeting announcement. Also attach any pictures of the designs or sketches, so interested folks can get a head start in thinking about the designs.

After the critique, there is some additional work you can do to close the loop, and set yourself up for your next critique, or follow up meeting. Make sure to take notes during the meeting of key questions that were raised, or new issues and ideas that came up that you hadn’t thought about before. Send out a mail after the meeting with these details, and what the next steps in the design process will be. How long will it take before you have new designs to show? Is there a usability study that is the next milestone? How will this effort integrate with the project managers plans? If you don’t provide answers to these questions, someone else will, and you’ll default to yielding some control over the design process to them.

There are some organizations or project teams create critique forms, listing the standard questions or criteria that should be considered in a critique discussion. These are handed out during the meeting, used by participants to take notes, and then might be collected at the end. This might be more process than your team might want – so it’s up to you to figure out how formal or informal the critique discussions should be. I might recommend something like this for the first time you do it to help define this kind of meeting, but probably not as a general practice.

How often to run critiques

It depends heavily on the project and the team. If you have easy access to the people you are working with on the project, you might not need to have defined meetings for critiques. If the team is healthy, critique like conversations are probably happening in the hallway all the time. As long as you are in most of them, and people see you, the designer, as the driving force for the design effort, things might be fine. On the other hand, If you as the designer feel that most of the design conversations happen without you, intentionally or not, creating a weekly critique discussion can help put you back in the middle of the creative process.

(Note: A separate type of design meeting is a brainstorming discussion, where the dominant goal is to generate new ideas and explore as many different ideas as possible. This should not be confused with a design critique.)

Possible discussion points / questions to use

Here’s a sample list of design questions that might be of us to help guide the discussion. Again, depending on your goals for the critique, you might focus on, or avoid, some of these.

• What are the user scenarios the site is designed for? Walkthrough how each design would enable those scenarios.
• What known usability / design / business issues are these sketches trying to solve?
• What is the intended style of the design, and is it appropriate for the target audience?
• What is the intention of the style, and does it achieve the desired effect?
• Are there standard brand elements that should be used, and are they used appropriately?
• Are there similar software products or features that these designs should relate to?
• What usability heuristics does each design support well? (or not?)
• Where in the design are the most likely places for users to have trouble? and why?
• Are there reasonable design changes that might avoid these problem points?
• Does each design idea take advantage of things the user might already have learned?
• What are the pros and cons of each design idea, relative to each other?
• Are there any hybrid design ideas that are worth exploring, based on the designs in the room?
• What open issues might best be resolved by a usability study or other research?

Other references for design critiques

• Feedback without frustration: a short presentation about running great critiques
• How to give and receive criticism
• The art of web design critiques – mostly about one on one discussions about designs.
• Teamwork and Critique – A nice set of slide decks on engineering processes, including critiques.