A guide to interactionary – how to run your own

By Scott Berkun, March 2005

If you are considering putting an Interactionary together, this is the page for you. It covers the considerations for planning, organizing, and running an Interactionary. Enough detail is provided here to recreate what we did at CHI 2000 / 2001. For summaries of what we did there, see Interactionary 2000 and Interactionary 2001.

Create a small team to help you organize

The event at CHI had four people involved in planning and running the session. We also had help from the CHI session chairs, and the technical staff for the conference in Amsterdam and Seattle. In a more informal setting, fewer people are needed, but it’s more than a one person job.

Among the four organizers, we split the work in the following way:

• MC/host: someone has to introduce the session, and act as the MC for all of the transitions. They also manage the panelists, and explains anything that needs to be explained to the audience.
• Person mover: Someone has to keep an eye on the teams that are waiting off stage, and make sure that they don’t come out too early. To keep the session fair, waiting teams have to be in a soundproof room.
• Scoring: The judges scores have to be tallied and posted to the audience. The MC is too busy working the audience, so a dedicated person to collate and tabulate makes sense.
• Production: depending on how large a stage you have, and if video cameras are involved, someone needs to manage the AV folks, and make sure that they understand what’s going to happen. In a small setting with less than 100 audience members, this isn’t a big deal. But anything larger probably requires some extra thought.
• Planning: everyone participated in this. This included figuring out the timings, the problems, and style for the session. Picking the judges, teams, writing abstracts, etc.

Even if you run and organize the event alone, it’s worthwhile to consider how you will manage your time to cover these different tasks.

Selecting teams

When we sent out a call for participation to several web design and usability related distribution lists – we received more than 20 responses. This gave us some flexibility in selecting teams.

You should evaluate teams based on their experience in design, as well as your expectation for their stage presence. Since they’ll be up on stage, a team that is shy, quiet, or overly self conscious won’t convey much to the audience. It’s a limitation of the format, but great designers that are uncomfortable with being on stage will not do well here. We found that teams that were willing to express a sense of humor or personality in their submissions to us, were more likely to thrive in the format than those that didn’t.

If you can, balance the types of teams that participate. Have teams that come from different organizations, or that work on different kinds of design. Interactionary works because of the contrasts, so try to represent different design or usability philosophies if you can.

Building the jury / panel

The panel is the primary mechanism for stability. If you have good panelists, they can bring value to teams that struggle, or that fail to get very far towards coming up with solutions.

Balance your choice of panelists based on a combination of their knowledge and experience in interaction design or usability, with their abilities to be expressive and personable in an extemporaneous context. They won’t have time to prepare thoughts or responses, and will have to interpret for the audience what they think has happened. Again, it is useful to consider balancing the group. The more diverse the perspectives, the more context the audience will have for thinking about what they saw.

Scoring

We picked four categories, and allowed the judges fairly wide latitude in interpreting their meaning. Each category was worth 10 points. For 2001 we eliminated one category, but otherwise used the same structure. The scoring is the framework that allows the audience to follow along, even if they know nothing about design. It’s important to emphasize in the introduction that the scoring is not the primary goal – it’s just the framework that allows them to witness design.

• Teamwork: How does the team organize? Do delegate tasks, or share the work to be more effective? How do they communicate? How effectively are the skills and size of the team used?
• Process: What approach is taken to the problem, and the time resources? How many ideas are considered? Are problem statements generated, and assumptions defined?
• Final Design: Does the final design offered by the team satisfy or resolve the stated problem? Is it feasible given the constraints of the problem?
• User focus: How much emphasis is placed on understanding or considering the user experience of both the problem, and the potential solution? Are specific user needs identified, and then addressed?

We instructed judges to use relative scoring. They did not have to provide numerical scores until after all teams had completed their time on stage. This allowed the judges to balance the scoring, and apply the categories in context of what all of the teams were able to do. (As opposed to Olympic figure skating scoring, which has a rigorous and absolute method for defining scoring).

Time for competition

We allowed 10 minutes for each team in every version of Interactionary we’ve done. This was defined in part because of the 90 minute limitation of conference sessions, and the time required to bring teams on and off stage.

5 minute intro + 40 mins competition (4 x 10) + 20 mins panelist time (4 x 5) = 65 minutes

We also needed to allow time for the transitions between teams, and time for tabulating the scores. We had one of our two MCs working the crowd during the tabulation time.

It is possible to try to run the event using more or less time for each team, but there is a careful balance that has to be maintained. Too much time will bore the audience, and despite the hard work of the people on stage, the educational value will diminish over time. Too little time, and it is impossible for teams to get very far or generate and consider many ideas. Ten minutes seemed like a good balance for the basic format, but with a smaller audience and different rules, other amounts of time could work.

Coming up with the design problem

As strange as it sounds, I have always kept lists of interesting design problems. For Interactionary, we got together with my list, brainstormed some new ones, and then picked from the list.

We found that non-web problems worked best with large audiences. The more intricate the design, the easier it is to get lost in details and specifics that do not communicate well out to a large group. Physical designs, such as control interfaces for automobiles, consumer interfaces for elevators and vending machines, or anything that is commonly used by the general population work best. The concepts are familiar to everyone, and the details are broad enough that everyone can follow along. It also allows the work at the whiteboard to be easier for the audience to follow.

With a smaller or more homogeneous audience, tighter problem definitions that are more technical, such as web site navigation design, or information architecture related problems, could work as well.

The problem itself should have a defined context, user, and problem. Here are some examples:

You are to design an airport food vending machine for the year 2005. The machine will dispense a variety of prepackaged food items including candy bars, potato chips, as well as three freshly brewed hot beverages (tea, hot chocolate, and hot coffee) that are created on demand. The specific problem you need to solve is how users purchase and select the item(s) they want to buy.

Design a remote control for a children’s dump truck toy. The remote must provide for the following features: steering, acceleration, reverse and forward gears and dump functionality. The twist (revealed 3 minutes after start): design for a child with a broken right arm

Stage Logistics: placement, scoreboard and timing

Because of the number of people who need to be on stage, some basic theatre dynamics come into play. If possible, place the judges on stage, but angled to face the audience. They need to have a good view of the action, but because they will be speaking between teams, they need to be visible to the audience as well.

At both CHI sessions, we were able to have a live video camera of the stage, and of the whiteboards being used by the team. This allowed us to use the large overhead displays to show the action to the entire audience, even if they could not see the stage. This setup is unlikely to be available in most organizations, but it worked quite well. Without this setup, the layout of the stage becomes even more important. Check your site lines, and position the work area for the teams so that it provides maximal viewing to the audience.

Someone has to own the countdown timer. For 2001 we had an NBA style shot clock display that the team could see as they worked. In addition, we had the MC read off time warning as 5 minutes and 1 minute left.

Microphones can be a problem. It is critical that the team on stage can be heard by the audience. Wireless microphones work best, but depending on the size of the teams, it can create some additional logistics (you have to move the microphones from the team that just finished, to the next team, without forcing the audience to wait). If you have two sets of wireless microphones, you can stage them, so that the team that is waiting to go on is already wearing them. You also need to provide a way for the panelists to be heard – the simplest solution is to give the MC a hand mike, and have him/her walk to the panel table when it’s their turn to talk.

Lastly, debrief all participants on good stage presence. Tell them to face the audience whenever possible, and to try not to obscure their view of the whiteboard, or whomever is talking. If there is a video camera, explain where the sight lines are, and that they should try to not to block them. This shouldn’t effect participants too much, if you make clear that it’s just something to be aware of – not the primary reason they are on stage. Design first, stage presence second.

Preparation and running the event

It’s highly recommended that you plan a run through for the event at least a week before you do it. Go through all of your timings for bringing people on and off stage, for running any stage equipment, and for collecting and tabulating the scores. There are always kinks or unexpected problems that arise, and you want to discover them when the auditorium is empty. Before we did this at CHI, we did a beta run inside Microsoft.

Checklist

For reference, here’s a quick list of things to make sure you cover:

• Get a small team to help run the event (size dependent on how large and elaborate the event will be)
• Find a place to do the event. Get a reservation and confirmation.
• Find teams and get commitments to participate. Send out a CFP (Call for participation).
• Find panelists who understand what’s involved. Start early – firm commitments can take awhile.
• Choose a balanced group of teams.
• Have a reserve team and panelist, ready to to participate if someone cancels.
• Adapt the basic scoring to suit your needs. Or choose not to score the event at all.
• Decide how you will document what happens, and plan accordingly (video? paper? digital camera?)
• Plan for how many microphones you will need: teams, MC, and panelists.
• Examine the stage or room, and plan out how to ensure good sight lines for the audience. Is there a projection screen? Video camera for the whiteboard?
• Make a list of potential design problems to use, and then pick the best one
• Plan time to do a practice run of the event, and get the kinks out.
• Report in about how it went. Post something to the web, and we’ll link to it.