Executive in wait

Paper prototyping June 4, 2008

What is it ?

Paper prototyping is low-fidelity method of usability testing that is useful for Web sites and Web applications. They are used clarify requirements and enable draft interaction designs to be very rapidly simulated and tested.

What are the benefits ?

• With paper prototypes, potential usability problems can be detected at a very early stage in the design process before any code has been written.
• Easy iterations. Because the prototype is all on paper, you can modify it very easily to fix the problems you find.
• Eliminate technology variables from the usability testing equation
• Cost. If you are on a shoestring budget, paper is a great low-cost alternative to many software packages

What do you measure ?

• Efficiency — How long does it take people to complete basic tasks? (For example, find something to buy, create a new account, and order the item.)
• Accuracy — How many mistakes did people make? (And were they fatal or recoverable with the right information?)
• Recall — How much does the person remember afterwards or after periods of non-use?
• Emotional response — How does the person feel about the tasks completed? Is the person confident, stressed? Would the user recommend this system to a friend?

How is it done ?

• You first decide on the tasks that you’d like the user to accomplish.
• Next, you make screen shots and/or hand-sketched drafts of the windows, menus, dialog boxes, pages, popup messages, etc. that are needed to perform those tasks.
• Then you conduct a usability test by having one or two developers play the role of “computer,” manipulating the pieces of paper to simulate how the interface would behave.
• Users are given realistic tasks to perform by interacting directly with the prototype — they “click” by touching the prototype buttons or links and “type” by writing their data in the prototype’s edit fields. (Using transparency or removable tape prevents the prototype from being written on directly.)
• A facilitator (usually someone trained in usability) conducts the session while other members of the development team observe and take notes.
• The “computer” does not explain how the interface is supposed to work, but merely simulates what the interface would do.
• Debrief users afterward to measure interface recall

What do you don’t need for this exercise ?

• Don’t spend time making the prototype look neat before you test it — if it’s legible, it’s good enough.
• Straight lines or typed text. If the user can’t read something, it’s OK to tell them what it says. (But if the user doesn’t understand a term, don’t explain it — change it.)
• Images or icons. Use words instead. For example, for the company logo, you can just draw a box around the words “company logo.” If images are part of the content (a product catalog, for example), you can paste them into your prototype using restickable glue, which allows you to rearrange the page later
• Color.Color can’t save an inherently flawed design — do your initial testing with grayscale printouts of screen shots, or sketches using any dark-colored marker.
• Consistent sizing of components. Unless you’ve got a small or densely-packed display, don’t worry about adhering exactly to a grid. It’s OK if components are of varying sizes.

What is it good for ?

• Concepts and terminology. Do the target users understand the terms you’ve chosen? Are there key concepts they gloss over or misconstrue?
• Navigation/workflow. If there’s a process or sequence of steps, does it match what users expect? Do they have to keep flipping back and forth between screens? Does the interface ask for inputs that users don’t have, or don’t want to enter?
• Content. Does the interface provide the right information for users to make decisions? Does it have extra information that they don’t need, or that annoys them?
• Page layout. Although your scribbled screens may not be pretty, you’ll still get a sense of whether users can find the information they need. Do you have the fields in the order that users expect? Is the amount of information overwhelming, not enough, or about right?
• Functionality. You may discover missing functionality that users need, or functionality you’d planned but users don’t care about.

What is not good for ?

• Technical feasibility. Paper prototypes don’t demonstrate technical capability. It’s possible to create a paper prototype that can’t actually be implemented. To avoid this, its recommend that there always be at least one person involved who understands the technical constraints.
• Download time or other response time. Because a person simulates the behavior of the computer, the “response time” is artificial.
• Scrolling. Subtle problems with Web page designs discourage the user from scrolling either down the page or back up to the top. These problems cant be found with a paper prototype.
• Colors and fonts. If you really need to see how something looks on a computer screen, paper prototyping can’t show you that. It’s a good idea to involve the graphic designer in the paper prototype tests because he may find issues that influence the visual aspects of the final design.

Why not just use HTML to create a mockup ?

• Three researchers at Verizon compared the type and number of problems found with a low-fidelity (i.e., paper) prototype as compared to a working prototype. They found that there was a significant degree of overlap between the problems found using the two different methods. Although the working prototypes did uncover a few more problems, they also took significantly longer to develop than the low-fidelity ones — weeks instead of days. These results indicate that there are diminishing returns from taking the additional time to develop a polished prototype.
• Zero coding effort. While it’s possible to mock up a decent-looking interface pretty quickly in VB, Dreamweaver, etc., writing the code to make the interface respond properly to the user’s inputs can be time-consuming. With a paper prototype, the time spent coding is zero — even if you’re a real whiz, it’s hard to be faster than that!
• Avoid nit-picky feedback. A polished-looking design can actually encourage the wrong kind of feedback. e:g Those fields don’t line up. Paper prototypes avoid that kind of feedback because it’s obvious to users that you haven’t specified the look yet. This encourages users to focus on the concepts and functionality.
• Encourage creativity. Our brains respond more creatively to things that look somewhat unfinished. And users — especially non-technical ones — are often less intimidated by paper prototypes than by computers, so they’ll feel more comfortable exploring your design.

Tips for facilitators and observers

• When participants arrive for a usability test, explain that you know the design has some rough edges.
• Planning, preparing, and conducting a handful of usability tests typically take on the order of a week of hands-on time, spread out over 2-4 weeks
• Stay for the Entire Test
• Remain Silent While the Users Are Working
• If users get stuck on a task, that means that there is a wealth of information youshould be fervently taking notes on
• No Helping. During the test, it’s likely that users will have problems using the interface, and it is normal to feel a temptation to help. Please don’t. Instead, try to understand why it was that the user got stuck or went down the wrong path. It’s the facilitator’s role to get users back on track if they get really stuck.
• Avoid “Design Questions”. You will have an opportunity to ask questions after each task. Questions that ask the user their opinions about how to design aspects of the application (such as, “Where would you like to see these navigation buttons?”) can take a lot of time to answer and produce only limited results. Instead, focus on trying to understand the problem—we’ll come up with solutions later, outside the test.
• Here are more tips.

What are the steps?

• Pick the target user to focus on for this series of tests. (6-12 users). If you decide to test existing customers, you may already have an in-house source of users. If not you can use market research firms.
• Decide who will facilitate and observe the sessions.
• Defining the tasks you will ask the users to attempt with the interface
• Developing a paper prototype of the interface
• Holding a “rehearsal” to prepare for the usability tests
• Conducting the usability tests while members of the usability team observe and take notes
• Responding to the issues we discover from the tests by making as many changes between tests as possible.

Here is a good book devoted to this subject, if you need more info.