Users and Online vs Paper

["Jared M. Spool/User Interface Engineering (508)975-4343" <SPOOL -at- NEU -dot- EDU>]

Over the last few years, we've had a chance to observe hundreds of users
actually looking things up in manuals and on-line help.  Here are some
random snippets of what we've learned.  I hope this is helpful.

SOME USERS HAVE PREFERENCES

No matter how good the on-line help is, there are some users who will not
use it, short of a weapon pointed at their temple.  These people naturally
reach for paper doc.

USERS THINK THE INFO IS DIFFERENT

Users seem to expect a different type of material in paper than in on-line.
They do not expect them to be duplicates.  They seem to think that paper
will be more procedural while on-line will be more referential.  We've noted
that if they have a process question ("How do I create a chart?") they'll go
to paper.  If they have a referential question ("What is the difference
between a legend and a label?"), they're likely to go to on-line doc,
especially if the reference target is on the current screen.

PEOPLE LIKE LOTS OF MANUALS

At least, some do.  We did a study of how database system administrators
used documentation (with the goal of making paper doc go away).  What we
found was a lot of them keep the paper books open on their desk.  Often they
have multiple books open and switch between them frequently.  Their need is
to see the *FULL* screen while they are reading the doc, so on-line is
a problem.

OCCLUSION IS A BIG PROBLEM

Often, the on-line help users get covers the screen that they are trying to
get help on.  It takes a fair amount of effort and sophistication to get both
on the screen at the same time.  Users have trouble retaining the information
when they can't see both at the same time.  Paper doc makes this much easier.
(One can see what is in their lap and on the screen simultaneously.)

USERS RARELY GO PAST INITIAL SCREENS

Most of the references to help we have seen require that the target information
(or a significant clue to it) appear in the first screen.  If not, users exit
from help and do not look further.  Rarely will a user read past an initial
screen, including using the scroll bar.  This is problematic when using the
default size of the help reader, where the author may not have a choice as to
what is and is not showing.

HELP ENGINE SPEED PLAYS A SIGNIFICANT



 ROLE

We have observed several products where the speed of the retrieval engine
was so slow that users would reach for the manual and successfully look up
what they wanted while help was just loading.  Users do not seem to like
sitting and waiting for help.

BAD EXPERIENCES SPOIL IT FOR EVERYONE

If a user has a bad experience in help with one product, they are less likely
to ask for help in other products, even from other vendors.  However, if they
have a bad experience with paper doc, this does not seem to translate to other
vendors.

HELP BUTTONS WORK

In a test of Microsoft Excel vs. Lotus 1-2-3, we found that users were
significantly more likely to use on-line help in Excel and paper doc in 1-2-3.
The reason turned out to be that Excel had buttons in their dialog with the
word HELP on them, while 1-2-3 had little buttons with just a "?" in the
corner.  Users, when asked after the testing, said that they didn't know the
"?" button was help, or even noticed it being there.  (It was in the title bar
of dialogs, which people tend to ignore.)  Lotus' desire to free up screen
real-estate by not having big HELP buttons worked against them, unless their
goal was to reduce HELP usage, at which point, it worked.  (People tend to
hit buttons and not F1, by the way.)

SERENDIPITY PLAYS A ROLE

People often found information by scanning or skimming the docs.  People often
were attracted to a screen shot that looked like it might have the answer and
then would read the text around the screen shot.  It is much easier to do this
on paper than on on-line.  (We've played with the idea of having a "scrapbook"
of images that the user can skim through, to let them "flip" through the
electronic manual, as it were.  We haven't tested it yet.)

--------------------------- *** DISCLAIMER *** -------------------------------

While these results are based on actual usability evaluations of many different
products, it is very possible that your mileage will vary.  Every population
of users, their goals and their contexts differ and require unique solutions.

BACKGROUND

User Interface Engineering is a North Andover, MA consulting firm specializing
in usability issues.  If you would like more information on our research into
usability issues, courses and workshops, (including a two-day course on
Creating Usable Documentation,) and our forthcoming book on Wizards, please
feel free to contact us at

User Interface Engineering
800 Turnpike Street, #101
North Andover, MA 01845
(508) 975-4343
Fax: (508) 975-5353

or email me at Spool -dot- CHI -at- Xerox -dot- com -dot- 

Jared M. Spool
User Interface Engineering