Usability -results - long post

["Julie A. E. Tholen" <tholeja -at- ANUBIS -dot- NETWORK -dot- COM>]

Awhile ago I posted a request for info on who was doing
documentation usability testing. I got several great responses
and resources.  I would like to share the results.

Original Questions:
1. How long has usability testing been done at your shop?
2. Why/when was it implemented?
3. How is the testing done at your shop?
4. Who does the testing? (internal/external - name names!)
5. What has been the impact on the documentation?
6. What has been the end user reaction?

Sam replied that the did usability testing by getting typical users and asking
them to use the documentation and then observed the process. While not done in
a formal usability lab, the testing takes place in as close to typical setting
as possible. His company also sends employees to customer sites to see how their
products are being used. The company also has strict style/format guidelines forthe documenation

Ruth shared that STC-Twin Cities chapter did a panel presentation on usability
testing, with West Publishing participating (law/text books). She also provided
additional local business contacts that did usability testing.

Based on Ruth's info, I was able to contact one of the larger computing concerns
and got great info on their procedures.

Greg shared that while there usability testing effort started with documentation
the practice has now spread to the software division. Things that are tested
are interfaces, software designs, manual layout, CBT,  and online documentation.

They develop a formal testing protocol based upon inhouse information.
Initially, testing was video recorded. They now have a usability labe with two
rooms, a control room and a testing room. The rooms are separated by a one-way
mirror and had a couple of remote cameras, one to capture the screen, one for
the person testing. They also have a PC so that actions/events could be recorded
as they occurred during the test, including a headset for a recorder to make
verbal comments during the test.  This setup cost under $70K.

All testing at Greg's site was done internally.

The initial impact on the documentation was great. Many simple problems,
primarily in accessibility of information were cleared up. They simplified TOCs,
and put more efforts into indexes. They also improved online documentation.
They had been using a homegrown system and switched to an off the shelf product
that had more usability built in then their system could provide.

Customers that were test subjects did notice improvements, and end users have
commented on the software testing efforts, which are incorporated into the
testing process.

Bonnie shared that one of her clients is using the tutorial that she created,
so in essence it is getting tested with each new release of the software.  In
addition the hotline technicians are given copies of the doc to mark up as user
questions come in. Each new tester is given a copy of the manual and told to
read it before running the test cases and to mark it up as necessary. Another
client of hers cannot even be convinced to run the doc against the software,
let alone test it thoughly.

Lisa  shared a website that is dedicated to usability testing:
http://www.cris.com/~balbes/quality.html, and the instructions on how to
subscribe to the listserv.
TO: listproc -at- hubcap -dot- clemson -dot- edu
SUBJECT: subscribe utest <your name>
BODY:  <your email address>

One of the links "Quality Online" takes you to http://www.dcez.com/~qadude
 QA Dude, The Quality Information Center, which has over 140 links to online
quality resources!

As for my site, right now our manuals are sent out for reviews to tech support,
the system test lab, the core team(developers, engineers, marketing), training
and anyone else in the company who wants to see it.  They receive a copy to
be marked up, and depending upon their expertise, test it or review it for
content.  My current project is a cooperative one with a Canadian concern, so
the documentation is also reviewed by them, and they have set the standards for
the documenation's layout and style.

The input provided by the members of the list will be of great assistence in
evloving our testing procedures.  Many thanks,

Jules    "Life is short. Eat dessert first"
(J.A.E. Tholen)                         Network Systems Corporation
612.391.1354                            7600 Boone Avenue North
tholeja -at- anubis -dot- network -dot- com              Minneapolis, MN 55428