TechWhirl (TECHWR-L) is a resource for technical writing and technical communications professionals of all experience levels and in all industries to share their experiences and acquire information.
For two decades, technical communicators have turned to TechWhirl to ask and answer questions about the always-changing world of technical communications, such as tools, skills, career paths, methodologies, and emerging industries. The TechWhirl Archives and magazine, created for, by and about technical writers, offer a wealth of knowledge to everyone with an interest in any aspect of technical communications.
Jennifer Gidner wondered: <<Our small (2-member) Technical Writing shop
has been told that we will be measured this year by the results of a
customer satisfaction survey that we must create. Yikes!>>
The first step in designing such things is to sit down with the people
who will be interpreting the results and find out their parameters for
success: What metrics do they want you to collect, why, and how do they
intend to use those metrics? For a few thoughts along those lines:
http://www.geoff-hart.com/resources/2004/metrics.htm
It's particularly worth noting that since you have no baseline data at
this point, it's hard to say whether any given review is good or bad.
It's often more effective to use this first year as a way to identify
baseline statistics, then perform the review again next year to see if
you've improved or maintained a high level of satisfaction.
You may also want to try diplomatically changing this from a simple
metric exercise to a way of obtaining focused comments that will help
you improve. Management is often really taken by the idea of continuous
quality improvement, and you can make the chore of obtaining
measurements more useful to everyone if you seek feedback that provides
clear targets for improvement. A few more thoughts:
http://www.geoff-hart.com/resources/1995-1998/negative.htm
Don't forget that randomly targeted surveys are going to attract more
responses from people with an axe to grind, and few other responses.
Most user satisfaction surveys attract a response rate way below 5%
unless you add some hefty bribe to encourage participation. The
negative responses are good if your goal is finding problems in need of
improvement, but not so good if you don't want your managers to fire
you because of all the negative comments. <g>
It may be more effective to take an approach such as focus group
interviews or usability reviews, in which you bring people in or talk
to them on the phone (select them according to useful criteria such as
"broadly representative of our audience") and specifically get them to
work with you to review the documentation. More expensive and
time-consuming, but worth it if your goal really is to get useful data.
A powerful alternative is changing the problem you're trying to solve
away from rating satisfaction and towards evaluating success. The gold
standard for this is "reduced number of calls to tech. support". If you
can work with your tech. support department to identify the main
documentation problems (things people can't find in the manuals) and
can separate documentation problems from interface or usability
problems, so much the better. Don't let them blame you for things that
aren't your fault!
<<Such as, what are good questions? What is a good scale?>>
A good question has two characteristics. First (selfishly), it provides
the answer you need to impress the heck out of your manager so they
give you a raise and an extra week of vacation. <g> This suggests that
you need to ask questions whose answers will give you positive
feedback.
Second (less cynically), the answer to the question must give you
something you can act upon. To find out whether you can, ask the first
five people who wander past your desk to answer the questions. Analyze
the results and ask yourself the question "now what?" If you can't
answer that question, then your survey is useless other than to give
your managers useless numbers to compile in a file.
Let's say your satisfaction rating is 3 out of 5... perfectly
respectable. What do you do about it? If all you collected is numbers,
you can't do anything about it because you don't know what cost you 2
points. This suggests you need some of those "negative" questions in my
second article to help you pin down how to improve.
WebWorks ePublisher Pro for Word features support for every major Help
format plus PDF, HTML and more. Flexible, precise, and efficient content
delivery. Try it today!. http://www.webworks.com/techwr-l
Doc-To-Help includes a one-click RoboHelp project converter. It's that easy. Watch the demo at http://www.componentone.com/TECHWRL/DocToHelp2005
