Quality/validation

[PIP Mailbox <prc -at- PIP -dot- DKNET -dot- DK>]

>Date:    Wed, 31 Jan 1996 23:54:15 GMT
>From:    Charles Good <good -at- AUR -dot- ALCATEL -dot- COM>
>Subject: Re: Quality/validation

>I think it is possible to determine the accuracy of document content
>and to note any deficiencies such as overlooked topics. It is also
>possible to grade documentation based on the number of spelling and
>punctuations errors found, or the number of incorrect steps, or the
>number of incorrect references, etc.

>However, most quality systems are based on being able to measure
>something exactly and compare it against a recognized benchmark
>(which should NOT be your competitor's offering). The problem
>you run into with documentation is so much of the quality aspect
>is subjective. It's partially dependent on style, layout design,
>quality of illustrations, usefulness of tables, etc. These
>characteristics are harder to define as measurable parameters.

>In addition, quality deals with repeatable processes and controllable
>environments. However, if there is little automation and a lot of
>human creativity involved, then the process becomes less exact. A
>writer's mental and physical health affect his or her productivity and
>level of quality. Even the person reviewing the documentation in the
>quality/validation process must be considered susceptible to personal
>values and preferences, as well as being vulernable to biorythms and
>general health. Neither the writer nor the reviewer are machines that
>can be monitored to determine peak efficiency periods when they will
>do their best work. Therefore, can a human being become an instrument
>for measuring quality of a subjective product? Doubtful.

>Soooo... Quality of documentation is subjective and since quality is
>defined (nowadays) by your customers, I doubt you could even get a
>concensus among your various customers as what constitutes quality
>documentation. At best, they will make some generic statement like,
>"we want it accurate, easy to use, and portable". Unfortunately, those
>types of qualities are difficult to qualify in terms of measurable
>parameters that will yield meaningful trend data for continuous
>improvement of quality.

>On the other hand, validation is much simplier because it deals
>mostly with something either being right or wrong, or something
>is either documented or missing.

>Anytime you start to use validation as a sign of quality, you
>invite the guardians of semantics and philosophy to challenge
>your methodology and terminology.


Sorry, but I got a little sorry/angry when I read the above mail.

When something looks impossible there are two standard attitudes:
- Not possible - forget it!
- Sounds interesting - lets try!

Quality evaluations of instruction manuals is in fact possible. On a
"soft" area like this, it will never be 100% objective (what is 100%
objective?), but you can come quite far!

The 3 basic steps are:
1. Classify the product complexity and the (weakest) users of the manual.
2. Relative to this, according to the tools & rules of the profession
    (readability index, imperative mode, etc, etc.), determine if the
    rules are fulfiled for these classes, and if the classes fit together.
3. Make useability tests.

The tools for these 3 points exists. Point 1 and partly point 2 and 3 are
covered in my book, see my homepage at
http://www.pip.dknet.dk/~pip323. Point 2 is covered in more details
by the German TEKOM standard and German TUV's further improvements
on this field. Point 3 is covered in a number of publications, see, e.g.,
the litterature list in the TECHWR-L digest of 31 January 1996.