TechWhirl (TECHWR-L) is a resource for technical writing and technical communications professionals of all experience levels and in all industries to share their experiences and acquire information.
For two decades, technical communicators have turned to TechWhirl to ask and answer questions about the always-changing world of technical communications, such as tools, skills, career paths, methodologies, and emerging industries. The TechWhirl Archives and magazine, created for, by and about technical writers, offer a wealth of knowledge to everyone with an interest in any aspect of technical communications.
Strike III, and I'm outa here <g>: Jim Pinkham notes: <<I see I've
unwittingly stumbled upon an issue that may be a pet peeve of yours.
Or, if more fairly put, at least one on which you have some long-held
and perhaps well-settled convictions.>>
Very much so. I've spent a fair bit of time pondering and reading
about the topic of metrics, and readability indices fail to make the
grade on several bases.
<<The article to which you refer, I believe -- http://
scholar.google.com/url?sa=U&q=http://www.rbainformationdesign.com .au/
Readability%2520Formulas.pdf -- makes some very valid points.>>
I was thinking of a different article (and I got the journal wrong).
The correct citation is: J. TECHNICAL WRITING AND COMMUNICATION, Vol.
29(3) 271-287, 1999. LAST RITES FOR READABILITY FORMULAS. IN
TECHNICAL COMMUNICATION. BRADFORD R. CONNATSER ...
http://baywood.metapress.com/index/6EWHJ5C5AV1XKDGJ.pdf#search=%22%
22last%20rites%20for%20readability%22%22
In the article you cite: <<It is interesting to note that even the
aforementioned article's conclusions include this thought: "If a
document that was not created with a readability formula in mind gets
a very poor score, it almost certainly needs to be reorganized,
rethought, and rewritten." And quite correctly adds that even a good
readability score does not guarantee a good document.>>
I have enormous respect for Ginny's work, but have two reservations
about this one. First, "needs to be reorganized and rethought": no
readability index tells you anything about the organization of a
manuscript or about its logic. Second, as for "and rewritten", the
Abstract (what the authors consider to be the key points) tells a
different story: "READABILITY FORMULAS ARE INADEQUATE MEASURES of how
difficult written material is for adult readers, say these authors.
In fact, readability formulas are counterproductive because they
focus the writer’s attention on words and sentences and draw
attention away from important sources of readers’ problems.
Readability formulas are being used in contexts where they have no
research base, and they are being misused by writers who rewrite to
achieve a specific required score. A better way of assessing
readability is user testing."
I can support those conclusions fully. Also note that the article is
21+ years old, and that Connatser's article presents a much more
recent review of the literature.
<<I'm not sure that a tool can be tossed out for failing to measure
what it's designed to measure...>>
I have no objection to the statement that readability indices measure
what they're designed to measure. My objection is that what it
measures does not correlate at all well with readability or
comprehension. See Connatser's article for details.
<<Lest you be constructing a straw man here, we're not proposing "to
rely solely upon a mechanical count of sentence and word lengths.">>
No, I wasn't saying that anyone recommends that. My point is that
it's simply not a useful measure. If it adds no value, and if you're
going to have to rely on a reader anyway, why waste your time playing
with the readability score? I'll grant you that readability scores
may be helpful for really bad writers who don't understand run-on
sentences and inappropriate use of sesquipedalianisms <g>, but for
professional writers? Waste of time.
<<I believe, that shorter words and simpler sentences, **all things
being equal**, do improve readability.>>
It's the ceteris paribus part that poses the problem: sentences that
are long and complicated are sometimes inevitable in fields such as
science, where using the correct jargon often means long words and
long sentences. (Note the distinction between good and bad jargon:
good jargon is the language of the discourse community, and thus
communicates effectively.)
In today's manuscript, for instance, "shiny light produced from the
green parts of leaves" is not clearer than "chlorophyll
fluorescence", and would actually impede comprehension. Yet my
version would incorrectly get a worse readability score. That raises
a crucial point: the equations used to calculate the readability
score must have the regression coefficients adjusted to account for
differences in mean word length in each genre of writing. No one
formula fits all contexts.
<<As for moving beyond what seems intuitive to your causal
correlation contention, I'll refer you to a comment concerning a
fairly technical readership in the article titled "Effects of Peer
Review and Editing on the Readability of Articles Published in Annals
of Internal Medicine." (http://www.ama-assn.org/public/peer/7_13_94/
pv3083x.htm)...>>
Note that there's a significant methodological flaw in this study
that forces us to consider the results as unsatisfactory and probably
unconvincing: the authors compared the readability results for the
original manuscript and the same manuscript _after peer review and
editing_. Nowhere did the authors indicate what changes were made
during peer review and editing, or attempt to correlate these changes
with the improved readability. As we scientists note, "correlation
does not imply causality", and here, the evidence for causality is
weak. Worse yet, there appears to be no validation of the results:
the readability scores improved, but no readers were tested to
confirm whether comprehension improved. That's probably a fatal flaw
were this a journal paper rather than a symposium paper (which
usually does not undergo peer review).
Since there is no statement about the proportion of changes that
reduce word and sentence length versus the proportion of other
changes that result from editing (by editors or peer reviewers), it
is not possible to state what proportion of any purported increase in
comprehension results from improvements in the readability index
versus improvements in the text. After 20 years of editing for
journals, I can tell you that readability and comprehension improve
enormously as a result of editing, not as a result of changing the
readability index. The change in the readability index is incidental,
not causal. I can provide you with ca. 10 years of edited manuscripts
if you want to crunch the numbers. I sure don't... besides, I'm
already convinced. <g>
I leave the last words to others, since I've said my three says on
this topic.
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - --
Geoff Hart ghart -at- videotron -dot- ca
(try geoffhart -at- mac -dot- com if you don't get a reply)
WebWorks ePublisher Pro for Word features support for every major Help
format plus PDF, HTML and more. Flexible, precise, and efficient content
delivery. Try it today! http://www.webworks.com/techwr-l
Easily create HTML or Microsoft Word content and convert to any popular Help file format or printed documentation. Learn more at http://www.DocToHelp.com/TechwrlList
