DIGEST: Productivity Metrics-MS Word vs Structured

[HALL Bill <hallb -at- TENIX -dot- COM>]

Good people:

A while ago I asked if anyone had measured changes to technical document
productivity when moving between an MS Word and structured authoring (e.g.,
FrameMaker, Adept Editor, etc.).  Word is seen as "no-cost", and I need to
make a productivity case based on numerical metrics to convince executives
with no keyboard experience.  In hopes of getting a big enough response to
do some statistics, I circulated this request to the  Adobe FrameMaker
Forum, Framers eGroup, and TECHWR-L.  Three responded with similarly unmet
needs and six offered data. (All responses have been pruned.)

================================================
From: Laurie Morgan - 23 October 1998
     I would be very interested in the responses specific to converting
documents to SGML using Framemaker+SGML (time involved, staff
considerations, development process, etc.)

----------------------------------------------------------------------------
--------
From: macaw - 26 October 1998
     Please keep me posted when you get feedback on this. I'm fighting a
battle to keep framemaker in a new organization that seems to think we
should work in Quark.. heaven forbid... or Word.

----------------------------------------------------------------------------
--------
From: Christie Bradley (cbradley -at- iwa -dot- dp -dot- intel -dot- com) - 27 October 1998
     If you get quantitative data back on you Frame vs. Word, etc.
productivity issues, please let me know.... FrameMaker IS more productive.
But, like you, I need data. The only input I have gotten through list
resources has been conjecture or very fuzzy "feelings" about why people
prefer Frame.
     I am in the process of compiling a table that summarizes, from my
experience and the subjective collective comments I have encountered, what
are the strengths and weaknesses of Frame and MS Word. The piece I am
missing is any data that shows what "everybody knows" is really the case....
================================================
From: Sheila Carlisle - 25 Oct '98
     In general I'm not very surprised that documented productivity info is
so difficult to come by -- it's probably a combination of factors:
1.  [few] companies would take the time/effort do the studies; the easier
way out ... would be to base the decision simply on the feature comparison.
2.  of those that would, even fewer would share that information as it would
probably be seen as a form of competitive advantage to know that you're
improving your efficiency using a particular tool....
     Another related issue that I've been thinking about lately is the cost
of internal tech support for word-processor-based production vs. FM
production -- is that also included in your justification process? E.g.,
time spent troubleshooting corrupted files, retrieving archived versions,
rebooting because of hung systems, etc.
     (Sheila also sent some Adobe case studies and info on other products.)

----------------------------------------------------------------------------
--------------
From: Barbara Karst-Sabin (longish ramble) - 23 October 1998
     This is pretty hard to quantify, since the products have such different
nomenclature and methodology between/among them.  There's quite a learning
curve making the switch from one to the other....
     I've gone from Quark to Word to Framemaker and back to Word again --
each package can do some things better than others, and each package can do
things that the others can't.
     I've got to admit I'm having the most trouble re-adapting to Word after
using Frame and Quark.  It seems niggling, cranky, and clunky by comparison
and has lots more gotchas than the other two.
     Although you want the differences quantified (and I wouldn't even try
to guess), I can tell you that for lengthy doc or docs with lots of
formatting, I prefer Quark or Frame.  Word chokes way too often for my peace
of mind.
     I also think most people who have used Word... will be amazed at the
increased functionality of a Quark or Frame and never want to go back
(although Frame is fairly counterintuitive, once you learn it's quirks and
foibles, you really can do a lot with it). If you build training into your
equation, I think Word will be left in the dust (IMNSHO)...

----------------------------------------------------------------------------
--------------
From: Scott Turner - 27 October 1998
     ...These tidbits may also be useful.
     The US Navy has switched to SGML for all manuals, which has lightened
their vessels by as much as 5 tons. They maintain all manuals in SGML, not
word, since the structed format required cannot be matched by Word.
     The US Federal Drug Administration requires all documentation for new
drugs be submitted in SGML format, since the format is controlled through
DTD rather than a template file that may be lost in transit, or modified. We
are talking about doucments which run into the 10s of thousands here.
     The airline industry in the US, specifically American Airlines, uses
FM+SGML to create their maintenance cards and other manuals since the
structured environment of SGML, and the ease of use of FrameMaker, allow
them to maintain thousands of documents, and distribute them electronically.
Viewing or printing of these documents is, of course, controlled through
DTDs.

----------------------------------------------------------------------------
-------------
Steve Lewis - 27 Oct '98
     Without knowing exact figures (...companies I've worked for never
cared), I can give... some ballpark estimates....
     I came from a Word background, but learned FrameMaker because the
company I began working for already used it. [My] learning curve was about
two weeks to get about as comfortable with the product as I was with
Word....
     Throughout my experience with [FM], certain corporate policies were to
create templates for documents and drawings in both Frame and Word. Now,
given that... I knew both products reasonably well (with favoritism towards
Frame), creating the Word version of the document took... 2-3 times as long
and was extremely unmaintainable. Some things were also dropped since they
could simply not be done in Word.
     Also, I recently was asked to create a simple document with a complex
layout (e.g., rotated text frames, exact text positioning, etc.). Before I
was asked, other "professional" companies simply stated that it couldn't be
done. I completed the task in Frame in about 15-30 minutes. I'm sure of the
competence level or experience of the other companies that were asked.
     I guess what I'm trying to explain is that it's very difficult to
quantify... time savings from... using [either] product.... Sometimes, the
complexity of the document cannot be accomplished by using one product while
relatively simple to accomplish in another. Another primary factor is the
[authors]... If people... accept change, moving to a better product [FM]...
would be worth the learning curve. It's also quite realistic that people...
frustrated with Word's idiocies will just leave the company if they're aware
of something better....

----------------------------------------------------------------------------
-------------
Melanie Boyd - 27 Oct '98
     I agree with Steve -- it's difficult to measure time savings alone
because Frame allows you to do so many things that simply can't be done in
Word. I have used both products extensively in several different work
environments, but I've never worked for a company that took time to measure
the time it took to perform specific tasks. I can give you the following
rough estimates...
     - The first time I set up a book file in Frame and generated a table of
contents, it took about half an hour to figure out what I was doing and
actually do it. The first time I set up a Master Document in Word and
generated a table of content, it took almost 8 hours.
     - It takes me significantly longer to set up a template in Frame than
it does in Word, because in my Frame template I create master pages to
define the layout of several different types of pages; automatic numbering
for chapters, figures, tables, and steps; cross-reference formats to be used
in various situations; several different table formats to meet specific
needs; and conditional text settings to allow us to have three versions of
our documentation and only one set of files (having this option saves a
tremendous amount of time). In Word, I just define the paragraph and
character styles because none of the other features are available.
     - Although it takes me longer to set up the templates, this time is
more than made up for when we start using them.  In Word, we have to
manually format each table unless we want to use their predefined format. We
type text around cross-references that Frame would have inserted
automatically. We also manually enter step numbers rather than use Word's
automatic numbering, because Word will not allow you to use automatic number
AND indent a paragraph (we want to indent our text column to make headings
stand out more). I would love to know how much time it takes to manually
number steps, renumber them each time a procedure changes, and then proof
them at the end of the process.
     Sorry I haven't been able to give you any quantifiable measurements.
And yes, I know that I've started talking about which product is "better".
But it's hard to separate time savings with the amount of flexibility and
automation a product offers (and how reliable it is).  Currently, I use
FrameMaker for all of our complex documentation, and there are a lot of
things that I just wouldn't do if I were using Word. But if the documents
you need to create are very simple, then you might not realize the same time
savings that we do....

----------------------------------------------------------------------------
-------------
From: Martin Anderson (manderson -at- ameritech -dot- net) - 27 October 1998
1. ~50% improvement in productivity
2.  Framemaker 5.5.3 vs. Word 95 & 97
3.  Did not use structural controls in Framemaker
4.  Took six months to achieve stated improvement.  This is how long it took
one person to convert 350,000 words of doc from Word into Framemaker. That
person was me.
5. The tech writing team was six people, then it became one person (me).
The team was responsible for 13 volumes, as am I.  I hired a temp, making it
a two-person team. We worked in Word.
     Then I got Framemaker.  Then I fired the temp, so it's a one-person
"team" again.
     I base my productivity gains on the fact that the switch to FM allowed
me to halve my staff and still release 13 volumes of info twice a year.  If
you base the productivity gains from the original staffing levels, it would
be several hundred percentage points.
6. Most profound effect: the fact that FM forces a structured approach to
doc.     This made my doc *much* less quirky -- I wasn't always having to
work around weirdo style problems etc. I wasn't having to spend time
tweaking things, manually resetting running heads, etc. Related to this
point is the fact that the HTML output (using Quadralay's product) is vastly
superior to the Word export.  Mind you, I used a $200 add-on product for the
Word HTML output, so I think it's a fair comparison.
7. Environment: I write the manuals for a mainframe product (MVS, VSE,
OS/390).  This product has new versions twice a year, at which time all of
the manuals have to be released for the new version -- 13 manuals in all
(plus ancillary stuff)....  Everything is online-only, except for the
Installation Manual, which is printed.  At one point, this required a staff
of six. That was when everything was printed.  The shift to online-only was
*forced* by the downsizing, not by my conversion to Framemaker (I was
already online-only when I used Word 95/97)....
================================================

Many thanks to all for your replies.  The information above will definitely
help my own case:  A very conservative assumption for moving long/complex
documents from Word to a structured environment should be at least 25%
improvement, and more likely 35-50% or more (confirms my own beliefs and
feature comparisons).  A 25% figure should suffice for my case, supported by
a list of factors affecting author productivity.  For an author costing
$50,000/yr and spending more than 25% working in long/complex documents,
providing a $3,000/seat (about what either FM or Adept Editor costs here)
structured authoring application should pay for itself within a year.

Adobe et al. take note!!  A $1000/seat product would be cost-effective
against 'free' MS word even for casual authors (10% or less).  The
difference between being effective for 10% authors vs 25% authors might
increase the potential market volume for structured authoring systems by ten
times - and everybody except possibly billy gates would be better off.

If anyone else wants to add to the above, please email me directly, and if
there is enough new information, I will do another digest (or stat.
summary).

Regards,
Bill Hall  (hallb -at- tenix -dot- com)
Documentation Systems Specialist
Tenix Defence Systems Pty Ltd
Williamstown, Vic. AUSTRALIA

 From ??? -at- ??? Sun Jan 00 00:00:00 0000=