Re: Peer Reviews

[Stuart Burnfield <slb -at- FS -dot- COM -dot- AU>]

Yes to everything Sue Gallagher and Bonni Graham said.

I've had more experience with peer review as a programmer than as a
writer, but I think the lessons are very similar. Here goes:

- schedule time for the review.

- between 3 and 5 people, including the author. 6 is too many, unless
  one is an observer.

- don't start the review unless at least two of the reviewers have had
  a proper look through. "I was really busy so I just skimmed it"
  isn't good enough; postpone the meeting if you have to.

- give people roles. At a minimum, there should be a secretary, whose
  job is to write down every change that is agreed and later to see
  that it is actually done, and a chair-critter, who just keeps the
  review moving in a constructive way.

  The secretary and chairthing do not have any special authority over
  the author. They just make sure:

  AT THE MEETING:    everyone has a say
                     the talk is relevant and useful
                     every agreed point is written down

  AFTER THE MEETING: every agreed change was carried out.

- there is a school of thought that the boss shouldn't be present. This
  is hard to stick to, especially if the boss is the most experienced
  person and you are short of suitable people to be reviewers. I do
  believe it's very important that the boss doesn't chair the meeting.
  People can be reluctant to argue against the boss's point of view.
  If you _are_ the boss, please be aware of this.

- Also be careful of the bossy know-it-all (assuming this is not also
  the boss). Don't make this person the chair; they'll try to run
  things. Consider making them the secretary, as this might keep them
  too busy writing to be a nuisance. But make sure that what they
  wrote is what you agreed!

- the main danger is trivia. Avoid arguments over punctuation, layout,
  grammar, etc. If these are important they should have been decided
  outside the meeting. That's why the style guide is so important. If
  a new issue of house style arises, note it down to be settled later.

- if someone hasn't reviewed the document properly, they may be
  tempted to quickly find some minor errors to do with spelling, so
  they don't feel so guilty about not having done what they were
  supposed to. Minor fixes are great but they too should be handled
  outside the review. Ask reviewers to hand a list of these points to
  the author. They're not controversial, they don't need to be
  discussed, they just need to be fixed.

- find don't fix. The review is to find things wrong with the document,
  not to rewrite it. If the author wants advice they can ask for it
  outside the meeting.

  GOOD: "This paragraph is not clear"
        "This paragraph is not correct, because..."
        "This topic is too long"

  BAD:  "This sentence sounds better if you put it like this..."
        "I still think the punctuation should be _inside_ the quotes"

- Hate the sin, not the sinner. In other words, you're there to review
  the work, not to criticise the author. Remember that a lot of the
  author's self esteem is invested in their work. What seems like
  helpful criticism to you may feel like a personal attack to them if
  you don't phrase it carefully. Also remember: your turn will come...

- don't go over about 50 minutes. People can't concentrate any longer.
  If necessary schedule another session. Yes, everyone has work to do,
  and yes, it's expensive having them all sitting in the review, but
  it's more expensive sending out buggy work that reflects poorly on
  your company.

To sum up, whatever people can do by themselves or one-to-one with the
author should be done before or after the review. Use the review for
things that are likely to need discussion or agreement.

Phew! There's more, but I have to do some work.

If you can't find any references on technical writing reviews, look
for a good book on software development. There should be a section on
code reviews or walkthroughs.

Regards
---
Stuart Burnfield        slb -at- fs -dot- com -dot- au   Voice: +61 9 328 8288

I believe in love, hope, truth, joy, wonder, freedom, diversity...
note that these don't necessarily represent the opinions of my employer