TechWhirl (TECHWR-L) is a resource for technical writing and technical communications professionals of all experience levels and in all industries to share their experiences and acquire information.
For two decades, technical communicators have turned to TechWhirl to ask and answer questions about the always-changing world of technical communications, such as tools, skills, career paths, methodologies, and emerging industries. The TechWhirl Archives and magazine, created for, by and about technical writers, offer a wealth of knowledge to everyone with an interest in any aspect of technical communications.
Subject:On-Site Searching From:"Jared M. Spool" <jspool -at- UIE -dot- COM> Date:Fri, 7 Nov 1997 08:52:52 -0500
This following is an article I recently wrote for UIETips. I thought
you guys would find this to be of interest.
Jared
- o - o - o -
On-Site Searching
In a recent study, we sent users off on scavenger hunts of
information on various web sites. While all of the information
that they sought could be found just by using the links provided,
users would often use the search functionality provided by the site.
(We saw users use search on 75% of the sites in our study.)
This makes sense. Theoretically, if you know the keywords, you
should just be able to type them in and instantly find the page
you're looking for. However, in practice, it doesn't work that well.
Search engines didn't make finding information easier, they made
it harder. When users found information without a search engine,
they did 50% better than when they tried to use a search engine.
Overall, having a search engine on a site doesn't seem to make
the site more usable. Of the different sites we tested, the best
search engine helped users find the target information only 50% of the
time. Some were as low as 25%.
We think there are four major reasons for this:
1) Users don't know how to narrow searches
Few users did anything other than simple keyword searches.
(Almost no one used booleans or other features of the search
engines.) They would often type in very broad terms, such as
"Videos" when looking for how a video about the Wild West to
get a friend. This is the equivalent of walking up to a librarian and
just saying "Travel" and expecting they'll instantly find the book on
Hawaii that you are interested in.
When users got back a result set that was too large to be
practical, they often tried completely different keywords instead of
adding to the set they had. They didn't seem to know any strategies
that would have allowed them to take the stuff already found and
narrow it down further (for instance, "regular tire rotation").
None of the sites that we tested provided any useful information
on how to narrow a search. In fact, most seemed to assume that
users knew how to search effectively and didn't provide any clues at
all.
2) Full-text Search engines are not indexed
Users didn't seem to understand what a full-text search is. The
dynamics of full-text searches are different than looking something up
in an index, but users didn't seem to grasp this.
For instance, they were surprised when they typed in the word
"Tire" on the Car Talk site (http://www.cartalk.com) to find results
that contained the word "entire" or the phrase "I'm tired." Although
the site did present the option to search for entire words or partial
words, users didn't change the setting (the default was partial).
We also saw that users didn't understand that plurals and singular
words would produce different results and were surprised. Users
didn't know that typing errors would produce poor results and couldn't
tell that it was a typo, instead of a lack of content, that got them
the "nothing found" message. (For example, one user mistyped "Videos"
as "Vidoes" and got zero hits -- and then assumed that there weren't
any videos on the site.)
Full-text searching produces a lot of irrelevant information for
users. For instance, one of the tasks we had for the Smithsonian
Magazine site (http://www.smithsonianmag.com) was, "Your
six-year-old son has to do a report on dinosaurs for school. You
remember an excellent article in an old issue of the Smithsonian
Magazine that you think would be a fine reference. Go find it."
For this task, users naturally typed in the keyword "dinosaur."
The first article returned was on the American steel industry --
one of the great American industrial dinosaurs. (Go try this
yourself, it's great!)
Indexing is a craft that takes a lot of skill. No professional
indexer with any self-respect would ever put an article about the
steel industry under the topic of dinosaurs. As sites get bigger, this
problem will only become more of an issue. Full-text searches will
get more noisy and irrelevant as more words are introduced without any
sense of what makes them important to the content.
Successful searching is essentially an indexing task. To help
user search more effectively, this intelligence must be designed
into the site. We think that professional indexers and others who
have these skills will become more valuable in the years to come.
3) Multiple search areas are not clear
Many of the sites provided the ability to search different types of
content or different areas of the site. This capability differed from
site to site. For instance, the Smithsonian Magazine site lets users
search either Feature stories, Columns, or Back Issues '89-'94. Users
didn't know which one to pick.
Car Talk has four search areas that do not overlap. The designers
have scattered the search screens in several different locations
(usually with the content it searches). Users not finding information
in one area didn't know that they should search other areas.
Disney (http://www.disney.com) lets users narrow the areas to
search only after they've done an initial search. There is no
explanation to users as to how the results from the other areas
will be different from the search they just conducted.
4) Search engines are short cuts
As a short cut, the search engine is intended to get users quickly to
their content. But like other types of short cuts, users first need
to know how to get there "the long way."
The short cut requires that users understand how the search engine
works, how the content has been segmented and indexed, and how content
has been labeled (page titles). This is a lot to ask of users and
most are not up to the task.
Our data shows that the site uses a search engine, there is a
strong correlation to users failing to find the right information
(r-squared > .7). It also shows that users will gravitate to a search
engine when the links are not clear. On sites where no search option
was provided, users complained, but did 50% better than the best site
that had a search engine. (This is more proof that users don't
necessarily know what's good for them.) And with all the times that
users failed using search engines, they never seemed to associate the
failure with the search engine -- it never occured to them to try a
different tack. Instead, they gave up.
At this point, based on this information, our recommendation is for
site designers to focus on making the long way -- the links of the
site -- work effectively for users.
- o - o - o -
Over the next few weeks, we'll be publishing other findings from
our most recent study of web usability in UIETips, which you can
subscribe to by sending the word SUBSCRIBE in the body of a
message to UIETips-Request -at- uie -dot- com -dot-
Jared
==========================================================
Jared M. Spool User Interface Engineering mailto:jspool -at- uie -dot- com 800 Turnpike Street, Suite 101
(978) 975-4343 North Andover, MA 01845
fax: (978) 975-5353 USA http://www.uie.com
If you send me your postal address, you'll get
the next issue of our newsletter, Eye For Design.
==========================================================
