Date: Mon, 11 Apr 2011 09:39:06 -0700
Reply-To: DCHAS-L Discussion List <DCHAS-L**At_Symbol_Here**LIST.UVM.EDU>
Sender: DCHAS-L Discussion List <DCHAS-L**At_Symbol_Here**LIST.UVM.EDU>
From: Dave Einolf <dave**At_Symbol_Here**ENDEAVOUREHS.COM>
Subject: Re: Alertbox, April 11,
2011: Incompetent Research Skills Curb Users' Problem Solving
In-Reply-To: <AFA534E2-0B74-4A70-9443-E5FC28000CBF**At_Symbol_Here**>

It's Dr. Google here at my office.  ;

On Apr 11, 2011, at 9:27 AM, Ralph Stuart <rstuartcih**At_Symbol_Here**ME.COM> wrote:  Alertbox  April 2011 Users' Search Skills  
Jakob Nielsen's Alertbox, April 11, 2011:
Incompetent Research Skills Curb Users' Problem Solving

Users increas ingly rely on individual pages listed by search engines instead of finding b etter ways to tackle problems.
Although some analysts que stioned the finding of search dominance, it's a user behavior that gets stro nger every year. Today, many users are so reliant on search that it's underm ining their problem-solving abilities. Ironically, the better search gets, t he more dangerous it gets as people increasingly assume that whatever the se arch engine coughs up must be the answer.

One recent study participant referred to "my old friend Google" as t he place to go when given a task =E2=80=94 a remarkable indication of how cl osely people are tied to search these days.

During our user testing in Asia-Pacific last month, I watched use rs conduct more than 100 searches for a broad range of tasks. Only once did I see a user change strategy.

Give n the rarity of strategy shifts, we'd need much more data to precisely estim ate how often they happen. In this round of user research, our goal was to u pdate the Fundamental Guidelines for Web Usability seminar, so we focused on website design, not on search engine statistics.

Still, the rough estimate from our available data is obviou s: users change search strategy only 1% of the time; 99% of the time they pl od along a single unwavering path. Whether the true number is 2% or 0.5%, th e big-picture conclusion is the same: users have extraordinarily inadequate r esearch skills when it comes to solving problems on the Web.

In our study, for example, an interior decorator indiscriminately entered queries into any text box that caught her eye, wit h no understanding of which search system she was using or whether it was se arching the entire Web or only the site she was on.

< /div>
This example offers a striking case of confused mental mo dels. It also highlights a big problem with search today: it doesn't facilit ate any conceptual knowledge because it relies on quick in=E2=80=93out dips i nto websites.

Changing Research S trategy: An Example

Our recent re search yielded only a single positive case study. In it, our test participan t was trying to discover whether a friend had a cold or influenza given vari ous symptoms (such as a sore throat).
At first, the user s earched for symptoms, describing them in various ways. These simple query re formulations don't count as a strategy change because they were essentially v ariants on a single approach. (Without watching hours of video, I can't say e xactly how many users in our study changed the phrasing of their general que ries, but it was fairly common =E2=80=94 about 10=E2=80=9320% of the time.)< /div>

Searching for symptoms was a sing ularly unfruitful approach. Our user was swamped with a progression of quack sites describing various superstitions and homemade remedies. Most of these were quite well-meaning discussion groups and patient support sites, but th e content definitely didn't represent current medical science or trustworthy advice.

It's sad to think of the vast number of patients who get misleading medical guidance from the Intern et because the main search engines currently prioritize popular sites instea d of useful ones.

After a while, t he user realized he was getting nowhere by searching for symptoms. He thus r eversed his research strategy and started searching for the diseases, hoping to identify and differentiate symptoms between the two. This was much more s uccessful; he found several reputable medical sites with decent symptom desc riptions.

Advanced Search: Not Us ed

Another test participant was a lawyer who was preparing a presentation about the implications of a controv ersial court decision that had been handed down a few months prior to the st udy session. His goal was to find out what other experts had said about the d ecision.
Searching for various keywords that described th e case, the lawyer easily found many sites with pertinent information, inclu ding news media coverage, blog discussions, and whitepapers from other law f irms. However, almost all of these were written when the ruling came out and contained no analysis of the decision's long-term repercussions. They basic ally stated the decision and why it was good and/or bad.

Although most users never go beyond the first search engine results page (SERP), our poor user waded through many pages of SERP l istings, demonstrating a valiant desire to find newer coverage.

Considering that his main criterion was recenc y, our user could have chosen a much easier approach: using an advanced sear ch to filter the results by date. However, he never did so. (Remember, this w as a lawyer =E2=80=94 a highly educated person who regularly manages large a mounts of information. Average users would have been in an even worse situat ion.)

In general, we almost never s ee people use advanced search. And when they do, they typically use it wrong =E2=80=94 partly because they use it so rarely that they never really learn how it works.

The lessons are cl ear:

Don't assume that advanced s earch will help your website; you might build such features, but people will use them only in exceptional cases.
Spend the vast major ity of your resources on improving regular search (simple search).
One-Track Research Strategy: An Example

The sidebar details an example of a user expending substant ial effort with little result because she didn't modify her research strateg y. The user racked up 22 pageviews across 8 different sites (including the s earch engine) trying to find the most populous city in the world. She did fi nd an answer, but decided on it for the wrong reasons and without understand ing the underlying problem =E2=80=94 that there are two ways of counting cit y populations: with and without suburbs.
This outcome is a ll the more striking because the user was a schoolteacher who emphasized the need to teach students to critically evaluate online information sources.

Some users simply take the first a nswer they stumble across and run with it. But more careful users =E2=80=94 l ike the teacher in this example =E2=80=94 usually end up spending more time w ithout much more benefit because they're limited by the search engine result s.

After finding several widely d iverging estimates of "biggest city" (ranging from 12 million to 34 million p eople), it would have been reasonable to change the research strategy and tr y to find an authoritative site on the topic of urban populations. Such a sh ift would likely have provided more insight than relying on the simplified l ists posted on many sites that specialize in other topics and don't explain h ow they derive their data.

Search Is Too Good

The problem in the a bove examples =E2=80=94 and for many other users in our recent tests =E2=80=94 is that search engines are turning into "answer engines." Users are being t rained to limit themselves to pages included in the SERP listing. Indeed, fo r many problems, the actual answer is right there. But the concept that you m ight have to sometimes go beyond search listings is getting lost.
For many problems, there are better approaches than simply scrolling to the bottom of the SERP. You might, for example, try to locate a site tha t specializes in the problem. Or =E2=80=94 as in our cold/flu symptom proble m =E2=80=94 you might simply change the way you think about the problem.

Sadly, when one approach is so easy ( and works much of the time), users never develop the research skills needed t o try or even consider other approaches.

What can we do about this?

For today's Web design projects, we must design for the way the world is, n ot the way we wish it were. This means accepting search dominance, and tryin g to help users with poor research skills. For example, sites listing city p opulations could state explicitly that there are two ways to estimate popula tion, rather than simply offer a single estimate without further explanation . And proper medical sites could design pages for how patients search for in formation, rather than for how doctors think about it.
In the long term, we should try to improve the world rather than design to sui t its shortcomings. One example of how we might do this is to teach better I nternet research skills in schools.
Learn More

Mental models and the design lessons from ps ychology will be discussed in the full-day seminar on The Human Mind and Usa bility at the annual Usability Week conference.
The confe rence also has a 2-day seminar on Writing for the Web. The seminar on Web Pa ge Design discusses how you can help users who arrive through deep links ins tead of through your homepage.

&g t; Other Alertbox columns (complete list) 
> Sign up for newsletter that will notify you of new Alertboxes
Copyright =C2=A9 2011 by Jakob Nielsen. ISSN 1548-5552
< br>

Previous post   |  Top of Page   |   Next post

The content of this page reflects the personal opinion(s) of the author(s) only, not the American Chemical Society, ILPI, Safety Emporium, or any other party. Use of any information on this page is at the reader's own risk. Unauthorized reproduction of these materials is prohibited. Send questions/comments about the archive to
The maintenance and hosting of the DCHAS-L archive is provided through the generous support of Safety Emporium.