The Distributed Yahoo: 'NewHoo'

A new, volunteer-driven Web directory service hopes to tap the spirit of open development to build a better Web index. With 1,200 editors signed on so far, NewHoo thinks it can use the power of the Web to make a more complete directory. By Chris Oakes.

What do you call yourself if you're out to do what Yahoo does, only better? NewHoo, of course.

That's the name of a new Web directory that aims to build the largest, most active Web directory "on the planet," in the words of NewHoo co-founder, Rich Skrenta.

A lofty goal, but the group thinks it's marshaled the forces to accomplish it: lots of editors, from all over the Web and the world, collecting sites in the same spirit of open software development used by Mozilla.org and the Apache Web server.

Like Yahoo (YHOO), NewHoo is a directory-based index of Web sites, whose link listings are selected and categorized by human beings. But in contrast to Yahoo's staff of in-house editors, the service is maintained by volunteer indexers from around the Web.

NewHoo's goal is to become the largest and most comprehensive directory on the Net, and it hopes to meet it with people power. To become a NewHoo editor, a person just chooses a topic and joins at the NewHoo site. The number of Web-combing editors resulting from this approach will let NewHoo surpass Yahoo in indexing efficiency, its founders hope. Indeed, as the Web has continued its massive growth, Yahoo has been criticized for falling behind, not doing enough to add new sites or modify existing ones.

NewHoo reported today that since going live about five weeks ago, it has registered 1,200 editors working the Web by subject. Right now, says Chris Tolles, another NewHoo founder, the index includes over 40,000 sites.

"I can't give you a deductive reason as to why this works," Tolles said. Whether it has to do with aspects of human psychology or sociology, he can't say, "but it just worked." Tolles thinks it may be based on a "natural librarian in people that wants to straighten things up."

Tapping this human impulse, Tolles said, is the only way to deliver a large-scale categorization of the Web that people want. In its first five weeks, news of the service has spread only by word of mouth and a few brief mentions in Internet publications. This only goes to demonstrate, Tolles said, that NewHoo's volunteer, distributed approach has struck a Web nerve.

The benefits of the distributed, volunteer effort of human indexers is already evident in certain NewHoo sections, such as the site's section covering the thyroid gland. "Ours is four times as big [as Yahoo's] because the guy who edits it is fantastic."

Regional content areas are another category showing NewHoo excellence, Tolles believes. "Instead of some guy in San Mateo figuring out what's going on in France, Luxembourg, or Poland, there's finally a place on the Web where regional content is controlled by the people who live there."

People have joined the effort, he said, "saying I have encyclopedic knowledge of 'X' and I want to put that knowledge where people can see it."

The notion of topical enthusiasts picking relevant sites also describes the approach of the Mining Company. But Mining Company spokeswoman Amanda Hass was quick to point out what she sees as key differences between the two services.

"The 'V' word tips you off," Hass said. "Our guys are not volunteers. They have revenue at stake." The 600 specialists building the Mining Company's subject categories, she said, receive a share of the service's profit, while the selection of Mining Company "Guides" is rigorous. Only about half of the applicants are accepted, she said.

"Our cancer guide," she noted as an example, "is a radiation oncologist who is also very Net savvy." Plus, Hass said, all guides do more than index -- they produce original content related to their areas of expertise.

Tolles says the primary difference between NewHoo and the Mining Company -- outside of the fact that the Mining Company does more than sheer indexing of sites -- is a question of scale, since NewHoo already outnumbers the service in editors. "We'll have a much larger [editorial] base."

Srinija Srinivasan, Yahoo's director of surfing, thinks that the fledgling project is "an intriguing idea, but not a new idea. Anyone that can help users navigate the Web is good for users and good for Yahoo because it increases the substantive quality of information and navigation out there."

But Srinivasan thinks that her company's estimated 70 surfers have an advantage over the specialized approach of NewHoo and others. "We go out of our way to hire a diverse and eclectic team. At heart, they need to be a generalist and understand a wide range of things."

"But as we evolve, the trick is to find a diverse team of generalists with specialties across the board," she added.

Yahoo and LookSmart are the only other Web directories seeking a comprehensive Web site library that is indexed by human hands.

But as Web growth explodes, critics have charged that up to a third of the sites seeking a listing in the popular Yahoo directory don't make it in. The company has admitted that some sites may take months, even years, to get listed at all.

Srinivasan said earlier this year that the company's process is not based on one single order of business. Instead, Yahoo is a "media company," she said, "aggregating information that we think is of use." Yahoo's team of surfers, far from attempting to process the tens of thousands of weekly submissions, "aggregates" information -- which means doing whatever it can, within reason, to keep abreast of "what's happening [on the Web]," in Srinivasan's words.

Danny Sullivan, editor of Search Engine Watch, said earlier this year that Yahoo lists as few as a quarter of submitted URLs, in his estimation.

It is NewHoo's mission to beat Yahoo at its own game, and the key to that is sheer numbers.

Meanwhile, Sullivan said that the biggest questions for NewHoo is whether or not it will sustain itself and whether people will use it. "You have to actually attract people. Usually when a new site gets launched there's a bit of buzz about it but then people forget about it."

To be successful, he said NewHoo will have to do what the Mining Company has shown itself able to do: Come back after the initial excitement wears off and "go out and publicize ourselves and make it work.... They've said open source [software development] is their inspiration, and that certainly is a part of it."

"It will be very interesting to see whether or not it does attract grass roots support." But meanwhile, Sullivan likes NewHoo's spunk. "It's a very good idea in the spirit of what the Web used to be."