Lesson 9 Specialty search engines

It would be nice if the only search engine we'd ever need were Yahoo and Altavista. However, there are many things these sites don't do. We need to supplement our Web search strategy with a few other tools. I call these specialty search engines. Specialty search engines will often find infomation that is not in key word search engines.

We'll go through a couple of the major specialty search engines but this lesson is not meant to be all-inclusive. There are literally thousands and maybe more specialty search engines out there. You can use the tools you've already learned to find some of them.

Here are some of the major categories of information you can search using these tools:

Usenet or newsgroups are like large bulletin boards where users can post messages on a myriad of topics. Each "new group" is a dedicated bulletin board to a particular topic. There are tens of thousands of different news groups ranging from the useful to the obscene.

Usenet, like Web pages, is a separate protocol running on the Internet. They use a news server that acts much like a Web server, serving up news or Usenet groups to the public. Instead of a Web browser, you need a news reader to read them (though they are accessible through some email packages as well). Internet service providers will make some or all of the groups available to you. You are restricted to seeing only those sites your providers makes accessible to you.

News groups are not available through your browser. However, there are a few gateways on the Internet that let you view them. One such gateway is Deja at http://www.deja.com/usenet.

News groups are titled in a way that roughly makes sense though the headings often are ignored. The convention is:

alt	alternative - the weird stuff
news	Usenet stuff
rec	recreational, sports, hobbies
biz	business -- including a lot of computer business stuff
comp	computers, hardware, software, etc
k12	educational topics on just about anything
soc	society and culture
humanities	humanities, literature and the arts
misc	the catch-all
talk	discussions, debates and yakking
sci	science

A typical group might be soc.culture.albanian where people discuss Albanian society and culture. Deja provides a catalog-style index to find groups on your topic of interest.

People post a message to a group. Other people will respond. Others will respond to a response creating a thread of messages that looks something like this:

You can search these threads by author, subject, Usenet group or by any word that shows up in the message using Deja's power search. Once you get a result, you can either:

When you are reading a posting, if you click on the tiny word "thread" in the upper left corner, you can view see the whole thread...an important option if you're trying to figure out who wrote the initial posting.

Note that each of the individual postings can only be accessed through Deja and not through Altavista. The postings are stored on a news server somewhere and not on a Web page so Altavista never sees them.

Be aware that email addresses are easily spoofed so never quote someone from a Usenet group. Get their name and number and call them directly. You never know who people really are behind the keyboard.

"On the Internet, nobody knows you're a dog."(Peter Steiner, The New Yorker, 1993)

Listserv and Majordomo are two email programs that allow you to creat a Usenet like bulletin board using you email. Subscribers join the listserv and when they post a message to the list, everybody who is subscribed receives a copy.

Generally, these groups are used by hobbyists, professionals and others to discuss topics of interest. I'm a member of the Investigative Reporters and Editors list that discusses topics related to investigative reporting. To find discussion groups, try these sites:

Discussion groups are very useful for finding experts. While Usenet groups are considered "seedy", email discussion groups are quite common and many professional groups have their own lists. However, since they are not related to the Web (it's an email protocol), it's sometimes hard to find them and you must use word of mouth. Some groups might archive their lists to the Web or have information on how to sign up. Be aware that many of these are moderated (private) lists and you must be approved to sign up.

Remember that key word search engines sometimes take months to get around to sites. Therefore, daily news on the Web can't be accessed via a search engine. Here are a few sites that index news sites every day:

Like Usenet groups, File Transfer Protocol sites use a different protocol from the Web. These sites usually include things such as sound files, graphics, large sets of data and other things that don't lend themselves easily to Web pages. You can search FTP sites here: