Search Engines
Search Engines and How Engines Rank Your Site:
Search engines and Web directories are comparable to digital card catalogs, but the organization and presentation of information isn’t a precise system; there are no librarians inserting new sites in alphabetical order. Instead, the process involves digital spiders or even humans scouring millions of Web pages and attempting to categorize the information, coupled with software that uses a complex set of criteria and algorithms to try to determine the most relevant sites based on keywords input by the user.
The current state of search engines can be compared to a phone book that is updated irregularly, is biased toward listing more popular information, and has most of the pages ripped out. But despite their shortcomings, search engines do serve a vital role on the Internet; they’re often the first tools users use to locate specific information.
While search engines may not always be the most up-to-date resources, they are still the best tools for finding information on the Internet. Whether you’re searching through information in a library or online, finding exactly what you’re looking for is going to take some work; you won’t walk into any library to find the best books open to the pages that contain the best information. Keeping track of the entire Web may be an impossible task even with mechanized software help. Studies in recent years have indicated that today’s popular search engines index only a fraction of the material that currently exists on the Internet. Given the ever-changing state of the World Wide Web, the information in a search engine’s index can be quickly outdated, so a spider’s work is never done. A spider must return to sites at regular intervals to keep up with changes to pages already contained in a search engine’s index and to add new pages as they come into existence. As a result, index updates take more time as a spider’s work beat grows larger.
Directories, Search Engines, and Combinations:
A Webmaster and Web surfers should know the difference between these types of sites, because they are often misused and confused.
1. Directories:
Directories are created by a
person who actually profiles your submission, as opposed to a search engine
which uses a "robot" or "spider" to do this. Sites are submitted and then
assigned to an appropriate category or categories. Because someone is
actually profiling your site, rather than a robot, directories often
provide much more targeted results than search engines. For example: Yahoo!
is a directory.
2. Search Engines:
Commonly referred to as
"spiders" or "crawlers," search engines are searching
the Web for new pages at all times. Because they are automated and index so
many sites, search engines may often find information not listed in
directories. To the flip side, they may also pull up unrelated information
for the topics you are searching for.
3. Combination Search Engines:
Many of the medium sized
engines have helped confuse the issue of what the difference is between an
engine and a directory by combining the two. These are search engines that
also work in conjunction with a directory. Usually, these include reviewed
and rated sites.
When you go to an engine and plug in a
keyword, subject or title, that search engine parses millions of documents
and almost instantaneously puts those pages in an "order of
relevancy." While search engines are very quick, they are not known to
be 100% accurate in their retrievals.
Engines and directories often times bring irrelevant pages into the
results, meaning it may take a while longer and more investigation to find
what you are looking for. Overall, search engines do a tremendous job in
pulling up relevant information, and getting your site listed in the top of
these engines can mean tremendous results for your site.
Search engines do not have the ability to ask questions, so they rely on
what you have entered for your search. While this may be changing with the
likes of intelligent agents, do not expect the same kind of customer
service you might find from your local librarian.
There are a number of ways that search
engines pull up your pages. The first thing almost all engines look at is
the Title of a site. If the keyword is found in the title, it is considered
to be more relevant and will pull up more quickly than a site without the
keyword in the title.
Search engines will check to see if the keywords you have entered appear in
the top of the Web page, like in the headline or in the first few
paragraphs of text. Engines will assume that if the topic is important, it
will be mentioned within the first part of your site.
Frequency will also factor in how search engines determine relevancy. A
search engine will determine how often keywords appear in relation to other
words in a Web page. Those with a higher frequency are often deemed more
relevant than a page with less frequency.
Note: some search engines index more Web pages than others. This means that
no two engines will bring up the same pages. Each has their own style of
pulling up information, resulting in different information being considered
more relevant.
Some search engines also give Web pages a bonus for various reasons. For
example, WebCrawler uses link popularity as part of its ranking method. It
can tell which pages in its index have lots of links pointing at them. These pages are given a slight bonus during ranking, with the reasoning
being if a page has a lot of links to it, it is probably a very popular
page.
Some combination type engines, those containing directories, may give a
bonus to sites they have reviewed. The reason being if the site was good
enough to earn a review, chances are it is more relevant than an unreviewed
site.
