The search engine market, and the search engines themselves, have undergone huge changes recently, partially due to advances in technology, and partially due to the evolving economic circumstances in the technology sector. However, most are still using a mixture of the following criteria, with different search engines giving more or less weight to various criteria :

Title: Is the keyword found in the title tag?

Domain/URL: Is the keyword found in the address of the document?

Page text: Is the keyword being emphasised in some way, such as being made bold or italic? How close to the top of the text does it appear?

Keyword (search term) density: How many times does the keyword occur in the text? The ratio of keywords to the total number of words is called keyword density. Whilst having a high ratio indicates that a word is important, repeating a word or phrase many times, solely to improve your standing with the search engines is frowned on, as it is considered an attempt to fraudulently manipulate the results pages. This often leads to penalties, including a ban in extreme cases.

Meta information: These tags (keywords and description) are hidden in the head of the page, and not visible on the page while browsing. Due to a long history of abuse, meta information is no longer as important as it used to be. Indeed, some search engines completely ignore the keywords tag. However, many search engines do still index this information, and it is usually worth including.

Outbound links: Where do the links from the page go to, and what words are used to describe the linked-to page?

Inbound links: Where do the links from the page come from, and what words are used to describe your page? This is what is meant by "off the page" criteria, because the links are not under the direct control of the page author.

Intrasite links: How are the pages in your site linked together? A page that is pointed to by many other pages is more likely to be important. These links are not usually as valuable as links from outside your site, as you control them, so more potential for abuse exists.

So there you have it. As we said earlier, there are some minor variations as each search engine has its own approach, and its own technology, but they have more similarities than differences. It is worth remembering that this applies only to crawling search engines, that use automated programs to gather information. Directories such as Yahoo! or the Open Directory Project work on a completely different principle, as they are human reviewed.

[back]