Making Your Pages Easy for Search Engines to Index (2)

Search engine spiders want to read the text on your pages, and especially the introductory text near the top of the page. This mirrors the way human beings assess pages — by reading them, starting at the top.

Here are some guidelines to keep text-hungry spiders happy:

1. Provide text. Pages without text rarely gain high rankings. This is especially important for home pages. If there's no text on the opening page then the spider might stop right there and not even bother to look at the rest of your site. It's one reason for avoiding Splash pages at the front end. Ideally you should provide at least 150 words of text on your home page.

2. Make full use of early paragraphs to include relevant keywords. Most search engines place emphasis on early text, and less on the words further down the page. The numbers vary from engine to engine, but you can assume the first 50 words are crucial, the next 50 are important, the 50 following are likely to be read. After that, it's anybody's guess, though some engines do manage to fully index pages with more than a thousand words. Try to get your important keywords — the expressions you expect your visitors to use in their searches — included in your first 150.

3. Don't overdo any repetition. If you repeat your keywords too often, you could be penalized. There's no magic number to aim for, but if you repeat keywords three times or less, you should be safe.

4. Concentrate on the main text. You might have a separate top table (perhaps containing an advert and logo) plus a left hand column with links. These will appear in the HTML file before your main, central text block. There's a temptation to think these areas are more important than the main text area because spiders read them first. If these outlying areas contain a lot of text (unlinked) then this may well be true. But many engines try to ignore peripheral HTML blocks, especially if they're heavy on links, and head straight for the center. It's not too difficult for them to do. They simply look for the largest title (within <h> tags) on the page and assume that whatever follows that is the most important text area.

5. It's not much use getting your keywords in the right place if you've chosen the wrong ones. It doesn't help the spiders either. They'd prefer you to choose the right keywords so their indexing works as intended. It's worth spending a few hours on deciding your keywords, maybe trying out a few expressions in the search engines and seeing if they deliver the sites you want to compete with.

6. Spiders have lists of stop words — mainly related to adult content and profanity. When they find one of these words they may abandon your site altogether. If you have a page that includes a possible stop word, hide it from spiders by making it an exclusion in your robots.txt file (see later). Also watch out for words that have two meanings, one of which is sexual. Spiders don't understand context.

7. If you have pages full of links, make sure there's plenty of text to accompany them. Pure link listings are often ignored by spiders, but if you add a couple of sentences describing each link, the problem disappears.

Popular Sites are Exceptions

Often you can learn a few tricks by looking at the most popular sites on the Web and seeing how they do things. But not in this case. The most popular sites are given a special status by search engines and indexed under slightly different rules than regular sites. They are more likely to be indexed thoroughly and frequently, which means they don't have to try as hard. Also, because it's assumed they won't try to spam the engines, they're forgiven the occasional mistake, such as overusing a keyword.

Titles and Filenames Count

Spiders like to see useful page titles, and some also appreciate relevant filenames. It helps them, but unfortunately the mechanism has been abused, so they're wary. Try to use filenames and page titles that match your text content and keywords, rather than using them to cover keywords that don't otherwise get a mention. Words in filenames can be separated by an underscore — this is a convention that IT professionals used before the Internet arrived, so it's perfectly acceptable. But if your filenames turn into a long sequence of keywords, spiders will assume you're trying to spam them.

Meta Tags

These go in the file header, in two sections — keywords and description. The meta tag system has been so heavily abused that some engines simply ignore them. But it's still worth spending a few minutes on creating them for the engines that remain interested. Keep them short and don't use words that are missing from the main text. If you spend a long time working on meta tags, you're probably trying to manipulate the system and you may well be found out. Create them quickly, using the simplest, most obvious content, and it's more likely they'll work as intended.

READ MORE