Home 

Attracting Traffic Through Internet Search by Richard Seltzer

Articles which appeared in Internet Search Advantage (formerly known as Power Searching with AltaVista), a monthly newsletter published by The Cobb Group, a division of Ziff-Davis

Use Content to Attract Traffic to A Web Site

Content can drive traffic to a Web site by way of search engines. But many well-established and expensive sites are designed in such a way that search engines cannot see the text on their pages. In such cases, simply registering pages with search engines accomplishes nothing.

My approach is intended to make visible the content that you already have so search engine users who are interested in that content can find you -- and to accomplish that without affecting your current, branded site design.

First, let's take a closer look at the typical problems -- what kind of page designs get in the way.

Then let's look at the faulty solutions typically proposed by search optimization companies -- such as metatags and doorway pages.

Then let's look at the best and simplest solution.

Unfortunately, branding rules and politics may prevent you from solving the problem the simple way.

But there are workarounds that allow you to leave your corporate design as is and make your pages more visible to search engines without violating search engine rules.

Typical Problems

Many sites present dynamic pages -- generating pages on-the-fly from databases. Such pages typically have a ? in the URL, which serves as a stop sign for search engine crawlers. These crawlers need to avoid being trapped at dynamic sites, which could generate huge number of pages, clogging search engine indexes with useless content.

Other sites use javascript in such a way that very little text is visible to search engine crawlers. Even the links to other pages at your site may be buried, so that if a crawler finds one page at your site, it can't follow a trail of links to discover the rest of your site.

Other sites use frames or tables, which while not blocking crawlers, wind up confusing search engine users. For instance, when a frames page is indexed, each window is indexed separately, so someone finding a match and clicking on it will be presented with that window alone, out of context. And when pages dependent on tables are indexed, the words are interpreted as appearing in sequential order, left to right, instead of associated by columns and rows. As a result, phrases get jumbled.

When companies realize that their pages are poorly represented or not at all represented in search engine indexes, they typically target the symptoms rather than the cause of the problem.  They'll set measurable goals that are irrelevant to their true business needs -- such as "ranking" for specific key words -- and then hire experts who try to trick search engines into delivering those the desired results. In the process, the "experts" may  break the rules that search engines have set up to try to keep their indexes truly useful, and thereby get the company's pages completely thrown out. In any case, the experts typically have their hands tied -- they are unable to add new useful content to the site and are unable to change the site's basic design. Hence they often propose adding metatags to existing poorly designed pages and/or creating doorway pages.

Metatags


There are two kinds of metatags that matter to search engines. The description metatag indicates the text that should appear in a search engine results list. The default is the first couple lines of text. If that text is nonsense -- random words that happen to be associated with graphics -- or if there is no plain static text at all, because of the way the pages are generated -- a description metatag might be useful, but it's just a band aid for a problem that you created for yourself with your page design.

Keep in mind that the two most important parts of a Web page for purposes of search engine ranking are the HTML title and the first couple lines of text. Yes, the description metatag can give you a coherent description in results lists, instead of gibberish; but the first lines of text still retain their priority for ranking. In general, search engine ranking algorithms pay little of no attention to metatags, mainly because they are very vulnerable to "spam". Search engine companies want to ensure as much as possible that what actually appears on the page is a good match for their users' queries; and they trust the actual static words on the page far more than anything that might appear in metatags. So a page that has no description metatag and has a good clear description in the first couple lines of text is likely to do better in ranking than a page with those very same words presented in a description metatag and little or no static text on the page itself.

The second kind of metatag is called "keyword", a term that confuses many people, leading them to think in terms of databases. Many presume that search engines index keywords, and hence that the keyword metatag is very important. In fact, only two of the major search engines, Inktomi and AltaVista, pay any attention at all to keyword metatags. And AltaVista gives plain text on a Web page higher priority than anything in keyword metatags, for purposes of ranking.

Today's search engines in fact index every word on every page. And many search engine users enter multi-word queries. So the more useful text you have on your pages, the more likely it is that searchers who want your kind of information will find you. Instead of spending time and money generating key word metatags, you should add more and more useful content to your site.

To get a concrete feel for how this works, try a high end web traffic analysis program, like WebTrends Log Analyzer. (You can download it for a 30-day free trial). Such a program will let you see how people come to your pages -- showing not only the volume of traffic from each of the major search engines, but also the queries that people used to find you. You are likely to be amazed at the variety and the detail of the queries. Very few people enter single word queries -- and those who do are not likely to be good prospects.

To get a sense of the typical behavior of search engine users, check stats generated for my site using WebTrends on one particular day in January. Go
to www.samizdat.com/jan25/jan25.htm [no longer online] In the left column, click on "Referrers and Keywords". Then click on "Top Search Engines", "Top Search Phrases", and
"Top Key Words". The key words are relatively useless, while the phrases are rich and informative.

Keywords only matter for advertising -- search engines will sell you ad space on pages generated when certain words appear in the query. But for actual searching, keywords are meaningless. Put your effort into generating more good content.

Also don't waste your time with key-word position checkers -- programs, like WebPosition Gold that tell you how your whole site or particular pages rank for particular queries. Such programs bang away repeatedly and automatically at search engines, adding an enormous load to those systems and hence slowing response time for actual users and forcing the search engines to invest more to keep performance at acceptable levels. While the results you get from such programs might make you look good to your boss, they mean little or nothing in terms of how much traffic your pages are likely to get by way of search engines. For traffic, you need content, and lots of it.

Doorway Pages

Some search engine optimization companies will propose creating "doorway" pages for your site.

A doorway page consists of artificially generated content, designed to emphasize pre-selected keywords. The page itself is meaningless. Its sole
purpose is to fool search engines, with the objective of coming near the top of lists of matches when queries include those key words. In fact,
search engines strongly discourage that practice, and when they detect it, many will blacklist those pages and sometimes even all pages from the same
site -- kicking them out of the index.

You might think, "Why worry? There's more than a billion pages out there. How will anyone ever know?"  As was evident in presentation after presentation at a recent search engine conference in Boston, your competitors and the people hired by your competitors to design their sites watch what you do very carefully. They understand these techniques and how to spot them. And when they spot them, they blow the whistle -- loud. Search engine folks are inundated with alert messages about wrong doing, sent by competitors of the companies that are using those techniques.

Also, doorway pages typically reside on the servers of the "search optimization" companies that produce them. They automatically
redirect traffic to actual pages at the customer's site. That means that the customer receives more traffic, but to the search engine the traffic appears
as if it is going to the optimization company. So when the search engine calculates the "popularity" of pages for purposes of ranking, the customer
site gets none of that credit.

In other words, many people go to great lengths and great expense to try to fool search engines. But the risks are great -- not just annoyed search engine users, but your sites becoming blacklisted.

It is far less expensive and far more effective to write good, useful content and present it simply so it can be properly indexed.

Do It Right

Since search engines index every word on every page they find, the more useful text you have at your site, the more likely your pages will be found by people who are interested in them. This is a random game -- the more content you have, the more dice you throw, so the more likely you'll win.

Focus on building content, not on trying to trick search engines.

Large pages are more valuable than short ones -- large in terms of text, not graphics. Graphics are useless for search engines.

For maximum effectiveness, you need plain static HTML pages (not ASP pages), without frames or tables or java applets.

The most important text should appear at the top of the page. In fact, the first couple of lines of text should make sense as a description of the page.

And the HTML title (not the file name, and not the headline that appears on the page, but rather the title in the HTML header) should be carefully written to mention everything that is important about the page, in very few words. That title will appear in search engine results lists as the words that are linked to your page. And words that appear in HTML titles are typically given very high priority by search engines -- in other words if two pages match a given query and one of those pages has the query words in the HTML title, that is the page that will appear on top.

These static pages can have static graphic images (jpg or gif), if you like. Such images will not help you with search engines, but won't hurt you either -- so long as none of the text is embedded in images.  

Corporate Branding Rules

In some cases, a simple text-heavy design goes counter to the aesthetic taste of the Web designers or management. More often, corporate branding rules get in the way.

Even at Digital Equipment, the company that invented the pioneering search engine AltaVista, corporate branding rules got in the way of making the company's pages findable by search engines.

Typically, the branding folks come up with a "template" based on what they have done in the past with their print brochures and without realizing they are costing the company traffic and hence revenue with such rules.

Branding rules, which are intended to present a consistent corporate visual image to the world, were often created before the Web or by people who had no real knowledge of how the Web works. Once in place, such rules can make it very difficult to do what is necessary to use content to attract traffic to a Web site by way of search engines. And bureaucracy often makes it very difficult to change such rules once they are in place.

Basically, there are two parallel design goals when building a Web site.

1) Design for the optimum user experience at the site, with a consistent look-and-feel that follows corporate branding standards.

2) Design for optimum traffic and revenue growth by making your content clearly visible to users of the major search engines.

Far too often, large companies put all their emphasis on the user experience at the site and forget the search engines which are necessary for getting people to the site in the first place.

Only people who find your site can appreciate the experience.

You need to take steps to draw more people to the site. But the design that optimizes the user experience will probably get in the way of search engines, and hence cost your traffic and business.

The Mirror Page

If you have no chance of converting the branding and marketing folks from their current search-engine unfriendly site design, consider creating mirror pages.

This approach keeps your current branded design in place, but you can still take full advantage of your text content to attract more visitors and hence generate more business.

Brand plays an important role on the Web -- for existing customers and partners and for users who have found your site and want to navigate through it for further information.

But to attract new prospects using search engines, you need to take a different and parallel approach that emphasizes text.

Your mirror pages have the exact same text content as your standard pages, but they are presented in static HTML, with meaningful HTML titles and with the first couple lines of text written so they can serve as a description. They have no metatags. They have a minimum of graphics.

If the standard pages are very tiny -- forcing the visitor to look at a whole series of pages to read or print what is really a single document -- the mirror pages will consolidate that text (serving at the same time a "printer friendly" version).

Each of these pages has links to your main site, with an explanation that people should go there for the optimum experience -- with all the graphics and dynamic effects. But people who were looking for specific information and found this mirror page by way of a search engine will be quite happy because you have provided those people with exactly what they were looking for. There are no links from mirror pages to other mirror pages, and no links from standard pages to mirror pages.

Static HTML pages attract traffic by way of search engines and provide new visitors with the information they want, very efficiently.

Build A Sitemap

Also create a sitemap page that consists of a hyperlinked list of all your mirror pages. You submit that sitemap page, not your home page to the search engines.

Search engine crawlers follow a trail of links to "discover" the content on the Web. If your pages today are not well represented in search engines, chances are good that you are using a technique that either halts crawlers (e.g., dynamic pages with question marks in the URLs) or that present links to other pages at your site using a technique that makes those links  invisible to crawlers, such as dropdown menus. By sending a crawler to a static HTML sitemap with links to every one of the search-engine-friendly pages at your site, you make it easy for that crawler to find all of your content.

Also, search engine crawlers typically stop after going several layers deep into a site (following 2-3 links in succession). (Last I heard, Lycos only
went one layer deep). If you have a sitemap with links to every mirror page, and if you submit that page, instead of your home page, to the major search
engines, you make it so every page at your site is just one layer deep.
 

Comparing Doorway and Mirror Pages

Doorway pages are designed to fool search engines. Mirror pages are designed to make it easy for search engines to find and index your real content. The "key words" and "key phrases" that doorway pages focus on are of no real value. Search engines index every single word of every single page. And people who are looking for unique phrases that are matched on you pages are the very best prospects.  Your measure of success should be in terms of leads and sales, not in terms of "high rankings" for particular search words or hits or even visitors. People who are fooled into going to one of your pages by a phony doorway page and who don't see what they were looking for on the page they arrived at are worth nothing to you.

To put it another way, some schools prepare their students to pass a particular test; while others prepare them for success in the real world. Don't mistake the measurement for the reality. Don't be impressed by the "rankings" of competitors or by the "rankings" that search engine optimization companies claim they can deliver. Focus on your real business objectives. Don't try deceptive tricks. They backfire. Provide real content in a form that can be well-indexed and that is also immediately useful to your customers and prospects.

Your mirror pages have text and links to encourage visitors to go to your standard site. If those standard pages provide real benefit and you explain that benefit clearly, visitors should head there. Do not use automatic redirects and other cloaking mechanisms that take visitors to a page they didn't ask for That's "spam." Treat your visitors with respect, and abide by the sensible rules of search engines.

People arriving at mirror pages from search engines immediately see what they were looking for. Those arriving at doorway pages are automatically redirected to other pages where they typically do not find what they are looking for because the exact words of the query do not appear on the destination page.

You want to target search engine users who are actually looking for the content on your pages. The more pages you have with the full-text indexed,
the better your chances that those people with unique queries will find your pages.

Also, mirror pages reside on your site. Any traffic directed there by search engines counts toward the popularity of your pages and site when those search engines determine which pages should appear high on lists of matches.

Mirror Pages

Check www.jeremyjosephs.com  That is the site of freelance writer, Jeremy Josephs, in Montpelier, France. The standard site looks professional, but got zero traffic.

Then check www.jeremyjosephs.com/sitemap.html That will take you to his sitemap, with links to mirror pages for his site. Check a few of the links to get to plain-text search-engine friendly versions of his articles and books. By using this technique, he now gets over 1000 visitors/week.

Remember that your goal is not "maximum positioning" for your pages with regard to particular keywords. Your goal is not just getting more pages into search engine indexes. It's a question of traffic and new business.

Also, don't expect immediate results. It may take months before your new mirror pages get into the search engine indexes. Search engines vary as to how frequently they crawl and update their indexes, ranging from about a week with AltaVista, to 6-8 weeks with many, and up to 3 months with some. And if you trigger spam alarms with doorway pages and other tricks, it could take years to untangle the mess.

There are no links from your full-blown pages to the mirror pages -- only links from the mirror pages back. Hence only people arriving by way of search engines or by bookmarks/favorites are likely to ever find these pages. But since the pages reside on your servers and provide your content directly, issues of branding could still arise. These pages have a text-heavy look and undoubtedly will not conform to your corporate branding rules.

You need to explain that these pages are not part of your site, rather they are an add-on, intended to draw traffic, and that the corporate rules should not apply here.

Think of search engine traffic like word-of-mouth referrals to your company. You brand your brochures and your advertising. But you wouldn't consider branding word-of-mouth marketing.

Instead of asking a friend, a Web user turns to a search engine and asks -- "where can I get information about xyz and pdq?" The search engine then provides a plain-text results list -- that includes no one's logos, no one's branding. And when users click on links in such lists, they expect to go straight to pages that have the kind of information they are looking for. This is very different from calling up a sales person and asking to receive a brochure or a spec sheet.

Explain the difference between optimizing the user experience and optimizing search engine results. Try to find ways to work together to generate new business for the company. 

Workaround

What if you can't convince management to allow you to create mirror pages? Your pages are all dynamic with a ? in the URL, so crawlers come to a halt whenever they come to one of your pages, not following any trail of links. And your pages have no useful text is visible to search engine crawlers. Are you totally dead in the water?  Not necessarily.

In this worst case scenario,
1) Write useful informative HTML titles for each and every page (every one different)
2) Write description and keyword Metatags for each and every page (every one different, and without much repetition among HTML title, description Metatag, and keyword metatag).
3) Create a sitemap page -- a static HTML page that ignores branding rules: just a list of the HTML titles of all your pages with links (including the ?)
4) If branding rules prevent you from posting this sitemap page on your company's site, then post it anywhere else -- even on free Web hosting space.
5) Submit this sitemap page to the major search engines. They should eventually check every page linked to from that sitemap page. They won't be able to follow the links any further (the ? stopping them), but they should be able to capture the HTML title and, in some cases, the metatags -- so you could still end up with some information about all of your pages in the major search engine indexes.
6) Each HTML title and metatag should be no longer than 255 characters

But don't lose sight of the fact that if you are forced to proceed this way to help your company attract new prospects through search engines, your company has serious management problems. Keep kicking and screaming in hopes that eventually your message will be heard and the corporate rules change. And meanwhile, be sure to update your resume and post it on the Web.

Key Words

Last week, an old friend contacted me wanting help in getting the web pages of the company he works for indexed by search engines. Over the last three years or so, he worked for a couple search engine companies. Now, in a new line of business, his new boss wants him to help make sure that their site shows up better on searches for certain key words. My friend wasn't sure how to do that.

I took a quick look at the site, and immediately saw that it was over-designed -- that the page design was getting in the way of search engines ever seeing and indexing the content.

Content can drive traffic to a Web site by way of search engines. But many well-established and expensive sites are designed in such a way that search engines cannot see the text on their pages. In such cases, simply registering pages with search engines
accomplishes nothing.

Many sites present dynamic pages -- generating pages on-the-fly from databases. Such pages typically have a ? in the URL,
which serves as a stop sign for search engine crawlers. These crawlers need to avoid being trapped at dynamic sites, which
could generate huge number of pages, clogging search engine indexes with useless content.

Other sites use javascript in such a way that very little text is visible to search engine crawlers. Even the links to other pages at
your site may be buried, so that if a crawler finds one page at your site, it can't follow a trail of links to discover the rest of your
site.

Other sites use frames or tables, which while not blocking crawlers, wind up confusing search engine users. For instance, when
a frames page is indexed, each window is indexed separately, so someone finding a match and clicking on it will be presented
with that window alone, out of context. And when pages dependent on tables are indexed, the words are interpreted as
appearing in sequential order, left to right, instead of associated by columns and rows. As a result, phrases get jumbled.

When companies realize that their pages are poorly represented or not at all represented in search engine indexes, they typically
target the symptoms rather than the cause of the problem.  They'll set measurable goals that are irrelevant to their true business
needs -- such as "ranking" for specific key words -- and then hire experts who try to trick search engines into delivering those
the desired results. In the process, the "experts" may  break the rules that search engines have set up to try to keep their
indexes truly useful, and thereby get the company's pages completely thrown out. In any case, the experts typically have their
hands tied -- they are unable to add new useful content to the site and are unable to change the site's basic design.

Today's search engines index every word on every page. And many search engine users enter multi-word queries. So
the more useful text you have on your pages, the more likely it is that searchers who want your kind of information will find you.
Instead of spending time and money generating key word metatags, you should add more and more useful content to your site.

Keywords only matter for advertising -- search engines will sell you ad space on pages generated when certain words appear
in the query. But for actual searching, keywords are meaningless. Put your effort into generating more good content.

Also don't waste your time with key-word position checkers -- programs, like WebPosition Gold that tell you how your whole
site or particular pages rank for particular queries. Such programs bang away repeatedly and automatically at search engines,
adding an enormous load to those systems and hence slowing response time for actual users and forcing the search engines to
invest more to keep performance at acceptable levels. While the results you get from such programs might make you look good to your boss, they mean little or nothing in terms of how much traffic your pages are likely to get by way of search engines. For traffic, you need content, and lots of it.

Since search engines index every word on every page they find, the more useful text you have at your site, the more likely your
pages will be found by people who are interested in them. This is a random game -- the more content you have, the more dice
you throw, so the more likely you'll win.

Focus on building content, not on trying to trick search engines.

Large pages are more valuable than short ones -- large in terms of text, not graphics. Graphics are useless for search engines.

For maximum effectiveness, you need plain static HTML pages (not ASP pages), without frames or tables or java applets.

The most important text should appear at the top of the page. In fact, the first couple of lines of text should make sense as a
description of the page.

And the HTML title (not the file name, and not the headline that appears on the page, but rather the title in the HTML header)
should be carefully written to mention everything that is important about the page, in very few words. That title will appear in
search engine results lists as the words that are linked to your page. And words that appear in HTML titles are typically given
very high priority by search engines -- in other words if two pages match a given query and one of those pages has the query
words in the HTML title, that is the page that will appear on top.

These static pages can have static graphic images (jpg or gif), if you like. Such images will not help you with search engines, but
won't hurt you either -- so long as none of the text is embedded in images.

So, I told my friend, forget about key words. Instead, focus on generating content -- text that is useful to your customers.


Surviving as a Small Company in the Age of Google


(adapted from a speech delivered at NEXPO, the newspaper technology show, in Washington, DC, June 20, 2004)

Over the last ten years, the Web has evolved from a promising innovation to an essential part of life and business.  Today, hundreds of millions of users can instantaneously access billions of pages. Much has changed in that short time. Many businesses rose from nothing to become important forces on a global scale, and just as quickly vanished, to be replaced by new companies with new business models. In just one decade, we we experienced a tidal wave of change -- the equivalent of what previously might have transpired over the course of two to three generations. And just as that wave of change took us by surprise, we are now likely to be surprised by the absence of such a wave, or the contrast between the after waves and ebb tide and the momentous rate of change we just experienced. Many were mesmerized by rising stock prices of Internet stock prices and imagined that they would rise forever, and many came to expect that the overall rate of change in the world of business would continue to accelerate. But, no, unpredictability remains unpredictable. In the midst of change, stable patterns appear and persist. And in the midst of stability, change slowly swells, perhaps to fall again or perhaps to form another tidal wave.

In the days when the Internet wave was at its peak, AltaVista ruled as the premier search engine -- the best, the simplest, the most powerful way to find whatever you anted on the Web, whenever you wanted to -- helping many to find useful information and resources on the Web, when the Web was growing faster than it has ever grown and when it was at its most volatile and unpredictable.

Due to many issues, AltaVista, despite its technological excellence, failed as a business, as did its parent company, Digital Equipment. Digital was swallowed by Compaq, which was swallowed by Hewlett-Packard. AltaVista was spun off as an independent company, then swallowed by Overture, which was swallowed by Yahoo.

Today, Google reigns supreme as the simple, complete, powerful way to find anything on the Internet. And "to google" has become a common English verb.
But despite the vast changes in the business of Internet search, the basic principles of how search engines work and how to design Web pages so users will be likely to find them remain the same as they were in the days of AltaVista.

If you ;work for a large well-known, well-funded business, don't bother to read article. Such companies get lots of Web traffic from their existing customers, and due to their name recognition and advertising. They are big and important enough to negotiate their own terms with search engines. Small companies should not try to imitate the Web designs and Web business plans of their large competitors. Rather, they should seek low-cost and no-cost ways to build Web traffic, including using simple search-engine-friendly Web page design.

Web-Search Basics -- How to Be Found and What Gets in the Way
 
A search engine does not instantaneously check the entire Web whenever it gets a query. Rather it periodically sends out robot programs (crawlers) to gather content from the Web and stores that content in an "index". It then matches query words with words stored in the index. The freshness and the completeness of the information of the index differ from search engine to search engine and over time. In other words, the index is assembled mechanically and automatically, following programmed rules, not based on case-by-case human judgment. The credibility of search engines and the loyalty of their users depend on their reputation as fair information  brokers -- that the same rules apply to all, regardless of the size of the company that provides information on the Web, and without taking money as payment for inclusion in the index. Search engine companies that appear to breech that trust, like magazines that appear to sell their editorial content, put their user base and hence their business survival at risk.

Some, including Google, successfully perform the high-wire balancing act of accepting paid text advertising that is displayed when queries include "keywords" that advertisers have paid for. They are careful to display these ads in such a way as to distinguish them from regular search results, and they are careful to keep the underlying index free, complete, and without taint of paid inclusion. In fact, the text ads at Google are often closely targeted to the query words and often prove useful to users.  And small companies can and should use Google's Adwords (http://adwords.google.com) as an inexpensive way to attract potential customers, and should also consider becoming an "adsense" partner of Google (http://www.google.com/adsense) placing code on their pages, by which google-generated text ads targeted to the content of their pages appear unobtrusively on their Web and pages and they get paid when visitors click on those ads.

That's a quick and simple way for a small business to generate a little ad revenue from its Web pages, without having to spend any time trying to sell ad space.

In any case, rules are at the heart of the search engine business -- rules for inclusion in search engine indexes and rules for matching queries with information in those indexes and rules for determining the order in which results will appear. If you understand the basic rules and play by them, search engine users may come to your content pages in large numbers. If you don't play by the rules, you may be poorly represented or not represented at all.
Web-Search Basics: What's Strange About Search Engines
 
Yes, search engines, like Google, have separate indices for graphics. But basic Web search deals with text only --= no graphics at all.  THey could care less about your logo or photos of your products or how slick the graphic design of your pages is. Text is all that matters.
Yes, search engines will present information from your Web site in lists of matches to queries related to your text content. But they present that information in their format, not yours -- ignoring your branding rules. And if your page design includes tables or frames or java script, chances are good that the information may appear jumbled (in the wrong order) or truncated and incomplete.

Also, the search engines will match queries with specific individual pages at your site, rather than your home page. The user is their customer, not you the small business with information displayed on the WEb. Search engines will do their best to send their users straight to the information they want, rather than to the carefully constructed context that you built at your Web site by the way you organized your pages and links.
Also, search engines place value on and index best Web pages with lots of text information. Large text-heavy pages get more weight than small pages with pleasing graphic design. If search engine traffic matters to you (and it certainly should if you have a small business):

Because search engine traffic has become so important to so many businesses, search engines are under constant attack from businesses seeking to trick them and gain unfair advantage. In reaction to repeated attempts at trickery, today search engines typically ignore or give little credence to:

Search engines run special programs to try to sniff out common tricks and they then punish the perpetrators. For instance, they typically look for
pages that just redirect users to other pages, and other bait and switch techniques, and also content management software that seeks to identify the type of visitor (such as search engine crawler) and then presents information tailored for that visitor.
Low-Cost Ways to Get Search-Engine Traffic
If you value search engine traffic, you should make your Web pages as simple and informative as possible.

It doesn't pay to try to trick search engines. Their rules and policies change frequently, without notice. If you try to second-guess the details of how a search engine like Google works, you can waste a lot of work and money. Fir instance, you might be tempted to capitalize on the fact that Google gives weight to pages that are linked to by lots of other pages. So you might, instead of setting up one large Web site with many directories, buy numerous domain names and have your many "separate" sites link to one another. Such a tactic might get you a short-term boost in traffic, but sooner or later Google will sniff out that practice, and then you will probably lose that advantage and could be blackballed. Similarly, many companies have invested lots of effort in setting up pro-forma link exchanges with unrelated sites. But Google now only gives weight to links to sites with related content, and might reduce a page's weight for unrelated linking on an abusive scale.

Practical Fixes -- Instead of Redesigning Your Whole Site
Perhaps the principles mentioned here make sense to you, but you are in no position to completely redesign your site. In that case, you might want to create mirror pages -- static html pages with the same text content as your present graphically designed search-engine-unfriendly pages. These mirror pages will also be "printer-friendly" and accessible by the blind, who use computer-based devices that convert text to voice and can "read" Web page text out loud.
rtising at eBay, you should consider buying keword-based text advertising at Google Adwords as well. (Check http://adwords.google.com for details on how to do that.)


"Google It"

This article is a transcript of  from the radio program "The Computer Report," which was broadcast live on WOTW 900 AM, Nashua, NH 12-2 PM Sundays.

Over the weekend,  I saw the latest Jennifer Lopez movie -- "Maid in Manhattan," a well-advertised mildly-entertaining piece of fluff.

At one point in the movie, Jennifer's son, age ten, has a question that she can't answer, and she replies matter-of-factly, "Google it."

They are walking up a New York City street. The mother, an underprivileged Latina who works as a maid in Manhattan, has lived within a four-block radius in the Bronx all her life. Presumably, she never went to college. The theme of the movie contrasts her life style and that of the very privileged future US senator (son of a US senator) who falls in love with her. It's a modern Cinderella story, reminiscent of Flash Dance, except in Flash Dance the Cinderella had talent and had to prove she had talent to achieve her dream. In this case, Jennifer Lopez just has to be gorgeous.

But this stereotypically underprivileged person has seen the Web and Google, or used them or heard about them so often that she takes their capabilities for granted, as a normal part of her life and her son's life.

When she uses the expression "google it", no other explanation is required either for her son or for the movie-going audience. There's no mention of the Internet or search -- all of that is implied in the newly-coined verb "google". No big deal. To me the fact that that is not a big deal to characters of this kind in this kind of a movie is a very big deal indeed. To me that signifies that the Web has gone completely mainstream, that it is not just high-tech that we read about and hear about everywhere, but rather is an ordinary expected part of our daily lives, that we  depend on it and take it for granted like refrigerators and stoves and microwaves and televisions.

The ten-year-old kid is shown to be bright when by chance he happens to be in an elevator with a New York state assemblyman who wants to run for the US Senate. The kid knows the candidate's voting record on environmental issues and makes some intelligent observations. The assemblyman's overzealous idiot assistant is shocked that the kid knows so much. But the assemblyman and the kid both take it for granted that all that info is readily accessible by all on the Web. Smart people know that and use that capability, regardless of their wealth, education, or background. Only fools don't.

To me this movie marks a stage in Internet history, somewhat like the cartoon in the New Yorker, back in July 1993, that showed two dogs looking at a computer monitor, and the one dog said to the other, "On the Internet, no one knows that you're a dog." That was then followed by the first appearances of URLs and email addresses on billboards and in radio and TV ads; and the first intelligent use of the Internet as a plot element in a high-tech popular movie with Sandra Bullock in "The Net," then the first use of the Internet as a plot device in a romantic comedy with Tom Hanks and Meg Ryan in "You've Got Mail."  But in both those movies, and their imitators, the writers and directors felt it necessary to explain the technology.  The people who used the technology were early adopters, a cut above the ordinary. You never see a computer in "Maid in Manhattan" (or at least I didn't notice one). In the other movies, computers were everywhere, and, in all probability, computer manufacturers paid the studios to have their equipment prominent displayed, as has been the case since the early days of PCs.  Now it's a different ballgame. Computers and the Internet aren't just a part of the everyday office environment, the Internet and Google in particular have become an ordinary part of the English language -- not just how we do business, but how we think, how we deal with our children and with the complex world we live in.

Google, like Frigidair, scotch tape, post-it, and Xerox has become so pervasive, so well known, that the brand name is used as an ordinary word. The brand name has become so successful that the trademark is at risk. The other words reached that status in large part because of massive advertising campaigns. I've never seen an ad for Google.

The other brands became household words because those products were the first of their kind, or at least the first to be widely used. But Google wasn't the first Internet search engine -- far from it. Infoseek and Excite and Lycos were on the scene much earlier. Then they were pushed to the side by AltaVista, which dominated for a while with the advanced hardware and financial backing and the national and international advertising of Digital and then Compaq. But with that backing also came enormous corporate inertia and old economy thinking that held AltaVista back and dragged it in unnatural directions.

Google, which came on the scene late and started as a university research project with little funding, focused on search and just search, and continued to do so over the years, without being seduced into trying to become a general-purpose portal with fancy graphics and dozens of different applications.  It grew by word of mouth, not by advertising; by providing an excellent, unbiased, all-inclusive, easy-to-use Web search service, not by making claims on television. And it is now so dominantly popular that not Lycos, not Excite, not AltaVista, all of which advertised heavily and loaded their home pages with flash and irrelevancies, but rather the late-comer Google, with its simple and direct approach has become the household word, synonymous with Internet search, almost synonymous with the Web itself.

And their tradition continues.  Shortly before the Christmas shopping season, Google launched a new Web site called "Froogle", which focuses on product search for shoppers. Instead of adding this service to the Google site and cluttering it, taking it away from its core strength, they made a separate site. They didn't even add an ad or even a link from the Google site -- keeping that site as clean and simple as it was before.  Over the last few weeks, I've seen no ads for the Froogle site either -- whether in print or on the Web. Rather they relied on providing a quality free service and backing it with good public relations.  As a result, over the last few weeks, I've seen dozens of mentions of Froogle in on-line and print magazines, and have received email from a dozen friends suggesting that I give it a try. Word of mouth based on quality, not advertising, wins in the new business environment.


Kazaa

When the music industry killed Napster in the courts they probably did away with their last, best opportunity to generate significant revenue from online downloads. Napster had tens of millions of users and central control over their activities. It did such a great job and offered easy access to so much music that competitors faced a very difficult challenge. Napster was in such a dominant position that it could have begun to charge modest fees and used that revenue plus ad revenue to pay royalties to music producers.

Napster created a marketplace, trained tens of millions of people in use of P2P applications, and got them addicted to seeking new music online. The death of Napster cleared the way for new companies to fill that role, companies with lots of incentive to come up with alternative ways to do the same thing without central control, without any way for anyone to track what was going on, beyond the legal reach of the music industry. These Napster wannabes have the additional advantage that today the typical new computer system has far more disk space and that many more people have high speed Internet access through cable and DSL.

One of the most successful of the Napster wannabes is Kazaa. .

To join that community, you go to http://www.kazaa.com and download their software. On installation, that software sets up a "shared" directory on your PC. You can move whatever files you want to that directory, and any music that you download from Kazaa, as a default, goes to that directory. When you do a search for files, your search is propagated through computers running the Kazaa software and you get back a list of matches that appear in those shared directories. As a default, your computer also becomes a Kazaa "supernode," so when new users sign up, randomly, some of them will fetch the Kazaa software from your computer. Also, as a default, anyone can upload anything from your shared directory at any time, without warning, and without leaving a trace that they were ever there.

The Kazaa software includes a "theater" which makes it easy to sample the files you are interested in before downloading them, and also to play the ones that you already have.

The fact that high speed cable and DSL access keeps you continuously connected to the Internet, and that many people who are connected that way leave their computers on lots of the time if not always, makes this service far more efficient and effective that something similar would have been a few years ago, with intermittent, slow dial-up access.

And the combination of high speed access and gigabytes of unused disk space means that it is easy to fetch not just music, but also far larger video files. The typical files my son Tim gets are well over 100 Megabytes, and consist of combinations of music and video adapted and cleverly edited from his favorite TV shows, such as Gundam Wing and Dragonball Z. And some of the files available consist of complete episodes of those animated TV shows.

Sounds great, feels great. And this is all "free".  But beware. On the Internet, you often pay a price for free services, a price that isn't measured in money paid.

If you want to venture into this untamed realm, you should take a very close look at the default settings.  First, keep in mind that while Kazaa can provide some level of virus filtering, the default setting turns that off. Be sure to turn it on; and also make sure that you have good up-to-date anti-virus software running on your PC.

Also, keep in mind that you do not have to make your PC a supernode and stock your shared directory with interesting files, and allow other people to upload from your shared directory. That is not a condition for participating in the community. In fact, if you wish, you can shut your computer off from access by other Kazaa members, either all the time, or when you are doing serious work on your computer, and don't want your system resources to be randomly taken over by strangers as they upload your files.

Also, while with Napster you could select a dozen songs and let Napster fetch them as quickly as possible while you did other things; at Kazaa, even with high speed access, it often takes quite a while to get the tunes and videos that you want, one at a time; and sometimes, but rarely, after repeated tries, you still don't get what you want.

Also, whenever you use Kazaa, your eyeballs will be bombarded with annoying animated popup ads, often cleverly disguised to fool you into clicking on them.

And use of Kazaa drains your computer's short term memory; so no matter how much RAM you have (and we have 256 Megabytes), sooner or later you won't be able to see animations and won't be able to play music, and if Kazaa is running in the background the other applications you are running will slow down or even freeze, and you have to reboot.

But even with these drawbacks, the challenge of the hunt is often as satisfying as enjoying the music itself. I suspect that even people who have enough money so it means nothing to them to buy music, would prefer to hunt it down this way, for that extra jolt of excitement that comes from playing this wild file-fetching game.


Search Engines and Web Page Design

This article contains the script that was heard on the radio program "The Computer Report," which was broadcast live on WCAP in Lowell, Mass., and was syndicated on WBNW in Boston and WPLM in Plymouth, MA

Back in 1995, in the early days of the Web, I delighted in the ability to put an entire book in a single document. With plain-vanilla static HTML and no graphics, very large, textually rich pages loaded quickly. And with internal links it was easy to have a detailed table of contents at the top of a large "page" from which you could click to any chapter, and footnote numbers in the text linked to the list of footnotes at the end, and links from there back to where you were before in the text. But then along came search engines.

Thanks to the search engines, people could now find your pages even though you did nothing to advertise/market them. That was a very valuable and important free service. So we began to pay close attention to how they worked and how they ranked the pages they indexed, because changes in the ways we designed our pages could make a big difference in traffic.

When I was researching my book The AltaVista Search Engine, back in 1996, when I was at Digital, I was surprised and annoyed to discover that AltaVista only indexed the first 100K of the Web pages it found. It would pick up links from the rest of the text, but only the beginning of a large page was fully indexed. On the one hand, AltaVista gave extra value to large, text-heavy pages, as opposed to short ones that consisted of just a few sentences and graphics and links. But on the other hand, they set this arbitrary limit. Knowing that, I went back and broke up my biggest, most useful pages into series of shorter ones -- watching that 100K limit. So instead of presenting books as single documents, I broke them into chapters. The result was less useful to my visitors, but the difference in terms of search-engine-generated traffic was important. Later, without making this information public, AltaVista changed that limit to 67K, so people, like myself, who had redesigned pages to comply with their arbitrary limit were unintentionally penalized.

On the plus side -- at least from my perspective --  the fact that search engines indexed every word on every page meant that you could create Web pages designed to be found by particular people or particular sets of people -- as a research tool or a marketing ploy. All you had to do was include the people's names (if they were unique) and the names organizations and activities that you knew they were interested in and likely to search for.  I started writing articles about that technique, which I call "flypaper", back in 1996 http://www.samizdat.com/search.html#fly ; and I recently wrote another article telling about the amazing results that flypaper brought in my research into the Sergei Solovieff mystery: a variant of the Spanish Prisoner Scam http://www.samizdat.com/solovieff.html

Also, back in the early days of the Web, "anchors" were insignificant. The anchor is the set of words that is highlighted in an HTML page to indicate that it is associated with a hyperlink. Click on the anchor words, and you go to another page. Many sites would use something innocuous and meaningless like "click here" for all their links. I preferred to write out the complete URL and have that as the anchor, so visitors could know where they were going when they clicked, and also so they might remember the address itself for future reference (and for that reason, I kept my URLs as short and simple and logical as possible). Then along came Google, which treated anchors in an unusual way. Like other search engines, Google filled its index by sending out crawlers which followed trails of links. But Google also paid special attention to anchor text, and "remembered" the anchor text associated with particular links, and used that text in its ranking algorithm. They even added to their index pages that they had not yet crawled to, but only knew of indirectly through the anchors that linked to them. So if hundreds of different Web sites all had anchors that read "tyrannosaurus rex" and all those anchors linked to the same page, that page would likely come up very high on a search for "tyrannosaurus rex" regardless of what was on that page or whether Google had ever visited it. Over time, Google covered more and more pages directly and added an algorithm for estimating the relevance of a page to the text in the anchors pointing to it. But still they include many pages in their index that they know of only indirectly by anchors. And because of this practice of theirs, my practice of using the URL itself as anchor text means that my links to useful and important pages at other sites probably give those sites less benefit than if I used catchy phrases as anchors instead.

Worst of all, in terms of distortion of Web site design and encouragement of non-productive practices and business models, has been the concept of "popularity." Google and other search engines decided to define "popularity" in terms of links to a page from pages at other sites and decided to give that "popularity" lots of weight for ranking in search engine results. Before, links to your pages from other sites were helpful because visitors might click on them, giving you extra traffic. Now such links were even more important -- not for the clicks, but rather for the unwarranted interpretation that the search engines gave to them. Pages with lots of links to them got high ranking on searches, and hence got lots of traffic -- even if no one ever clicked on those links.

This mechanism gave a boost to businesses that helped mediate link exchanges among sites that had never heard of one another and might have nothing at all to do with one another.  It also led to the creation of many Web pages that consisted of nothing but links -- useless links to other sites with content that had nothing in common with the linking site. Eventually, search engines like Google caught on to this practice (which they had unwittingly encouraged) and figured out how to estimate the relevance of links. Hence, those links no longer provide the traffic boost they used to; but those useless pages and link exchange programs linger on.

The value of link-based "popularity" also meant that if you were going to create a set of sites -- either by yourself or by recruiting others to run them for you -- you would be better off buying a separate domain name for each, rather than running each in a separate directory of the same site or a separate sub-domain of the same domain. If you used separate domain names, Google and other search engines using a similar algorithm, would interpret the numerous links from one of your sites to another as if those were independent sites, and hence would give you a big boost in the rankings for "popularity". For instance, Webseed built a business with a couple thousand volunteer-run Web sites, each with its own domain name. The links among these sites made these sites very "popular" by Google's algorithm, which led to substantial traffic, and (in the days when banner advertising was viable) helped generate revenue. From Webseed's final messages to its volunteer Webmasters (of which I was one), I gather that, eventually, Google caught on, and figured out how to discount  incestuous linking (at least in the case of Webseed); hence, Webseed's traffic dropped precipitously, and their business model collapsed.

So why should we care? Millions of Web sites and Web-based businesses are dependent on one another. What one business does in pursuing its own best interests can affect other businesses in unintended ways. For instance, a company sending out a spam email with a subject line intended to fool people into opening it immediately trains the recipients to doubt any future message sent with a similar subject line. The more spam messages sent, the more words and phrases become "tainted", limiting more and more the vocabulary available for legitimate communication. We're all drinking from the same waterhole, and when one person pollutes it, we all suffer.

The people who make the rules and formulate the algorithms for the major search engines should take into account that their decisions affect more than the internal working of their systems and more than the satisfaction of their visitors. Those decisions can make an enormous difference in the traffic to the sites that they index (and that they don't index), bringing some companies sudden success and destroying others. Those decisions can also lead to strange, unintended distortions in Web page design, as companies, in their struggle to survive, do their best to understand the underlying mechanisms of search engines and make changes intended to boost their search-engine-generated traffic.

But while standards get publicly aired and debated by bodies with representatives of the interested parties, the details of search engine design and their ranking algorithms remain shrouded, as proprietary trade secrets; and the designers can make changes whenever they please, without telling anyone beforehand or even afterhand. And those secret decisions can have enormous repercussions throughout the Web.

We have here a case where private business interests can collide with the good of the overall community, a case where the normal rules governing "trade secrets" might in the future be modified. That could happen by the search engines themselves recognizing their responsibility and sharing such information in ways they never have before, and seeking input and feedback from affected companies and individuals. They could do that publicly and individually as a way to enhance their image, or privately through participation, say, in the Worldwide Web Consortium, where design changes might be openly discussed, without full disclosure to the public -- giving an opportunity for experts to probe and seek to understand the business and technical implications and the possible unintended consequences, without giving away crucial proprietary information.


Flypaper: Draw Traffic To Web Pages

This article is based on my personal observations. It was heard on the radio program "The Computer Report," which is broadcast live on WCAP in Lowell, Mass., and is syndicated on WBNW in Boston and WPLM in Plymouth, Mass.

When old friends who I hadn't been in touch with for 10-30 years started sending me email -- about half a dozen of them each month -- at first I was flattered. Isn't it amazing that all those people would be looking for me?Then it dawned on me -- why should they look for me?

With a few quick queries I soon established that they weren't looking for me at all. They were looking for themselves. They had gone to search engines and had entered their own name as the query. And since I have a lot of content at my Web site -- including lots of my writing -- many of my old friends are mentioned there. Searching for themselves, they chanced on me; and wound up sending me email.

If I had wanted to find them, I could have spent a lot of time looking and might never have succeeded. But because I had my own Web pages and, by chance, those pages had the right kind of content, and that content was indexed by search engines, the old friends found me instead.

The developers of AltaVista and other search engines intended to allow people to find answers to questions and to locate specific information that they need. But instead, it turns out that many people look first for themselves -- satisfying their curiosity about how often they, and others with the same name, are mentioned and what's said about them. Next they look for particular things that are near and dear to them -- often just out of curiosity, rather than need. It was this behavior and the fact that I had my own personal Web pages that led to me getting so many email messages from old friends -- them finding me by looking for themselves.

I soon realized that what I had done by accident, others could do deliberately -- setting out "flypaper" rather than going hunting with a "fly swatter." While hyperlinks are a way to point people away from your Web pages to other resources on the Internet, "flypaper" provides a way to draw people to your pages and encourage them to get in touch with you directly.

It's a neat flip of your usual expectations -- you connect with the people you want to by making their names and their subjects of interest findable at your site.

You can create Web pages and organize the content on those pages specifically for the purpose of drawing particular people and particular kinds of people to your Web site and hence getting in touch with them.

So how could a business use the flypaper approach? If you want to connect with a particular person, and your phone calls and email are going unanswered, create a Web page that mentions that person and topics that that individual is interested in. Say, on that page, all the good things you've been meaning to say about how you could both benefit from working together.

Be sure to put the person's name and the company's name in the HTML title and in the first line of text, so the ranking algorithms at AltaVista and other search engines will put your page high in the list of matches when people search for those words and phrases.

You needn't have hyperlinks from anywhere to your "flypaper" pages. Just be sure to submit the individual URLs to search engines, especially to AltaVista and InfoSeek, which are good about quickly adding material to their indexes. AltaVista will usually add your pages over night.

The next time the target person does a search for him or herself at AltaVista, your page is likely to appear at the top of the list. When that happens, that person may get in touch with you, and suddenly your position in the upcoming dialogue is greatly improved because they contacted you instead of you contacting them. There are no guarantees, but it's certainly worth a try; and the odds are getting better all the time as more people use the Internet regularly.

That's what I call targeted flypaper -- where you are trying to get in touch with one particular individual.

You also could try general flypaper.

For example, at my Web site I have a list of every book I've read for the last 41 years. It's just a list. When I posted it, I doubted that anyone would be interested. I posted it as a lark, for the fun of it. But because of search engines like AltaVista that Web page draws lots of traffic to my site. I've gotten email from authors, agents, editors and others who like the same books.

How can you apply this concept? Say you work for a school. Create a Web page that lists every alum and the year of graduation and other public info about them. Add URL at the search engines and you'll get email from some of them. As you begin to draw audience to your site with flypaper of this kind, you might give these people reasons for coming back, becoming a loyal audience -- part of a new on-line community.

However you decide to use flypaper, be open to the unanticipated value of saving, recording, and posting information of all kinds.



The Joy of Being Found: Use Flypaper to Connect with Old Friends, Customers & Employers

When old friends who I hadn't been in touch with for 10-30 years started sending me email -- about half a dozen of them each month -- at first I thought: isn't that amazing that all those people would be looking for me? And isn't it great that search engines like AltaVista make it so easy to find people on the Internet?

Then it gradually dawned on me -- why should they look for me? Just like me, they probably each have a hundred or more people who they once were close to (old roommates, business associates, etc.) who they've lost touch with. And why, out of all those others, should they actively come looking for me?

With a few quick queries I soon established that they weren't looking for me at all. They were looking for themselves. Yes, they had gone to search engines (most to AltaVista), and there they had done what most people do at those sites -- they had entered their own name as the query. And since I have a lot of content at my Web site -- including lots of mywriting -- many of my old friends are mentioned somewhere there, typically in the list of thank you's at the end of a book. Searching for themselves, they chanced upon me; and delighted at that unexpected occurrence, they sent me email.

So when the number of long-lost friends finding me starts to slow down, I should create a new page at my Web site where I mention folks I haven't mentioned elsewhere. In other words, instead of trying to systematically find other people on the Internet, I'll set things up to make it easy for those people to find themselves in documents at my Web site -- what I call the "fly paper approach."

It's a neat flip of your usual expectations -- you connect with the people you want to by making their names and their subjects of interest findable at your site. And the same approach could also work well in the world of business, when you are trying to connect with potential customers or potential employers.

Unintentionally, I've seen several business-related instances of this phenomenon over the last few months.

Ebooks Multimedia in San Francisco, maker of interactive CD ROMs for children, was looking for content that they could turn into product in time for this Christmas. Using search engines, they found my book The Lizard of Oz at my Web site. This is a book that I self-published22 years ago, and which had simply been gathering dust. Within about a week of their first contacting me, we had a signed contract, and they are now at work on the project.

Soon thereafter, a movie producer in Iceland looking for new material found my never-produced screenplay Spit and Polish. That's not likely to lead anywhere, but it's an opportunity that I would never have dreamed of pursuing actively myself.

In both those cases, instead of my having to identify prospects, write query letters, and submit manuscripts -- which takes time, effort, and money -- they found me. And because they made the first contact, the conversation started at a different level -- they had a particular need, and they had already determined that my work might fill it.

The most dramatic instance of this principle was totally unexpected -- a kind of opportunity that I would never have dreamt of.

A Gary Trudeau fan, looking for a copy of Bull Tales (Trudeau's first book, published back when he was an undergraduate at Yale), found it mentioned at my Web site in a list I have there of every book I've read over the last 38 years. He sent me email to find out if I still had a copy. He also noticed at my site that my daughter (now a sophomore at Sarah Lawrence) is into acting. It turns out that he is the writer/producer of several popular TV shows, and she was in LA over the summer acting in a movie written and produced by my sister Sallie. After a few friendly email messages, I wound up trading my copy of the book for my daughter to get an audition for a possible part in an episode of one of those TV shows. Nothing immediate is likely result, but she learned a lot from the auditioning experience and made contacts that could prove important in the future.

Another close relative has been looking for a new job. She did all the traditional things -- checking newspapers and job-related Web sites. She's had some interviews, but none of the opportunities looked particularly tempting. Then, out of the blue, she got a call from a headhunter who had found her resume on our Web site. A company located far closer to her home than her present job was looking for someone with her credentials. She's gone for a couple of interviews and it looks very promising and tempting. Regardless of how it turns out, this is another instance of how effective it can be to make yourself findable on the Internet.

So how could a business use the flypaper approach? Instead of (or in addition to) pounding on the doors of prospective customers or partners, set up pages that mention them and the topics that are of most interest to them. If possible, mention the names of the key individuals. You needn't link from your home page to those flypaper pages at all , but do link from the flypaper back to either your homepage of other pages you'd particularly want those folks to see.

Then the odds are good (and getting better all the time as more and more companies and people come onto the Internet) that some of them will find you while looking for themselves or looking for the topics of most interest to them, possibly leading to the kind of business contact you want.

And, of course the same approach could be effective in trying to recruit particular individuals to come and work for you, as well as trying to get particular employers to come looking for you.

So get to work -- set out your flypaper. It needn't take much time or effort, but the payoff could be significant.

Search Engine Optimization and Misconceptions

This article was heard on the radio program "The Computer Report," which is broadcast live on WCAP in Lowell, Mass., and is syndicated on WBNW in Boston and WPLM in Plymouth, Mass.

Over the last couple weeks, I've been having an email conversation with a freelance writer in France. We first met when he opened a HumanClick chat session at my marketing page, and we've been exchanging messages by email ever since.

He's been paying a high price to have his Web site designed and maintained by a firm in England, and has not been getting much traffic. Typically, he asks me for advice, then forwards my answers to the design firm, which then replies, and I react to the reactions.

This three-way dialogue (paraphrased below) illustrates common misconceptions not only of Internet newcomers, but also of the design firms that serve them.

FREELANCE WRITER : My site isn't quite finished yet. I still have to add some of the articles that I have written, and I am waiting for the design firm to register my site with the relevant search engines.

ME: Search-engine registration something you should do yourself -- it is very simple (it's silly to pay someone to do it). And it's something you should keep on top of all time (e.g., submitting each and every one of your pages separately to AltaVista, and entering new and altered pages whenever you make changes).

Since your content is primarily text, you should design your pages yourself, as well. 

Your current pages are pretty, but are virtually useless in terms of search engines -- so complex as to be search engine unfriendly. 

You also should add much, much more content. Search engines only see text -- the more the merrier (I have over 1200 documents at my site, some of which are entire books). 

FREELANCE WRITER: My long-standing, but still to be realized, aim in life is to direct people towards my site -- and to pick up work accordingly.

ME: That's basically what I do. In October, I had over 100,000 page views. I'm averaging 1200-1500 unique visitors a day -- all because of my content. (No advertising).

In addition, there are the benefits of building a reputation, establishing and cementing contacts that could prove valuable later, etc.

About once every two weeks, I'm contacted by one reporter/writer or another who wants to interview me as part of an article because they found related material at my site. (Yesterday, one from a newspaper in Modesto, California, about online shopping.)

Also, not all the "wins" are represented in cash. E.g., I translated two books by a Russian officer (Alexander Bulatovich) about his experiences in Ethiopia around 1900 (from the Russian). I was unable to sell the translation to a book publishers. (The publishers all said that there was no market for anything about Ethiopia, regardless of its merit). I then posted the full text of both books at my Web site. I got interesting email from people all over the world (including a grad student in Poland, who then based her doctoral dissertation on my texts). Then I got a snailmail letter from an elderly professor in Addis Ababa (Professor Pankhurst), who is known as the number one guru on that period of Ethiopian history. Someone had found my translation on the Web and printed the whole thing out and given it to him. He said that this really "must" be published. Shortly thereafter I got email from a professor in Bremen, Germany, who happened to be the great-grandson of the Emperor Menelik II (mentioned in the books), who was adamant that they must be published, and even offered to help the wouldbe publisher financially. With those two messages as ammunition, I went back to an editor who had previously rejected the manuscript, and he almost immediately accepted it. It was finally published last summer. As this is an "academic" book, I'm paid in copies and reputation.

Also, a play that I wrote just after college, back in 1970, and that has never been performed (Without a Myth), I posted at my Web site. I was contacted by an independent theater group in Spokane, Washington, who wants to put it on. They've scheduled it for this December, in the main public theater in that city.

See my article about "flypaper" for more such anecdotes.

FREELANCE WRITER: This seems to be beyond my own skills - which was why I asked this firm to create a site for me. As far as I understand matters, there is nothing worse than simply have it out there - and not properly registered at search engines and working for you. Otherwise its kind of like a brochure or pamphlet, however beautifully produced, sitting out there in your draw. With no one ever seeing it.

ME: Yes, but you should and can do all that for yourself -- and you'd do a far better job of it. 

DESIGN FIRM: Your site is search-engine friendly (as indeed is the database).

ME: Yes, your design firm has set up a very slick and attractive site for you, with .asp pages assembled from elements in databases, based on javascript. But the only text on these pages that search engines can "see" is the brief metatags. Everything else is buried in the database. This approach is totally search-engine unfriendly.

DESIGN FIRM: Your site does not necessarily need to "contain much much more content". Search engines work on a density method. The number of times words appear on a page determines the relevance and how it scores on an engine. Too high and the page will not be listed, too low and it may not be found.

ME: That is absolutely untrue. Search engines, like AltaVista, assign little or no value for repetition of words. Plus, keep in mind that only the text in the metatags of your pages is visible -- very very little text. AltaVista, in particular, gives more weight to pages that are rich is useful text.

DESIGN FIRM: Just adding content does not increase the relevance.

ME: I'm not talking about single-word or single-phrase relevance. Search engines only "see" text. The more text you have, the more likely that people searching for the kinds of content you have will see one of your pages in the list of matches. Most people type phrases and series of words -- not just a single word.

Factors affecting relevance/ranking include: do the query words appear in the HTML title or the first couple lines of text? how large is the page? (the larger the better). etc. 

DESIGN FIRM: I have prepared pages before that have had less than 50 words and scored 99.5% on Infoseek!

ME: That's useless information. It doesn't matter what kind of score you get at one particular search engine for a particular single-word or single-phrase query. What matters is how much traffic you get.

DESIGN FIRM: Were you to publish every article in full, you risk people poaching it and claiming it as their own. This does happen, so is one reason why I suggested going down the route of giving short descriptions in the database and then giving the article in full on payment.

ME: If you are paranoid, you can do periodic searches of the Web for sentences from particular texts -- if someone steals your property, you can sue. But don't be naive. Anyone can take printed text and scan it or type it in and post that. (Material on the Web is no more vulnerable to stealing that any other published material.) And, of course, anyone anytime anywhere can simply paraphrase what you have written and thereby legally appropriate it. Those risks have been around a long time. What the Web does is open opportunities for you to reach a wider audience, to become better known, and especially to become better known by people who might want to hire you to write new material.

DESIGN FIRM: True Internet marketing is a long and expensive process. The page I mentioned earlier took 9 weeks to score with that relevance and about 10-20 hours of development (one page, one search keyword). So, to do the job really thoroughly costs a lot of money.

ME: That is wasted time and effort, and money thrown away.

DESIGN FIRM: You an go to firms that do it on the cheap through economies of scale, but even then you would look to pay 500 - 1000 just for one or two search words. Essentially, the whole process is more than just search engine relevance, though. It really depends on your target market. In your case, this is editors who I would doubt will go around search engines too often looking for writers. More so, writers will come to them. If you can contact an editor with a link to your professionally finished site, you stand a much higher chance than with a wordy or poorly designed site. The principle is the same as a CV - short, concise, factual, and relevant.

ME: Even the advice about the CV (resume) is wrong. On the Web, long resumes are much better, get many more search engine hits and hence get you much more visibility (if they are properly constructed.)

FREELANCE WRITER: I just went to Google and looked for "Freelance writer and journalist" and my site came up quite high on the list. This is what I was after, in the first instance.

ME:The likelihood that someone would search for that exact phrase "freelance writer and journalist" is very slim. The best metric is how much traffic you get and how much business you get from that. The best leads are the ones that come from people doing unexpected searches that match with your content, and you get more of that if you have lots of content available at your site.

DESIGN FIRM: I have designed your site to be search engine friendly, with a density to focus relevance as much as possible within the budget you had available, and have listed to a large number of search enginesm particularly the big eight which most people use (Yahoo, AltaVista, Infoseek, Google, etc). Unfortunately, this process does take time so the results may not be apparent very quickly. I am continuing to monitor them, though.

ME: He's playing on your ignorance of the Internet and bleeding you.

DESIGN FIRM: As I mentioned to you before though, I think the majority of your business is still going to come from sending short marketing e-mails to editors with a link back to your site.

ME: As I detailed for you, I get significant business from editors finding my articles at my site -- people and publications that I had never heard of.

DESIGN FIRM: Lastly, there is more than one way to skin a cat as they say. So his approach is one approach. However, there are others. The way search engines score changes almost monthly and it is a dynamic process, so paying money to score highly one month, does not mean you will still be there the next month. I subscribe to a group of researchers who look into this and how search engines change every month, then give recommendations on page design, marketing techniques, etc. It really is an industry that doesn't stand still by any means!

ME: Sure there's more than one way to approach a problem, but his way costs a lot and doesn't generate traffic. You're much better off doing this stuff yourself. (His score-keeping method is irrelevant to your needs.)

I presume that you want to leave your existing pages in place and that what we will be doing will be adding new content or mirrors of some of your current content in more search-engine-friendly form. That would be my preference. These new pages would be simple and unadorned.

FREELANCE WRITER: So what does a cyber-goon such as myself make of all of this? All I know is that I now have a web site and I want it to work for me - as yours does for you. That is why I thought that it would be a good idea to be personally tutored by you - so that I could learn the ropes first hand.

I also take your point about the knock-on, or secondary advantages, of the web site. As Mrs. Thatcher once memorably remarked, it's a funny old world. (That was just after she got thrown out as leader of the Conservative Party, having won three consecutive general elections...) You never know where a web page can lead you. Or what opportunities can crop up. As you explained very clearly in your last note.

Maybe you and my web design man are in fact saying the same thing in different ways. I really don't know.

ME: No. We are diametrically opposed. The design techniques he is using produce pages that are "pretty", but do not generate traffic. Search engines love text.

To get started, first, you need to be able to connect to your site by ftp (file transfer protocol). (If you don't have it already, there's free software called ws_ftp that you can pick up from just about any large shareware site, like www.download.com and www.tucows.com). The automated database-driven system that your design firm has set up makes it impossible for you to do the kinds of things I recommend.You want a Web hosting service that allows you to simply upload your pages -- without a preset structure. What the design firm has set up for you takes control out of your hands, makes you dependent on them for running the site.

FREELANCE WRITER: I more or less told the chap what I wanted from the outset -- colors etc. -- and he put it together.

ME: I'm sure he gave you exactly what you asked for. The problems are: 1) you didn't know what to ask for, and 2) the techniques he used to produce the pages

If it were possible to upload new pages to your site using ftp, and if it were possible for you to directly edit the pages that are already there, then we could do good things.

In that case, I would recommend:


info@samizdat.com      privacy statement.


Google
  Websamizdat.com