Transcript of the live chat session that took place Thursday, November 7, 1996.
These sessions are scheduled for noon-1 PM US Eastern Time (GMT -4) every Thursday.
These sessions are hosted by Richard Seltzer. If you would like to receive email reminders of our chat sessions, simply send a blank email message to email@example.com or go to http://groups.yahoo.com/group/businessonthewebchats and sign up there.
For transcripts of other previous sessions and a list of future topics, click here.
For an article on how to make "business chat" work (based on this experience), click here.
Since the chat itself happens at a rapid pace, it's often difficult to note interesting facts in particular URLs as they appear on-line. Here's a place to take a more leisurely look. I've rearranged some of the pieces to try to capture the various threads of discussion (which sometimes get lost in the rush of live chat).
Please send email with your follow-on questions and comments, and suggions for topics we should focus on in future sessions. So long as the volume of email responses is manageable, I'll post the most pertinent ones here for all to see.
Threads (reconstructed after the fact):
We're here to share experiences about doing business on the Internet -- particularly the World Wide Web. What works? What doesn't work? Why? What are the trends that matter? How can you/should you adapt to the Internet culture and environment?
I work for the Internet Business Group at Digital Equipment in Littleton, MA. In that capacity, I end up talking to people from large companies about how they can use the Web for business. I also have my own personal Web page -- which is content rich and no frills -- which I do for practically nothing and draws a fair amount of traffic and attention.
In a chat session like this things can get pretty frantic. It's sometimes difficult to follow the threads of conversation. And there's no time to write down interesting URLs and facts. So last week, I took a copy of the raw transcript and edited it to make the threads clearer and posted it at my own little Web site so anyone could take a look. You can see it at http://www.samizdat.com/chat16.html I plan to do the same today. Barring technical difficulties, I hope to have a transcript up later today. I'll post it at the same site, naming this one /chat17.html
If you are here for the chat on Business on the WWW, please identify yourself, and let us know your interests.
barbara (18.104.22.168) - 11:59am -- Hi! I'm here.
Richard Seltzer (22.214.171.124) - 12:01pm -- Hello, Barbara, we have two topics -- instead of one -- on the agenda today: 1) continuing our discussion on "new kinds of money" -- particularly, the question of how to "bootstrap" one of these new forms of virtual currency, quickly getting lots of individuals and companies to use one so it then becomes generally available. 2) the impact of search engines on Web site design. If you are here for one or the other of those topics, please let me know.
Richard Seltzer (126.96.36.199) - 12:04pm -- Several people sent email with concerns/ questions related to "new kinds of money". They hoped we would discuss that topic, but all of them (unfortunately) said that they would be unable to connect today. To get that discussion going, we need live folks who wish to ask/answer questions. The alternative is the impact of search engines. My book The AltaVista Search Revolution was just published by Osborne/ McGraw-Hill, and I have a lot to say on that topic if the audience here is interested.
Tom Dadakis DadaCom firstname.lastname@example.org (188.8.131.52) - Thu, Nov 7, 12:06pm -- Hi,joining in a few minutes late.
Richard Seltzer (184.108.40.206) - 12:08pm -- Welcome, Tom -- are you mainly interested in continuing the discussion on "new kinds of money" or the question of the impact of search engines on Web page/site design?
Bill Wendel (220.127.116.11) - 12:08pm -- I'm interested in your second topic: the impact of search engines on Web site design and would like to know more about your book,too. I love AltaVista and it has revolutionized the way I do research!
Warren Agin - Law Solutions (18.104.22.168) - 12:15pm -- Hello, joining in a little late and can't stay long.
todd (22.214.171.124) - 12:16pm -- hi. I'm more interested in the search engines/directories topic.
Jim Dorval (email@example.com) (126.96.36.199) - 12:29pm -- Hi, sorry I am late, just extinguishing the last flames. Vivo Software. Multimedia over the internet.
Richard Seltzer (188.8.131.52) - 12:10pm -- Barbara -- Regarding the "small money" approach, I'm expecting that Russ Jones, a Millicent expert from Digital, will be joining us shortly. He'd be the best one to point you to examples. In the interim I'll try to get started on the search questions.
Tom Dadakis DadaCom firstname.lastname@example.org (184.108.40.206) - Thu, Nov 7, 12:10pm -- Most of the search engines are interconnected now but you have to know whether there is a robot searching the site or there is some human intervention, like at yahoo. But Richard would be the expert in describing how the search engines work. Richard, why don't you explain how they work.
Richard Seltzer (220.127.116.11) - 12:07pm -- Barbara -- First, let's distinguish between "directories" and search engines. Yahoo is a directory. The information in their database comes from "submissions" -- it is categorized by the person doing the submission and checked (by hand) by folks at Yahoo. A full-text search engine like AltaVista gets its information by sending out a robot program, which fetches Web pages and brings them back to be indexed. Over time AltaVista should retrieve data on any public Web site that has links to others and that doesn't include commands to exclude robots. You can, however, make sure that your site/page is indexed sooner rather than later by "submitting" -- connecting to the site, clicking on Add URL, and providing your Web address.
Richard Seltzer (18.104.22.168) - 12:13pm -- Tom -- The robot programs (spiders) appear to a Web site like just another user. They retrieve information which then is indexed. AltaVista indexes every single word -- some others deal only with key words. Also, the robot that AltaVista uses ("Scooter") sends out dozens of "threads" simultaneously -- in other words it acts as if it were dozens of users connecting to dozens of different Web sites simultaneously. This speeds up the fetching process -- which otherwise could take far too long, considering there are over 30 million pages on the Web today. Is that clearer?
barbara (22.214.171.124) - 12:15pm -- I still would like to know if there are specific search engine sites or directories. They all seem so general and non-focused.
Warren Agin - Law Solutions (126.96.36.199) - 12:17pm -- Barbara, I know there is a site which searches websites with a legal orientation.
Richard Seltzer (188.8.131.52) - 12:20pm -- Bill -- The biggest barrier to doing what you are interested in doing is the fact that most of those sites hold their information in databases rather than as plain HTML pages. Robots, like AltaVista, simply stop at the front-page of such sites. So a search tailored to find homes for sale, say, in Vancouver would only find those pages that aren't in databases and aren't block in other ways. Intelligent agents could probably be programmed to go through the registration and/or query process for a few such sites. But it would be a daunting task to program one for all the many different real estate sites today.
Bill Wendel (184.108.40.206) - 12:24pm -- Richard, thanks for your answer. Can you elaborate on: "Intelligent agents could probably be programmed to go through the registration and/or query process for a few such sites. But it would be a daunting task to program one for all the many different real estate sites today." If voluntary standards were set to facilitate such searches, it would be a great public service and probably a good business opportunity, too.
Warren Agin - Law Solutions (220.127.116.11) - 12:25pm -- Bill, sounds sort of like "Submit-it" which uses the URL to complete the submission forms for several different sites.
Bill Wendel (18.104.22.168) - 12:29pm -- Warren Agin: Thanks for your suggestion. I just retrieved 8,000 references to submit-it using Alta Vista. I peruse some of them later.
Todd (22.214.171.124) - 12:30pm -- FYI, there's a free service out there, I think it's "submit" or "submit it" or something" that posts your page to a dozen or so directories at once.
Richard Seltzer (126.96.36.199) - 12:32pm -- Bill -- The nature of the problem you raise depends largely on how many sites you want to be able to search. If it's a dozen or maybe even a few dozen, then, as Warren suggests, an approach like Submit-It make work. But I believe the number of such sites is escalating rapidly. Possible solution -- enter into some kind of business arrangement with the main sites -- encouraging them to use standard methods (that's a tough one) or at least to cooperate with you (that's also tough). You could aim to create a "meta-site" one site that is an entrypoint for the whole industry. But a lot of other folks would probably also like to play that role. No guarantees of victory there.
barbara (188.8.131.52) - 12:34pm -- Todd, there are a lot of services like that, but, basically, once you type the information in their main directory, you usually have to individually make sure the information is submitted to their list of search engines. They are a good starting point, but it's not a large time saver. It does give you good lists to work from, though.
Richard Seltzer (184.108.40.206) - 12:37pm -- Back to the real estate question -- I had an interesting experience with job search sites recently. My wife was looking for a job. So we connected to about a dozen different sites. They all were set up very differently with different cascades of menus and search methods. We found nothing. Then, out of the blue, my wife got a call from a headhunter, who had found her resume at our little Web site using AltaVista. Think about it -- if you were a headhunter, would you rather do dozens of different kinds of searches at dozens of different sites, or would you rather just do an AltaVista search and find the folks who had sense enough to post their resumes as plain HTML files (as opposed to putting them in the databases of job search sites, where search engines could never index them).
Todd (220.127.116.11) - 12:40pm -- Bill, Richard- Home Scout attempts to be such a site. (http://homescout.com). It's rather rough at this stage though.
Richard Seltzer (18.104.22.168) - 12:41pm -- Todd -- Thanks for the pointer. I'll take a look at homescout. I'm particularly interested in how people set up "meta-sites". I feel it's an interesting new business opportunity. It's too much hassle going to the sites of every individual company when your purpose is to find a particular kind of product or information that could be anywhere.
Warren Agin - Law Solutions (22.214.171.124) - 12:43pm -- Richard - I have a metasite geared toward bankruptcy professionals. A pretty small segment.
Warren Agin - Law Solutions (126.96.36.199) - 12:43pm -- Basically, I just provide lists of resources available on the Internet, and make an effort to organize them in a usable fashion.
Richard Seltzer (188.8.131.52) - 12:48pm -- Warren -- Sound like a step in the right direction. The kinds of features that move it up a notch would be 1) forum/chat -- any way to let users interact at your site on the focus topic 2) something more than just hyperlinks -- for instance specialized search focused at the key sites in your area. You mentioned LawCrawler, which uses AltaVista. You also could put together a kludge -- compose specific complex searches using AltaVista and limited to the sites (host:) in your "industry". Then bookmark the search results and cut and paste those URLs on your pages and anyone clicking on that link would automatically make that complex search of just those sites. (NB -- this only works well when the target sites have their main info in HTML pages, rather than databases.)
Warren Agin - Law Solutions (184.108.40.206) - 12:50pm -- I do have a broad search tool which allows users to identify the pages in the web which contain the desired key word. I also have a discussion group which, much to my amazement, actually attracts some postings. I also have some links which check out various databases like AltaVista for key words like "bankruptcy"
Richard Seltzer (220.127.116.11) - 12:52pm -- Warren -- Sounds good. What's the URL? (I know you've posted it in previous chat sessions -- just so the folks today can see it.)
Warren Agin - Law Solutions (18.104.22.168) - 12:53pm -- http://www.agin.com/lawfindKeep in mind that this is pretty low budget. I get about 50-100 hits a day. Not quite Alta Vista.
Bill Wendel (22.214.171.124) -- 1:08pm -- Thanks for addressing real estate searches. While you're checking homescout, try IRED.com, and REIPA (Real Estate Information Providers Association). email@example.com We are overhaling our website (http://www.realestatecafe.com) and would like to incorporate searches and intelligent agents to help consumers save money! The Consumer Federation of America says consumers could save $10 billion per year if they knew more aobut how real estate agents get paid. I firmly believe that reforms in the RE industry will be one of the Internet's first multi-billion paybacks for consumers!
Bill Wendel (126.96.36.199) - 12:20pm -- I am interested in #4) how a search engine like AltaVista ranks information and hence how to optimize how high your pages will appear on a search list; and #8) how and why to exclude robots from certain directories and files.
Tom Dadakis DadaCom firstname.lastname@example.org (188.8.131.52) - Thu, Nov 7, 12:18pm -- I think people would be more interested in how they get listed and how they can improve their positioning on the list by using keywords or meta tags in their html pages listing certain words (sometimes their competitors names) to come up under certain searches.
barbara (184.108.40.206) - 12:19pm -- Tom, isn't using competitor's names against intenet netiquette?
Tom Dadakis DadaCom email@example.com (220.127.116.11) - Thu, Nov 7, 12:21pm -- Barbara, you are absolutely correct but wouldn't Digital like to show up everytime someone searches for IBM :->
barbara (18.104.22.168) - 12:28pm -- What one would like to do and what one with a conscience would actually do are different. I don't work for Digital, but I'm sure they are too ethical to put a competitors name on their page.
Richard Seltzer (22.214.171.124) - 12:25pm -- Regarding Metatags -- AltaVista recognizes two kinds -- (details can be found in the on-line Help) One kind gives you the opportunity to state exactly the words that you want to appear as the description in the results list. (Keep in mind that the default is the first couple lines of text that appear on your page. If your page clearly states what you are up to, there is no need to mess with metatags. And if it doesn't, you should take another look at your page design.)
Richard Seltzer (126.96.36.199) - 12:28pm -- Second kind of metatag allows you to add "key words" -- in other words, if key terms that are important to your business and that people are likely to search for do not appear in the text of your page, you can add them in a metatag. But once again, if those words don't appear on the page, you should think twice about maybe editing the text to include them. Some people reportedly have begun to abuse this variety of metatag by including their competitors' names as key words. That practice is of questionable value and also could lead to legal complications if your intent is to confuse or mislead the user.
Todd (188.8.131.52) - 12:35pm -- Richard - I have a page with
of the script as the description. Also the second kind of metatag would
be good for synonyms.
Richard Seltzer (184.108.40.206) - 12:40pm -- Todd -- Yes, metatags or descriptors can let people know about the content of pages that are primarily graphics or Java etc. That's a good use of them. Also, synonyms make sense as keywords. Keep in mind too that if someone's name is important, you might want to enter variants of it as key words as well -- these search engines don't recognize that Robert is the same as Bob or that Seltzer, Bob is the same as Bob Seltzer unless the person entering the query follows a strict format and enters all the likely alternatives (which doesn't happen very often.)
Warren Agin - Law Solutions (220.127.116.11) - 12:45pm -- Or files you don't want the public to find. I often post stuff such as pictures for friends, but don't link it to my primary webpage.
Todd (18.104.22.168) - 12:46pm -- Richard - I had post a question earlier that got buried in the heat of battle: why might AV miss a page other than that it's too new? Are you confident that it is covering the web, or has it grown to the point where that's no longer feasible? How do you know how well AV is doing?
Richard Seltzer (22.214.171.124) - 12:51pm -- Todd -- To the best of my knowledge, AltaVista is still fetching everything that's public. They keep throwing more and more hardware at the project, and it's amazing how fast the response rate is for over 21 million hits a day. Keep in mind how the robot works -- it fetches a pages, sniffs out all the links on the page and then goes to all those links. So if a page is not pointed to from other pages, or has very few pages anywhere in the world that link to it, it may take a while for Scooter to randomly get around to it. That's why it's a good idea to submit the URLs of your new pages. That way Scooter retrieves the pages immediately and they are fully indexed in about a day.
Richard Seltzer (126.96.36.199) - 12:33pm -- Warren -- I haven't tried HotBot yet, and hence don't know how it's structured. (Once you get hooked on one search engine and are comfortable using it to find what you want, it takes something major to induce you to change.)
barbara (188.8.131.52) - 12:49pm -- Todd, I just looked for something under hotbot and got no response. I did the same with Alta Vista and got 10,000. True it's too much to sift through for what I want, but they did give me some answers while hotbot did not.
Richard Seltzer (184.108.40.206) - 12:58pm -- Briefly about "flypaper" -- I was getting 2-3 email messages per week from old acquaintainces I hadn't heard from in 20-30 years. It finally occurred to me that they weren't looking for me (any more than I'd be likely to do a search for them). On inquiring, I discovered that they were looking for themselves. In fact, that is by far the most common thing that people do when they connect to a search engine like AltaVista -- they look for themselves. I had so much text at my site, covering so many of the things I've been involved in in my life, that these people were mentioned there -- they found me by looking for themselves. Just keep that psychology in mind when designing Web pages -- make sure that your target audience/customers can find themselves or the topics that are of most interest to them somewhere at your site.
anonymous (220.127.116.11) - 12:52pm -- Hi...I think it's crucial as web designers that we understand how the various search engines are constructed. That means reading all the FAQs and Help Files...and even understanding what librarians know so well-- how to find info
barbara (18.104.22.168) - 12:58pm -- Anonymous has a good point. You do have to read up on the individual search engines and directories to know what they specifically want and how to submit to it. It took me a long time to get my submission to Yahoo in the right categories.
Carol J. Snyder (snyderinfo*Web Design) (22.214.171.124) - Thu, Nov 7, 12:55pm -- Hi, Richard, et al. Definitely! The topic of Search engines have a lot to discuss.
Tom Dadakis DadaCom firstname.lastname@example.org (126.96.36.199) - Thu, Nov 7, 12:57pm -- I think people will have to learn to do more selective searches so they can find the information they are looking for. A list that returns 10,000 entries doesn't do anyone any good. How can we make more selective searches?
Carol J. Snyder (snyderinfo*Web Design) (188.8.131.52) - Thu, Nov 7, 12:59pm -- People will need to understand Boolean logic (AND, OR, NOT) in doing searches and how to limit...all things that Information Specialists/Librarians do everyday... or there will be a market for info providers...(as search engines grow, they're too unwieldly...)
Richard Seltzer (184.108.40.206) - 1:02pm -- Carol -- I try to demystify that AND/OR complexity in the book The AltaVista Search Revolution. I can get into it more next week if there is interest.
Carol J. Snyder (snyderinfo*Web Design) (220.127.116.11) - 1:01pm -- www.snyderinfo.com (email@example.com) (was anonymous in this discussion) cheering for "knowing how to think like InfoSpecialists/Librarians think"! Info is Power
Tom Dadakis DadaCom firstname.lastname@example.org (18.104.22.168) - 1:01pm -- see you next week
Richard Seltzer (22.214.171.124) - 1:00pm -- All -- Time's running out. Thanks to all for joining us. Please post your email addresses and URLs for followup.
Also, please send me email email@example.com with your followup messages/questions and check for the transcript of this session later today http://www.samizdat.com/index.html#chat
Thanks to all for your participation. Richard
Previous transcripts and schedule of upcoming chats -- www.samizdat.com/chat.html
To connect to the chat room, go to www.samizdat.com/chat-intro.html
The full text of Richard Seltzer's books The Social Web, Take Charge of Your Web Site, Shop Online the Lazy Way, and The Way of the Web, plus more than a hundred related articles are available on CD ROM My Internet: a Personal View of Internet Business Opportunities.
Business Boot Camp: Hands-on Internet lessons for manager, entrepreneurs,
and professionals by Richard Seltzer (Wiley, 2002).
No-nonsense guide targets activities that anyone can perform to achieve
a library for the price of a book.
This site is Published by Samizdat Express, 213 Deerfield Lane, Orange, CT 06477. (203) 553-9925. firstname.lastname@example.org
Return to Samizdat Express