NYCPHP Meetup

[inforequest]

tedd tedd-at-sperling.com |nyphp dev/internal group use| wrote:

>At 4:17 PM -0700 4/24/06, inforequest wrote:
>
>  
>
>><meta name="robots" content="noindex, nocache, follow"> is an important
>>meta tag today for all the major search engines.
>>    
>>
>No offense meant, but this meta-tag is just plain dumb.
>  
>
None taken - I welcoem the discussion.

I would not disagree... much of what goes on these days seems dumb to me 
as well.

>The truth is, that meta-tags are not actually needed. As a test, not that one test is conclusive, I created a site that has a current PageRank of 5 without ANY meta-tags whatsoever and without even 
>submitting the site to a single search engine -- explain that.
>  
>
I'll try, but there are others on this list that are better versed in PR 
than I am.

Page Rank is a reflection of backlink support. Get a few links from pr6 
pages that don't link to to many others, and you'll earn yourself a PR5. 
It has nothing to do with meta tags or even page content for that 
matter. Unfortunately, PageRank has little to do with your appearance in 
the results sets as well. It does seem to get spidered alot more, 
though, and it gives links on your pages more "authority". Hopefully, 
you know what to do with that authority (these days it seems the best 
use of a high PR is to sell links to less-knowledgable people who think 
it is important).

>The explanation is that robots index sites, on their own time table, and make their own assessments regardless of meta-tags. SE's method of evaluating sites is a dynamic process continuously in a state of flux and is certainly not constrained, limited, or influenced greatly by the use of meta-tags. We've gone far past that.
>
>tedd
>
Here's an example of creative use of the robots tag. Suppose you have a 
shopping cart full of product info pages, product purchase pages, and 
follow up "you bought this you might also like that" pages. You don't 
want anything cached, because you don't believe Google has a right to 
show your content without your knowing about it/countng it/putting ads 
onto it/whatever. You don't want the actual cart pages (product 
purchase) to be indexed, because they are just lists of somebody's 
order. You want the follow-on pages of suggestions to be indexed because 
of their excellent cross-referenced linking (a.k.a. effective use of 
PR). So you set the product pages to "index, follow, nocache" and the 
cart pages to "noindex, follow, nocache" and the follow-on pages to 
"index, follow, nocache" and the spider does as you request. Most of you 
out there know how to do that dynamically via PHP..it's just writing out 
the meta info on page load.

What would the robots.txt alternative approach be?

-=john andrews
http://www.seo-fun.com