IssueTrackerProduct IssueTrackerProduct

About my baby Open Source project

Kung Fu Kung Fu

Fujian White Crane Kung Fu

Zope Zope

What I have and am doing with Zope

Photos Photos

Photoalbum, both old and new.

Receptsamlingen Receptsamlingen

In Swedish only. About my "Collection of Recipes" website.

Contact me Contact me

My contact details and how to contact me.

  Mobile version of this page Mobile version of this page

del.icio.us Save this page in del.icio.us

Previous:
MathML and displaying Math on the web
 Next:
Labels in HTML forms

Related blogs

Interesting float/int casting in Python

Related by category

Web development

 

Robots.txt Validator

http://www.searchengineworld.com/cgi-bin/robotcheck.cgi

robots.txt, bots, robots, robotstxt, validator, validate

24th of January 2004

"robots.txt" is a file you can create on your site to help indexing bots to index your site correctly. These bots first scans your robots.txt file to see which pages to ignore.

This page is a good tool to keep in mind to validate your robots.txt files. robotstxt.org has more information about the wannabe standard.


Comment

Peter - 24th January 2004  [«« Reply to this]
Just found out how to exclude my printer-friendly version and PDF version (see bottom righthand corner)

Disallow: /pv$
Disallow: /pv/pdf$

Let's hope it works.
 
Name:
Email:
hide my email address.

Your email address will be encoded to prevent email-extraction spiders from reading it so you won't get spammed if you decide to show your email address.