"robots.txt" is a file you can create on your site to help indexing bots to index your site correctly. These bots first scans your
robots.txt file to see which pages to ignore.
This page is a good tool to keep in mind to validate your
robots.txt files. robotstxt.org has more information about the wannabe standard.
- MathML and displaying Math on the web
23 January 2004
- Labels in HTML forms
26 January 2004
- Related by Keyword:
20 October 2014
- Interesting float/int casting in Python
25 April 2006
- Related by Text:
- jQuery and Highslide JS
08 January 2008
- I'm back! Peterbe.com has been renewed
05 June 2005
- Anti-McCain propaganda videos
12 August 2008
- I'm Prolog
01 May 2007
- Ever wondered how much $87 Billion is?
04 November 2003