← Back to branch summary

~ubuntu-branches/ubuntu/natty/moin/natty-updates

« back to all changes in this revision

Viewing changes to wiki/htdocs/robots.txt

  • Committer: Bazaar Package Importer
  • Author(s): Jonas Smedegaard
  • Date: 2008-06-22 21:17:13 UTC
  • mto: This revision was merged to the branch mainline in revision 18.
  • Revision ID: james.westby@ubuntu.com-20080622211713-inlv5k4eifxckelr
Import upstream version 1.7.0

Show diffs side-by-side

added added

removed removed

Lines of Context:
wiki/htdocs/robots.txt
 
1
# if you want to add own robot rules, do it BEFORE the final rule matching *
 
2
 
 
3
User-agent: *
 
4
Crawl-delay: 20
 
5
# This has to match script url + cfg.url_prefix_action - it
 
6
# saves lots of search engine load and traffic by disallowing crawlers
 
7
# to request action related URLs.
 
8
#
 
9
# NOTE - in order to make this have any effect, you have to set
 
10
#        url_prefix_action to "action", cf. HelpOnConfiguration
 
11
 
 
12
Disallow: /action/
 
13