← Back to branch summary

~ubuntu-branches/ubuntu/natty/moin/natty-updates

« back to all changes in this revision

Viewing changes to MoinMoin/web/static/htdocs/robots.txt

  • Committer: Bazaar Package Importer
  • Author(s): Jonas Smedegaard
  • Date: 2008-06-22 21:17:13 UTC
  • mto: This revision was merged to the branch mainline in revision 18.
  • Revision ID: james.westby@ubuntu.com-20080622211713-inlv5k4eifxckelr
Import upstream version 1.7.0

Show diffs side-by-side

added added

removed removed

Lines of Context:
MoinMoin/web/static/htdocs/robots.txt
1
 
# if you want to add own robot rules, do it BEFORE the final rule matching *
2
 
 
3
 
User-agent: *
4
 
Crawl-delay: 20
5
 
# This has to match script url + cfg.url_prefix_action - it
6
 
# saves lots of search engine load and traffic by disallowing crawlers
7
 
# to request action related URLs.
8
 
#
9
 
# NOTE - in order to make this have any effect, you have to set
10
 
#        url_prefix_action to "action", cf. HelpOnConfiguration
11
 
 
12
 
Disallow: /action/
13