User-agent: *

Host: www.divextravel.ru

Crawl-delay: 40 # задает таймут в 40 секунд

Disallow: /search
Disallow: admin.php
Disallow: config
Disallow: header
Disallow: footer
Disallow: pntables
Disallow: referer
Disallow: /images
Disallow: /themes
Disallow: /manual
Disallow: /docs
Disallow: /includes
Disallow: /javascript
Disallow: /language
Disallow: /modules/NS-
Disallow: /pnadodb
Disallow: /themes
Disallow: /ref/

User-agent: Yandex
Crawl-delay: 20 # задает таймут в 20 секунд