User-agent: * Crawl-delay: 20 Allow: /wp-content/ Disallow: /wp-admin/ Disallow: /wp-includes/ Disallow: /feed/ Disallow: /trackback/ Allow: /wp-content/uploads/ User-agent: Googlebot Disallow: /*.php$ Disallow: /*.js$ Disallow: /*.cgi$ Disallow: /*.xhtml$ Disallow: /*.php* Disallow: /*.inc$ Disallow: /*.css$ Disallow: /*.pdf$ Disallow: /*.txt$ Disallow: /*?* Disallow: */feed/ Disallow: */trackback/ # compress text, html, javascript, css, xml: AddOutputFilterByType DEFLATE text/plain AddOutputFilterByType DEFLATE text/html AddOutputFilterByType DEFLATE text/xml AddOutputFilterByType DEFLATE text/css AddOutputFilterByType DEFLATE application/xml AddOutputFilterByType DEFLATE application/xhtml+xml AddOutputFilterByType DEFLATE application/rss+xml AddOutputFilterByType DEFLATE application/javascript AddOutputFilterByType DEFLATE application/x-javascript # Or, compress certain file types by extension: SetOutputFilter DEFLATE