User-agent: * Disallow: /wp-admin/ Disallow: /login/ Disallow: /wp-login.php* Allow: /wp-admin/admin-ajax.php User-agent: Ahrefsbot Crawl-delay: 10 User-agent: Screaming Frog SEO Spider Crawl-delay: 10