User-agent: * Allow: / # Important crawl directives for Google User-agent: Googlebot Allow: / # Block admin and API routes from crawling Disallow: /admin/ Disallow: /auth/ Sitemap: https://demstate.com/sitemap-v2.xml Sitemap: https://demstate.com/news-sitemap-v2.xml Sitemap: https://demstate.com/video-sitemap-v2.xml