Sitemap ignores the robot.txt
« on: June 19, 2019, 07:39:13 PM »
Sitemap ignores the robot.txt
In robots.txt prohibited:
Disallow: /*&v=list

Sitemap:
/catalog/teplye-kolgotki?page=2&v=list&s=hit&d=desc

Option "Use robots.txt" enabled, but do not work.
« Last Edit: June 19, 2019, 07:48:47 PM by coolbits »
Re: Sitemap ignores the robot.txt
« Reply #1 on: June 20, 2019, 09:03:35 PM »
Hello,

please let me know your generator URL/login in private message to check this.

Also, you can add this in Exclude URLs setting:
v=list