• Welcome to Sitemap Generator Forum.
 

exclude urls are being crawled

Started by orders178, October 20, 2017, 07:12:50 PM

Previous topic - Next topic

orders178

Hi there, I have urls like this in the exclude url area but they are being indexed.

wp-json/
?filter_by=
search.php?
mb3/

is there a limit to the amount of excludes you can have? I have a lot - could that be the issue?

XML-Sitemaps Support

Hello,

that should be working.
What is an example URL that is added in sitemap and full list of exclusions?

orders178

I am attaching a text file of the excludes and a sample of some of the urls that are getting into the sitemap that shouldn't be there. I will run it again to be sure that this is all the most accurate information for you.

XML-Sitemaps Support

Hello,

if you are using "*" in Exclude URLs setting, then it must be defined in "regular expression" syntax:
[ External links are visible to logged in users only ]

In your case, you need to escape "?" characters to avoid issues:
?filter_by=
search.php?

should be replaced with:
\?filter_by=
search.php\?

orders178

ok that makes a lot of sense! will give that a shot and let you know the results. thanks so much for your prompt replies

orders178