Issue With Posting.php URL's in Sitemap
« on: March 10, 2021, 02:30:18 PM »

Hi

I am trying to setup a sitemap for our phpbb3.0.9 forum, everything seems to be working as it should do accept for the fact that the crawler keeps crawling and including posting.php?mode=reply url's in etc sitemap.

I have tried all settings and different combinations of rules over and over again but for some reason they keep getting added to the sitemap.

The rules I currently have in pace are:

p=
mode=
mark=
order=
highlight=
profile.php
privmsg.php
posting.php
view=previous
view=next
search.php
/DIY-superstore/
/blog/
posting.php?mode=reply
/forums/posting.php?mode=reply
forums/posting.php?mode=reply
?mode=reply
rss/news/digest/rss.xml
/forums/rss/news/digest/rss.xml
news/digest/forum.xml
/digest/
/news/
/forums/digest/
/forums/news/
/forums/digest/
/digest/
mode=reply
=reply
mode=
reply&f
post&f
resources
reply
/*posting.php?mode=?
/*posting.php?mode=*
/forums/*posting.php?mode=?
/forums/*posting.php?mode=*
forums/*posting.php?mode=?
forums/*posting.php?mode=*

I have also added the above to the "Exclude from sitemap extensions" and "crawl, but do not include URL's" but the posting.php url's keep appearing. Any ideas on how to exclude them?

Kind Regards
Re: Issue With Posting.php URL's in Sitemap
« Reply #1 on: March 12, 2021, 04:45:13 PM »
Hello,

please replace
Code: [Select]

/*posting.php?mode=?
/*posting.php?mode=*
/forums/*posting.php?mode=?
/forums/*posting.php?mode=*
forums/*posting.php?mode=?
forums/*posting.php?mode=*
with
Code: [Select]
posting.php?mode=
Re: Issue With Posting.php URL's in Sitemap
« Reply #2 on: March 12, 2021, 05:10:48 PM »
Hi

Many thanks for the update, will make the changes and run some tests

Kind Regards