I've installed the sitemap generator on my domain [external links are visible to admins only], however when crawling the data, i got 3000+ pages to crawl and somehow the pages crawled seems redundant. How do i removed the redundancy in sitemap and how do i configure the sitemap generator to only crawl the important pages and ignore the unnecessary pages?
The weird redundant link i saw in my sitemap looks like this...
<loc>[external links are visible to admins only]</loc>
Thanks in advance.