XML Sitemaps Generator

Author Topic: Sitemap won't crawl a specific folder  (Read 12199 times)

mark42

  • Registered Customer
  • Approved member
  • *
  • Posts: 2
Sitemap won't crawl a specific folder
« on: March 26, 2009, 11:25:44 AM »
I have installed the sitemap generator and it crawls all the pages on my site except the ones in a specific directory.

I have looked to see if there is any no follows and robot.txt but everything seems to be in order.

This is the most important directory on my site as it contains 20,000+ of my products.

[external links are visible to admins only]
WON@T CRAWL - [external links are visible to admins only]


do you have an suggestions of why this would happen?

thanks

mark42

  • Registered Customer
  • Approved member
  • *
  • Posts: 2
Re: Sitemap won't crawl a specific folder
« Reply #1 on: March 26, 2009, 03:09:47 PM »
problem solved. don't know how but its now working.

next problem.....

crawling stops at 1804 pages and won't go no further. all very frustrating!!!!!!!!!!!! :-\

XML-Sitemaps Support

  • Administrator
  • Hero Member
  • *****
  • Posts: 10624
Re: Sitemap won't crawl a specific folder
« Reply #2 on: March 27, 2009, 01:02:39 AM »
Hello,

please PM me your generator URL/login.
Oleg Ignatiuk
www.xml-sitemaps.com
Send me a Private Message

For maximum exposure and traffic for your web site check out our additional SEO Services.

 

SMF 2.0.12 | SMF © 2014, Simple Machines
XHTML RSS WAP2