Won't crawl the whole site
« on: June 01, 2010, 11:31:43 PM »
I purchased the Unlimited edition the other day but I've never been able to crawl anymore than 1080 pages. My site is hundreds of thousands of pages long so I'm not quite sure of the problem.

Any suggestions?

Regards,
Mike Richardson
Re: Won't crawl the whole site
« Reply #1 on: June 02, 2010, 07:38:20 AM »
Hello,

could you please PM me your generator URL and an example URL that is not included in sitemap and how it can be reached from homepage?
Oleg Ignatiuk
www.xml-sitemaps.com
Send me a Private Message

For maximum exposure and traffic for your web site check out our additional SEO Services.
Re: Won't crawl the whole site
« Reply #2 on: June 03, 2010, 04:28:50 PM »
Hi,
I have same problem with my own site, I have thousands of pages yet only get 20 indexed by my standalone generator. I have included below my generator URL and also an URL that is NOT currently being indexed so you can check if I am doing anything wrong?

Location of generator:
http://www.expresscarentals.com/generator/

2 Examples of URLS - Not currently being indexed:

http://www.expresscarentals.com/locations/countries_i/ireland/index.html

And...

http://www.expresscarentals.com/locations/countries_i/ireland/Cork/cork_city_car_rental.html

I hope you can assist in helping me with this issue many thanks.

Re: Won't crawl the whole site
« Reply #3 on: June 03, 2010, 06:50:08 PM »
Hello,

I will also need login/password to enter generator interface, please PM it to me.
Oleg Ignatiuk
www.xml-sitemaps.com
Send me a Private Message

For maximum exposure and traffic for your web site check out our additional SEO Services.