XML Sitemaps Generator

Author Topic: Standalone Sitemap Generator will not crawl site  (Read 10624 times)

jbfagan1

  • Registered Customer
  • Approved member
  • *
  • Posts: 4
Standalone Sitemap Generator will not crawl site
« on: March 27, 2011, 08:39:37 PM »
Hello - when I click the Run button for starting the generator it says Completed but does not crawl any pages - shows a return as follows:

Sitemap details
Request date:
27 March 2011, 20:26
Processing time:
0:00:01s
Pages indexed:
0
Sitemap files:
0
Pages size:
0.00Mb

Our website has about 700-800 pages.

The sitemap is there but dates back to February 2011.

Can anyone help? Ive checked all the permissions for the relevant folders.

XML-Sitemaps Support

  • Administrator
  • Hero Member
  • *****
  • Posts: 10625
Re: Standalone Sitemap Generator will not crawl site
« Reply #1 on: March 28, 2011, 10:30:47 AM »
Hello,

please let me know your generator URL/login in private message to check this.
Oleg Ignatiuk
www.xml-sitemaps.com
Send me a Private Message

For maximum exposure and traffic for your web site check out our additional SEO Services.

jbfagan1

  • Registered Customer
  • Approved member
  • *
  • Posts: 4
Re: Standalone Sitemap Generator will not crawl site
« Reply #2 on: March 28, 2011, 11:36:41 AM »
Hello - discovered the problem. at some stage my robots.txt file on my site for some unknown reason changed from

User-agent: *
Disallow:

to

User-agent: *
Disallow: /

Which meant that Sitemap Generator could not access any files! not entirely sure why this was the case but would recommend anyone having the same problem to check the robots.txt file...

 

SMF 2.0.12 | SMF © 2014, Simple Machines
XHTML RSS WAP2