XML Sitemaps Generator

Author Topic: Full Crawl but no new Sitemap  (Read 17244 times)

Elanor

  • Registered Customer
  • Approved member
  • *
  • Posts: 6
  • Web Developer
Full Crawl but no new Sitemap
« on: May 29, 2008, 03:04:53 PM »
I've been having problems with permissions and files ever since we installed this. It works off and on, and I don't understand why. The main problem is it takes so frickin long to generate a site-map I can't just see if its working immediately (our site is rather large, only about 40k links in the sitemap, but about 100k pages to sort through - takes about a day)

I recieve the email after the cron job completes, and I get to the very bottom of the list and this is what I see:
Code: [external links are visible to admins only]
<h4>Completed</h4>Total pages indexed: 38832<br>Calculating changelog...PHP Warning:  Invalid argument supplied for foreach() in public_html/sitemap/pages/class.templates.inc.php(2) : eval()'d code on line 75

Warning: Invalid argument supplied for foreach() in public_html/sitemap/pages/class.templates.inc.php(2) : eval()'d code on line 75

XML-Sitemaps Support

  • Administrator
  • Hero Member
  • *****
  • Posts: 10624
Re: Full Crawl but no new Sitemap
« Reply #1 on: May 29, 2008, 10:08:32 PM »
Hello,

do you mean that sitemap is not updated after that and you have no new entries on Changelog page?
Oleg Ignatiuk
www.xml-sitemaps.com
Send me a Private Message

For maximum exposure and traffic for your web site check out our additional SEO Services.

Elanor

  • Registered Customer
  • Approved member
  • *
  • Posts: 6
  • Web Developer
Re: Full Crawl but no new Sitemap
« Reply #2 on: May 30, 2008, 12:50:23 PM »
Yes. When it ran on the 27th, it ran all the way through the site, got to the end said completed - and yet the sitemap is still dated May 15. I ran it again manually yesterday and again, the sitemap is still dated the 15th and no sitemap is shown to be in progress.
I was ignoring this problem before because half the time it would give me this error, yet the sitemap would still be updated.

What's wrong? ???

XML-Sitemaps Support

  • Administrator
  • Hero Member
  • *****
  • Posts: 10624
Re: Full Crawl but no new Sitemap
« Reply #3 on: May 30, 2008, 06:24:58 PM »
Make sure that you have 0666 permissions for sitemap.xml file and 0777 for generator/data/ folder.
Oleg Ignatiuk
www.xml-sitemaps.com
Send me a Private Message

For maximum exposure and traffic for your web site check out our additional SEO Services.

Elanor

  • Registered Customer
  • Approved member
  • *
  • Posts: 6
  • Web Developer
Re: Full Crawl but no new Sitemap
« Reply #4 on: June 02, 2008, 04:23:52 PM »
[external links are visible to admins only]
Make sure that you have 0666 permissions for sitemap.xml file and 0777 for generator/data/ folder.

Yes, both are set. However, I do recall having to change the owner of these folders in order to change their permissions via FTP to the user of the html account.. should they be owned by a different user? (this is on unix)

XML-Sitemaps Support

  • Administrator
  • Hero Member
  • *****
  • Posts: 10624
Re: Full Crawl but no new Sitemap
« Reply #5 on: June 02, 2008, 10:04:56 PM »
In case if you have 0666/0777 permissions, it should work with any user id.
Please try to reinstall generator in a new folder and recheck it.
Oleg Ignatiuk
www.xml-sitemaps.com
Send me a Private Message

For maximum exposure and traffic for your web site check out our additional SEO Services.

Elanor

  • Registered Customer
  • Approved member
  • *
  • Posts: 6
  • Web Developer
Re: Full Crawl but no new Sitemap
« Reply #6 on: June 05, 2008, 03:52:09 PM »
I've reuploaded the generator into a new folder and its running right now. Hopefully it works. Thanks for your help so far. I'll let you know the results.
 :-\

Elanor

  • Registered Customer
  • Approved member
  • *
  • Posts: 6
  • Web Developer
Re: Full Crawl but no new Sitemap
« Reply #7 on: June 06, 2008, 06:10:52 PM »
Same thing happened with new sitemap installation. The sitemap generator ran to the end, but did not generate a new sitemap.
If I click on the "Crawling" Tab it shows 39,000 links added to the sitemap, 0 left - but thats where it stopped, and its waiting to be restarted.

In my data/ folder I have crawl_dump.log and crawl_state.log but no sitemap log like usual.

What am I doing wrong?

XML-Sitemaps Support

  • Administrator
  • Hero Member
  • *****
  • Posts: 10624
Re: Full Crawl but no new Sitemap
« Reply #8 on: June 07, 2008, 12:53:08 AM »
Perhaps you have an endless loop of links on your site so the crawler cannot finish the process. Please let me know your generator URL/login in PM so that I can check that.
Oleg Ignatiuk
www.xml-sitemaps.com
Send me a Private Message

For maximum exposure and traffic for your web site check out our additional SEO Services.

Elanor

  • Registered Customer
  • Approved member
  • *
  • Posts: 6
  • Web Developer
Re: Full Crawl but no new Sitemap
« Reply #9 on: June 09, 2008, 12:34:57 PM »
Why would it give me a PHP error then? PM'd to you.

XML-Sitemaps Support

  • Administrator
  • Hero Member
  • *****
  • Posts: 10624
Re: Full Crawl but no new Sitemap
« Reply #10 on: June 10, 2008, 11:16:57 AM »
Replied to your PM.
Oleg Ignatiuk
www.xml-sitemaps.com
Send me a Private Message

For maximum exposure and traffic for your web site check out our additional SEO Services.

 

SMF 2.0.12 | SMF © 2014, Simple Machines
XHTML RSS WAP2