Quaestions about configuration
« on: December 18, 2012, 06:30:32 PM »
My site is [ External links are visible to forum administrators only ]

I have saved the xml and ror.xml sitemap files to the "/" file as follows...
[ External links are visible to forum administrators only ]
[ External links are visible to forum administrators only ]

I was wondering if I did this right.. For "Save sitemap to:" and "Your Sitemap URL:" I put
[ External links are visible to forum administrators only ]

Is this correct?

Also I wanted to create an HTML sitemap "Create HTML Sitemap:" and it shows:
C:\inetpub\vhosts\americancushioncompany.com\httpdocs\generator/data/sitemap.html

What do I put here because I assume I don't leave it at that?
Re: Quaestions about configuration
« Reply #1 on: December 19, 2012, 12:29:04 AM »
Hello,

for "save sitemap to" you should use file path instead of URL, i.e. in your case C:\....\sitemap.xml
Re: Quaestions about configuration
« Reply #2 on: December 19, 2012, 08:10:16 PM »
As for crawling, I have let it run for days now (tried a few times) and the pages crawled show more each time:

Pages crawled: 3727 (3726 added in sitemap), Queued: 4359, Depth level: 3

and it goes up each time but shouldnt it all go at once without having to restart?

Re: Quaestions about configuration
« Reply #3 on: December 19, 2012, 10:47:20 PM »
Hello,

it looks like your server configuration doesn't allow to run the script long enough to create full sitemap. Please try to increase memory_limit and max_execution_time settings in php configuration at your host (php.ini file) or contact hosting support regarding this.
 
Re: Quaestions about configuration
« Reply #4 on: December 20, 2012, 06:45:54 PM »
Can it download it in pieces?
Re: Quaestions about configuration
« Reply #5 on: December 21, 2012, 09:00:17 PM »
It can resume the process in case if it was interrupted, but generator needs to keep the full list of URLs to avoid crawling the same pages again and again.
Re: Quaestions about configuration
« Reply #6 on: January 10, 2013, 02:30:34 AM »
My sitemaps were created for ror, html and text but not for xml. Why is this?
Re: Quaestions about configuration
« Reply #7 on: January 11, 2013, 02:48:48 PM »
Hello,

please let me know your generator URL/login in private message to check this.
Re: Quaestions about configuration
« Reply #8 on: February 18, 2013, 03:52:15 PM »
I havent had time to mess with this but I am back now. Recently you helped me change the location of my Starting URL, Save sitemap to, etc but this is still not working. I have tried running the crawler on another computer at another location and still not working. Please let me know how I can get this to crawl the full site without stopping! Thanks
Re: Quaestions about configuration
« Reply #9 on: February 18, 2013, 04:09:50 PM »
You would need to update php configuration as mentioned above to allow the script running longer.
Re: Quaestions about configuration
« Reply #10 on: February 25, 2013, 08:43:33 PM »
My web host was wondering "what does the sitemap company suggest as the proper values for the memory_limit and max_execution_time settings?"
Re: Quaestions about configuration
« Reply #11 on: February 27, 2013, 11:05:48 AM »
It depends on website size, please try 256M and 1000 seconds.
Re: Quaestions about configuration
« Reply #12 on: March 02, 2013, 03:14:49 PM »
I am working with the server company but I went through with continuing the sitemap crawling in pieces and there are "0" in the queued so I dont know why it is telling me no sitemaps found still?
Re: Quaestions about configuration
« Reply #13 on: March 04, 2013, 07:17:18 AM »
Hello,

please let me know your generator URL/login in private message to check this.