Unable to complete sitemap
« on: September 23, 2016, 08:37:25 PM »
Hi

I need help with creating sitemap for my website. I installed xml-sitemaps on my server and seems working fine. But my website has 500K+ pages. Sitemap creation runs for while then it stopped. I resume it again but again it stopped after some time. I increase memory size also in config file, currently it is 1024. But still unable to finish sitemap.

I really need help on this to complete sitemap. Thank you.

The following is output of sitemap creating process :

Resuming the last session (last updated: 2016-09-23 18:24:44)
196384 | 0 | 14,326,204.4 | 6:09:31 | 0:00:00 | 4 | 519,759.9 Kb | 195974 | 302329 | 519759
196400 | 139777 | 14,327,393.6 | 6:09:33 | 4:23:00 | 4 | 520,027.2 Kb | 195990 | 302363 | 520027
196420 | 139757 | 14,328,477.2 | 6:09:34 | 4:22:57 | 4 | 519,983.6 Kb | 196010 | 302373 | 519983
196440 | 139737 | 14,330,029.5 | 6:09:35 | 4:22:54 | 4 | 520,160.0 Kb | 196030 | 302420 | 520160
196460 | 139717 | 14,331,619.1 | 6:09:37 | 4:22:51 | 4 | 520,231.2 Kb | 196050 | 302471 | 520231
196480 | 139697 | 14,333,227.8 | 6:09:38 | 4:22:48 | 4 | 520,298.6 Kb | 196070 | 302514 | 520298
196500 | 139677 | 14,334,815.7 | 6:09:39 | 4:22:45 | 4 | 520,368.4 Kb | 196090 | 302559 | 520368
196520 | 139657 | 14,336,405.2 | 6:09:41 | 4:22:43 | 4 | 520,435.9 Kb | 196110 | 302604 | 520435
196540 | 139637 | 14,338,004.3 | 6:09:42 | 4:22:40 | 4 | 520,516.2 Kb | 196130 | 302666 | 520516
196560 | 139617 | 14,339,595.7 | 6:09:43 | 4:22:37 | 4 | 520,583.0 Kb | 196150 | 302724 | 520583
196580 | 139597 | 14,341,071.0 | 6:09:45 | 4:22:34 | 4 | 520,655.2 Kb | 196170 | 302757 | 520655
196600 | 139577 | 14,342,307.5 | 6:09:46 | 4:22:31 | 4 | 520,617.7 Kb | 196190 | 302777 | 520617
196620 | 139557 | 14,343,611.0 | 6:09:48 | 4:22:28 | 4 | 520,672.7 Kb | 196210 | 302801 | 520672
196640 | 139537 | 14,344,913.4 | 6:09:49 | 4:22:25 | 4 | 520,822.2 Kb | 196230 | 302826 | 520822
196660 | 139517 | 14,346,055.0 | 6:09:50 | 4:22:22 | 4 | 520,779.1 Kb | 196250 | 302840 | 520779
196680 | 139497 | 14,347,450.9 | 6:09:51 | 4:22:19 | 4 | 520,920.3 Kb | 196270 | 302872 | 520920
196700 | 139477 | 14,349,041.5 | 6:09:53 | 4:22:16 | 4 | 521,002.1 Kb | 196290 | 302928 | 521002
196720 | 139457 | 14,350,624.2 | 6:09:54 | 4:22:14 | 4 | 521,073.3 Kb | 196310 | 302981 | 521073
196740 | 139437 | 14,352,194.1 | 6:09:56 | 4:22:11 | 4 | 521,160.6 Kb | 196330 | 303032 | 521160
Re: Unable to complete sitemap
« Reply #2 on: September 24, 2016, 10:13:43 PM »
Thanks for replay.

There were no last line. This was only last part of output. I tried increasing memory for script,  I set this for 4GB and script is currently running.
If script breaks again then I will update here.
Re: Unable to complete sitemap
« Reply #3 on: September 26, 2016, 07:48:16 PM »
Hi,

I want to exclude few folder of my website from sitemap. But problem is online interface [ External links are visible to forum administrators only ].....com/generator/ is not saving information to generator.conf.

I typed in "Exclude URLs: " text for excluding urls and clicked on save button it is not saving to generator.conf. I tried changing permission of this file also.

If there a way I can directly add these exclude url to generator.conf. I mean should be the name of parameter and what will be syntax to add multiple folders to exclude.

Thanks

Re: Unable to complete sitemap
« Reply #4 on: September 26, 2016, 07:54:46 PM »
Hello,

please try to create a file name ".htaccess" in generator/ folder with one line:
RewriteEngine off
 
Re: Unable to complete sitemap
« Reply #5 on: September 26, 2016, 07:59:46 PM »
Thank you very much.

It worked.

Now interface is able to update generator.conf file.

 :) :)