Memory Exausted

[admin]

Hi,

when talking about optimizing of your website I don't mean to reduce the number of pages on your site! I mean the optimization of the script that works at your site to produce pages (I assume it is database-driven). For instance, if the page on your site is prepared in average 2 seconds, then for 500,000 pages it will take AT LEAST 1,000,000 seconds (more than 11 days). But if your site produces the pages within 0,2 seconds each (because of software or hardware improvement), then the same sitemap will be created within 2 days (approximately).

[bug-software]

Still going

'146295',
'374660',
'881,603.8',
'12447:24',
'4860:23',
'6',
'327,682.6 Kb',
'374653',
'110349'
);</script>

And I don't think I can improve it more. It's a Dediated Dual Xeon with 1GB of Ram just for this site.

And I am starting to get concerned because the server begins to slow down.

I would really love the resume feature or at least that the memory wouldn't be eat this much.,

Right now the script is using '327,682.6 Kb'  and it's starting to slow it down.

Is there any change we can write some sort of written agreement and you disclose the source code? We really need the site indexed and I am guessing in 1 or 2 days it will stop.

I don't mind the time. What I can't face it anymore is the fact that the system locks and I start all over again so I kind of need a backup plan.

[admin]

Hi,

sure, I know it's too bad for a huge site to have sitemap generation interrupted without a chance to continue from the break point.
I will make sure to include the script state backup with a "resume crawling" feature in the next release. Is that ok?

[bug-software]

Yes. That would be great.

[bug-software]

By the way, it crashed

'144335',
'376620',
'885,526.8',
'12687:12',
'4862:12',
'6',
'328,878.0 Kb',
'376613',
'112434'
);</script>

[admin]

Hi,

please check the new generator version - the "resume operation" feature is added to 1.1.0 release.