Question

[keny]

Hi, i would like to buy this script but first i need to know if i can set a cron job to crawl 50 000 pages a day and resume them for next day until the crawl end (i have about 10 millions different pages on my web site)

And what for the new pages ? can i make a another job that will only crawl page that have been created for the last day ?

Bassicly my site change each minute whit about 5-25 news pages.

Thanks

[admin]

Hello,

sitemap generator cannot find "new" pages unless the whole site is recrawled.
You can setup a "maximum script running time" in configuration and setup a cron job, so it will work for say 30 minutes every time, then interrupt and resume the next day.