XML Sitemaps Generator

Author Topic: Question  (Read 18806 times)


  • Approved member
  • *
  • Posts: 1
« on: May 18, 2008, 09:38:46 PM »
Hi, i would like to buy this script but first i need to know if i can set a cron job to crawl 50 000 pages a day and resume them for next day until the crawl end (i have about 10 millions different pages on my web site)

And what for the new pages ? can i make a another job that will only crawl page that have been created for the last day ?

Bassicly my site change each minute whit about 5-25 news pages.


XML-Sitemaps Support

  • Administrator
  • Hero Member
  • *****
  • Posts: 10904
Re: Question
« Reply #1 on: May 19, 2008, 10:54:34 PM »

sitemap generator cannot find "new" pages unless the whole site is recrawled.
You can setup a "maximum script running time" in configuration and setup a cron job, so it will work for say 30 minutes every time, then interrupt and resume the next day.
Oleg Ignatiuk
Send me a Private Message

For maximum exposure and traffic for your web site check out our additional SEO Services.


SMF 2.0.14 | SMF © 2017, Simple Machines