15 million post

[tony3433]

Hello,

Just about to start using this powerful script although i bought it over 2yrs ago. My questions..

My new website is going to have about 15-18million post. which hints and tips will you offer me  to get the best out of this sofware considering this post size. Any particulat settings to help me get the best including server settings-e.g -max-excution, etc. The site will have 40 categories each category with a post of 300k.

Regards
Tony

[XML-Sitemaps Support]

Hello,

with website of this size the best option is to create a limited sitemap - with "Maximum depth" or "Maximume URLs" option limited so that it would gather about 200-300,000 URLs, which would be main pages representing "roadmap" sitemap for search engines.

[tony3433]

Thanks for the advise. Please which section of the script  below should i complete? and what should i put.?

• Other Sitemap Types (click to expand)
  
• Sitemap Entry Attributes (click to expand)
  
• Miscellaneous Settings (click to expand)
  
• Narrow Indexed Pages Set (click to expand)
  
• Crawler Limitations, Finetune (click to expand)
  
• Advanced Settings (click to expand)
  
  
  Thanks

[XML-Sitemaps Support]

Hello,

I'd recommend to keep all setting in default state, except for "Maximum pages" setting which will be limited in this case.

[tony3433]

Thanks

[tony3433]

is it possible to have a url one can use with cron services  instead of the /usr/bin/php /home/site/public_html/generator/runcrawl.php   .  i use setcron for all my cron jobs and i normally have url as [ External links are visible to forum administrators only ] for running the cron . is there any format to use with this service assuming the script is installed in  [ External links are visible to forum administrators only ]

[XML-Sitemaps Support]

Hello,

you can use [ External links are visible to logged in users only ]

However, command line cron task is recommended since it's running in less restricted environment usually.