XML Sitemaps Generator

Author Topic: crawling halts after some time  (Read 14159 times)

galaldoss

  • Registered Customer
  • Approved member
  • *
  • Posts: 1
crawling halts after some time
« on: December 19, 2008, 05:28:44 AM »
Hi,
I need your help for 2 points
1) The crawling halts after some time and thereafter nothing happens if I interrupt and resume crawling.

2) I want to exclude profile page URLs, I have 7k+ users and the sitemap is including their profiles which I want to avoid.

Please advise.
Site : http:.//worldslastchance.com/generator

Thanks in advance
Regads
Galal

XML-Sitemaps Support

  • Administrator
  • Hero Member
  • *****
  • Posts: 10622
Re: crawling halts after some time
« Reply #1 on: December 19, 2008, 08:11:15 PM »
Hello,

1. please try to increase memory_limit setting in PHP configuration on your server.

2. you should add your profiles URL substring in "Exclude URLs" option like:
Code: [Select]
profile.php
Oleg Ignatiuk
www.xml-sitemaps.com
Send me a Private Message

For maximum exposure and traffic for your web site check out our additional SEO Services.

 

SMF 2.0.12 | SMF © 2014, Simple Machines
XHTML RSS WAP2