Crawling stops all the time

[ViVa Computers]

Hello, I have the unlimited crawler script running on my site. In the beginning it is working fine but after a while it gets stuck when it reaches a area on my site. This area is were the shop is.
I tried to ban it from scanning but that doesn't work but actually I would like it to be scanned.

What can I do?

The site is [ External links are visible to forum administrators only ] and if you want I can give in private access to the generator.

[XML-Sitemaps Support]

Hello,

it looks like your server configuration doesn't allow to run the script long enough to create full sitemap. Please try to increase memory_limit and max_execution_time settings in php configuration at your host (php.ini file) or contact hosting support regarding this.
 

[ViVa Computers]

Ok I will do that but what is strange is also that I bought the web crawler service but it only comes to 6 pages while the unlimited script comes til far more?

[XML-Sitemaps Support]

Hello,

could you please PM me an example URL that is not included in sitemap and how it can be reached starting from homepage?
 

[ViVa Computers]

Hello Oleg,

I have send you a PM. How can I prevent the script from crawling certain areas of the site? If I can do that then for sure it will not be a problem.

[XML-Sitemaps Support]

You can use "Exclude URLs" setting to block parts of website from crawling.

[sebastijan.stembal]

Hi.

i already increase memory and limit time and i have still the same issue - stuck

any help?

brS

[XML-Sitemaps Support]

Hello, 

in this case I would recommend to run generator in command line if you have ssh access to your server.

[sebastijan.stembal]

yes this is work for me.
BR

Hello, 

in this case I would recommend to run generator in command line if you have ssh access to your server.