runcrawl error
« on: December 10, 2020, 02:08:11 PM »
So I've been trying to generate a sitemap, but the sitemap that results does not include a significant part of my site.

I notice that some forum recommendations for this problem include running runcrawl.php. When I try that, I get the following error:

/generator/runcrawl.php: 102: Syntax error: redirection unexpected (expecting word)

Any ideas?

Thanks!

~paul
Re: runcrawl error
« Reply #2 on: December 11, 2020, 04:55:08 AM »
I used:
/usr/bin/php /usr/home/xxxx/public_html/example.com/generator/runcrawl.php

As I recall, I found it on the same page with the Start Creating Sitemap button.

Thanks!

~paul
Re: runcrawl error
« Reply #3 on: December 11, 2020, 08:53:08 AM »
Please try
php /usr/home/xxxx/public_html/example.com/generator/runcrawl.php
Re: runcrawl error
« Reply #4 on: December 11, 2020, 01:02:12 PM »
Thanks Oleg.

When I try that, I get essentially the same result as when I use the "Start Creating Sitemap" button at the create sitemap page.

The output on the command line shows:
Resuming the last session (last updated: 2020-12-10 05:01:26)

// Numbers with what looks to be the timing

<h4>Crawling phase completed</h4>Total pages indexed: 1235

and so on.

I am still not crawling the entire site as there are a lot of pages that are not included.

Thank you for your assistance.

~Paul
Re: runcrawl error
« Reply #5 on: December 11, 2020, 05:28:42 PM »
Hello,

could you please PM me your generator URL and an example URL that is not included in sitemap and how it can be reached starting from homepage?