pages not being indexed
« on: January 17, 2009, 01:53:12 PM »
I just purchased the unlimited site generator and am having problems getting all of my pages indexed.  When I used our complete url - I only was able to get 14 pages indexed (the same issue with the free version).  When I removed the www - I am now receiving anywhere from 200-700 pages (depending upon how long the crawler lasts) but our total should be closer to 4000.

We receive various errors - connection timed out, server hung up...  I have tried changing various settings in the config and I usually ended up with worse results.  I have since put the config back to default.

We don't host our own server so the ability to change the php.ini settings is not available.

So far this isn't working as promised - it is very frustrating.

Any help would be appreciated.
Re: pages not being indexed
« Reply #1 on: January 17, 2009, 02:36:59 PM »

make sure that all your links on the site are *consistent*, i.e. if you use non-www domain (, then ALL internal links should point to non-www domain too - otherwise they cannot be included in sitemap per xml sitemaps protocol.
Re: pages not being indexed
« Reply #2 on: January 17, 2009, 03:04:28 PM »
I am not the one who built the site but from what I can tell - everything does reference www. - it does seem to be consistent.

What about the issues with the crawler hanging, timing out, etc.?  I need a way to successfully get all of my pages indexed.