XML Sitemaps Generator

Author Topic: Not crawling all my pages  (Read 30362 times)

james3

  • Registered Customer
  • Approved member
  • *
  • Posts: 4
Not crawling all my pages
« on: August 16, 2006, 12:47:11 PM »
Hi,

Just a quick question, I have found that only 1515 pages are being crawled?? I have 2800 + pages.

What can I do get it to crawl all 2800+ page, I have notices it only crawls to a link depth of 2.

Any ideas?? (please) or advise.

Thanks a million
James
[external links are visible to admins only]

XML-Sitemaps Support

  • Administrator
  • Hero Member
  • *****
  • Posts: 10622
Re: Not crawling all my pages
« Reply #1 on: August 16, 2006, 08:53:07 PM »
Hello James,

please make sure that all your pages are linked together, i.e. it's possible to find every page starting from homepage. If it's not a problem in your case, please send me a private message with your generator instance URL/login and example link that is not included in sitemap.
Oleg Ignatiuk
www.xml-sitemaps.com
Send me a Private Message

For maximum exposure and traffic for your web site check out our additional SEO Services.

skullman1

  • Registered Customer
  • Approved member
  • *
  • Posts: 2
Re: Not crawling all my pages
« Reply #2 on: August 16, 2006, 09:47:20 PM »
I too am having that problem. I have four html pages but the first page is blank. I have it set to record 250 pages each

XML-Sitemaps Support

  • Administrator
  • Hero Member
  • *****
  • Posts: 10622
Re: Not crawling all my pages
« Reply #3 on: August 17, 2006, 10:16:43 PM »
Hello,

do you mean that first page of HTML sitemap is empty in your case? Please ensure that you have correct permissions set for this file (0666).
Oleg Ignatiuk
www.xml-sitemaps.com
Send me a Private Message

For maximum exposure and traffic for your web site check out our additional SEO Services.

james3

  • Registered Customer
  • Approved member
  • *
  • Posts: 4
Re: Not crawling all my pages
« Reply #4 on: August 22, 2006, 09:54:06 AM »
Hi all,

Just a quick question, will javascript links be followed.

I.E. <a href="javascript:printpopups('../../print_page/mp_s_kruger/bongani_mountain_lodge.htm');">
      PRINT PAGE</a>

Just a thought as to why some pages are not getting scanned.

cheers
james

xml-sitemaps

  • Administrator
  • Sr. Member
  • *****
  • Posts: 146
    • XML-Sitemaps.com
Re: Not crawling all my pages
« Reply #5 on: August 22, 2006, 11:27:03 AM »
Search engine robots can have problems with JavaScript links. You can check to see what links are picked up on different pages of your site with our search engine robot simulator at http://www.xml-sitemaps.com/se-bot-simulator.html

Philip Nicosia
www.xml-sitemaps.com

 

SMF 2.0.12 | SMF © 2014, Simple Machines
XHTML RSS WAP2