XML Sitemaps Generator

Author Topic: Is this Sitemap Generator cannot crawl URL without filename?  (Read 7339 times)

sales1047

  • Registered Customer
  • Approved member
  • *
  • Posts: 2
Is this Sitemap Generator cannot crawl URL without filename?
« on: March 20, 2012, 03:59:19 PM »
We tried to crawl our site and seems its ok with the URL with filename like [external links are visible to admins only]

However our site does .htaccess rewrite and thus some of the URL would be just the directory name like

[external links are visible to admins only]
[external links are visible to admins only]

which we found those could not be crawled.

May I know if there is any setting to fix this behavior?

sales1047

  • Registered Customer
  • Approved member
  • *
  • Posts: 2
Re: Is this Sitemap Generator cannot crawl URL without filename?
« Reply #1 on: March 21, 2012, 02:39:25 AM »
Hi, anyone can help here?

My site is [external links are visible to admins only] and all the URLs end with .html are ok but those just the directory name cannot be crawled as shown below:

[external links are visible to admins only]
[external links are visible to admins only]
[external links are visible to admins only]
etc.

Please help and thanks in advance.

XML-Sitemaps Support

  • Administrator
  • Hero Member
  • *****
  • Posts: 10625
Oleg Ignatiuk
www.xml-sitemaps.com
Send me a Private Message

For maximum exposure and traffic for your web site check out our additional SEO Services.

 

SMF 2.0.12 | SMF © 2014, Simple Machines
XHTML RSS WAP2