XML Sitemaps Generator

Author Topic: Crawl and Directory problems  (Read 10386 times)

peterfarkasovsky

  • Registered Customer
  • Approved member
  • *
  • Posts: 1
Crawl and Directory problems
« on: July 09, 2009, 06:32:26 PM »
Hi!

I'm having some problems running the crawl. It seems to be scanning/crawling a lot more pages than I actually have on the site and making up directories. I copied on from the progress screen:

Current page: newemiplasticscom/eoat/newemiplasticscom/eoat/conveyor/forms.htm

eoat and conveyor are both subdirectories under newemiplasticscom. The paths should be newemiplasticscom/eoat or newemiplasticscom/conveyor. Maybe I have the paths on the Configuration page wrong. How can I tell what they are supposed to be?



« Last Edit: July 09, 2009, 07:24:22 PM by xml-sitemaps »

XML-Sitemaps Support

  • Administrator
  • Hero Member
  • *****
  • Posts: 10622
Re: Crawl and Directory problems
« Reply #1 on: July 10, 2009, 06:46:26 PM »
Hello,

try to add this in "Exclude URLs" setting:
Code: [Select]
newemiplasticscom.*newemiplasticscom
Oleg Ignatiuk
www.xml-sitemaps.com
Send me a Private Message

For maximum exposure and traffic for your web site check out our additional SEO Services.

 

SMF 2.0.12 | SMF © 2014, Simple Machines
XHTML RSS WAP2