Priorities
« on: June 22, 2010, 12:51:11 PM »
Hello,

Just a quick question.

Should the priorities of our main links (links present on our main menu bar) all be set to maxium priority as these are the most important pages?

Thanks,

S
Re: Priorities
« Reply #2 on: June 22, 2010, 05:50:23 PM »
Thank you for the reply admin.

I have one further question I'm hoping you can help with.

I have generated the sitemap for our site and used Unlimited depth etc.  This is a Joomla site.

8444 pages have been crawled which is great but we use a "directory" component to power a directory of locations on our site.

The issue I have is that the sitemap includes the following:


Code: [Select]
  <url>
       <loc>http://www.oursite.com/category/subcat/subcat/subcat/subcat/write-review</loc>
       <lastmod>2010-06-22T11:07:29+00:00</lastmod>
       <changefreq>daily</changefreq>
       <priority>0.6400</priority>
  </url>

This /write-review is a page that of course allows the user to write reviews, it isn't something that I feel should be included in the sitemap.

If I wanted to remove this, can I just manually delete it from the sitemap?  Or do I need to use the URL Exclude option?  And if I do, do I just enter:  /write-review, then regenerate the sitemap and this will be resolved?

The site map also includes the following things I'd like to remove:

/log-in/?return=aHR0cDovL3d3dy5tZXRhbHRyYXZlbGd1aW
/create-an-account
/forgotten-password

Can I just cut these out manually from the sitemap or do they have to be entered in Exclude URL in config? (e.g. enter /create-an-account, /forgotten-password, /log-in/)

Thanks
Re: Priorities
« Reply #3 on: June 22, 2010, 09:37:27 PM »
You should add this in Exclude URLs option:
Code: [Select]
write-review
return=
create-an-account
forgotten-password
and regenerate sitemap.
Re: Priorities
« Reply #4 on: June 23, 2010, 08:28:48 PM »
Thanks Admin.

It appears I am having some problems with this.

Originally, it crawled some 1700 pages until I changed the crawl depth settings to "0", then it crawled over 8000 url's.  Now, after recrawling it only crawls a maximum of 1746 URL's.

I have recrawled numerous times with different settings (such as totally clearning the Exlcude URL's) and it makes no difference.

Is there something else I should be doing aside to get it to crawl properly?

Thanks
Re: Priorities
« Reply #5 on: June 24, 2010, 12:48:09 PM »
Hello,

do you mean that some URLs are not included in sitemap (although they should be there)?
Re: Priorities
« Reply #6 on: June 24, 2010, 03:02:10 PM »
Yes, for some reason it was including over 8000 initially, not down to 1746.  I've run and rerun it, cleared out the xml files and so on and it hasn't made any difference.
Re: Priorities
« Reply #7 on: June 24, 2010, 05:06:26 PM »
Hello,

could you please PM me your generator URL and an example URL that is not included in sitemap and how it can be reached from homepage?