How to set "parameters"?
« on: November 16, 2008, 11:29:30 PM »
I just want to index our root web. For example: rootweb.com but when I try the free site generator it also indexes all of our subwebs and even our forums (exceeding 500 pages) all located in our public_html directory.

How do I prevent (exclude) subwebs from being indexed? I couldn't see how to set the "parameters."

Thanks,

Kerry
« Last Edit: November 16, 2008, 11:46:06 PM by iKerry »
Re: How to set "parameters"?
« Reply #1 on: November 17, 2008, 01:21:33 PM »
Hello,

using free online sitemap generator you can exclude them with robots.txt file on your server.

With standalone generator version you can define that in configuration.
Re: How to set "parameters"?
« Reply #2 on: November 17, 2008, 08:58:08 PM »
Hi admin,

Just to be clear: If I add "exclude" certain pages and subwebs in a robots.txt file on my server and then run the  free site generator that it I won't get the "exceeded 500 pages" message?

Thanks,

Kerry
Re: How to set "parameters"?
« Reply #3 on: November 17, 2008, 10:20:54 PM »
Update:

If I add subwebs in my robots.txt file doesn't that mean that those subwebs won't be searched/indexed  by search engines?

Also, where do I find the "parameters" option mentioned in step 1:
Quote
Enter your full website URL and some optional parameters in the form below.
Perhaps that doesn't apply for the free version?

Kerry
« Last Edit: November 17, 2008, 10:25:57 PM by iKerry »
Re: How to set "parameters"?
« Reply #4 on: November 19, 2008, 08:48:38 AM »
Hello,

if you exclude certain URLs in robots.txt then search engines will NOT index them as well, so you should be very careful with that option.

With free online generator the only optional parameters are: frequency, last modification time, priority attribute.

The standalone script is a much more advanced tool, you can check the demo here: https://www.xml-sitemaps.com/generator-demo/
Re: How to set "parameters"?
« Reply #5 on: November 19, 2008, 09:06:30 AM »
Thanks Oleg,

I understand.

Kerry