Including pages
« on: October 21, 2006, 01:03:22 PM »
Okay, second major issue before I can start setting up a cron job and automatically pinging Google, knowing that my sitemap doesn't need manual editing by me afterwards.

Issue 2:  How to include pages but not "include only"

Every one of my pages has translated versions in other languages.  Each one takes a while to process.  A wonderful feature of your sitemap generator is that I can choose not to parse these pages by simply including their foreign language substring (e.g., /es/) but still have them added to my sitemap.  That is so great.  That makes sitemap generation only 20 minutes vs. 20 hours with other less-configurable sitemap generators.

But ...

There are a handful of pages that are not showing up in my sitemap because they are only linked to from these translated pages.  Since these translated pages are not being parsed, the generator doesn't pick them up.

I can't put them into "Include ONLY" since the generator will index ONLY the urls in this field.

What I need is an "Include" field, where I can add urls the generator cannot see.  Is there a way to do this?  Otherwise, I have to manually add them to the sitemap after every generation.
Re: Including pages
« Reply #1 on: October 22, 2006, 02:08:50 AM »

the "Include only" option is desingned to "filter" the pages and include only certain page types in sitemap. It doesn't allow to add links that are not found during the crawling.
One of the options would be to create a special html page with links to all these pages and link to this "mini-sitemap" somewhere so that generator will find it. As a result, it will find and index all linked pages as well.
Re: Including pages
« Reply #2 on: October 22, 2006, 11:13:36 AM »
Mmm... yes, I see.  Thank you.