Ultimate Generator skipping pages for some reason

[harrillj]

Hello,
I really like this tool but am having an issue.  For some reason I can not figure out, pages are being skipped and not written to any of the sitemap files.  My site is a PHP site.  The pages that are being skipped are nearly identical to pages that are not being skipped.  When I navigate the site, the link is not broken to the pages being skipped, nor does the online GUI show any broken links.

I have a page for each county in a particular state.  So for example, all the pages for Alabama and Arkansas (one php page per county) are being picked up and are on the sitemap.xml file, however, no pages from Alaska are on the file.  The pages are being created with "Save As" then a minor tweak, so in reality the pages are nearly identical. 

Something is preventing all 27 pages in Alaska, one per county, from getting crawled.

I tried to attach some examples but can not upload a PHP file.

Thanks in advance

[admin]

Hello,

please send me in Private Message the example URLs that are not indexed and your Sitemap Generator URL so that I can check this further.
Thanks!

[harrillj]

Hello,

Sitemap XML url is:

[external links are visible to admins only]

Some example of some urls that are being skipped are:
[external links are visible to admins only]
[external links are visible to admins only]
[external links are visible to admins only]

Some examples of URLs that NOT being skipped are:
[external links are visible to admins only]
[external links are visible to admins only]
[external links are visible to admins only]


I wanted to ask another questions.  If I recall, the very first time I ran the crawl from [external links are visible to admins only], I think these Alaska pages were crawled.  In addition, it appers that each sibsequent time I run the crawl I get different results for all pages, including Alaska pages never showing up at all.   My question is..... are prior crawls "remembered", that is troed in a file and then skipped on subsequent crawls?    Is there a way to start over from zero, as if no crawls have ever been run at all?

Thanks

[admin]

Hello,

Sitemap Generator doesn't remember the information from previous sessions and starts crawling from the scratch every time.
Please let me know the URL of the page that IS included in sitemap AND has link to one of the pages you listed.

Please note that there should be the links to all of your pages at the site, otherwise sitemap generator will not be able to find them.

[harrillj]

Hello,

The page that is in the sitemap, and links to the pages that are not in the sitemap, is:

[external links are visible to admins only]

On the above page, there is a link for each county in Alaska.  There is a separate page for each county in Alaska that is not being crawled, or so it seems.

I attached a jpeg screen capture of the missing pages from the files panel in Dreamweaver.

Thanks

[admin]

Hello,

the problem is with pages at your site like:
http://www.closing-agent-search.com/xxxxx.php?KTLogoutnow=1
these pages are setting the cookies that leads to a problem with next page retrieved.

You should just add the following to "Exclude URLs" and "Do not parse URLs" on Configuration page:

KTLogoutnow=1

and re-generate the sitemap.

Please let me know how it works.
Thanks!

[harrillj]

Looks like this did the trick!  Thanks much...

[admin]

You are welcome!