• Welcome to Sitemap Generator Forum.
 

Sitemap not adding all urls

Started by info1728, May 13, 2011, 07:43:48 PM

Previous topic - Next topic

info1728

We are running a site with Joomla and Mosets Tree. We have over 2000 directory listings right now and I'm only getting 799 urls in the sitemap. I have trolled the forum looking for answers and most of the answers end with pm me. I have tried a couple things already, like adding ini_set() to add more execution time. Our memory is already set to a more than adequate number. I tried adding a second between urls.

It is still executing and here is a sample of what it is showing:

Links depth: 3
Current page: index.php?option=com_mtree&task=viewlink&link_id=866&Itemid=2
Pages added to sitemap: 799
Pages scanned: 1340 (42,432.8 KB)
Pages left: 1695 (+ 2849 queued for the next depth level)
Time passed: 0:37:29
Time left: 0:47:24
Memory usage: 4,700.7 Kb


The part of the url above in the code (link_id=866) is different for every page and we need all the pages to show in the sitemap. Do you have any suggestions? I have run this about 8 times since yesterday trying to get it to work correctly. I will be happy to pm you our info if you can help.

One other issue I came across - with Mosets tree, I had it set to view only 20 items per page and found that the sitemap didn't seem to get to other pages (there is a 'next' button on the bottom for the next 20 results in the site.)

XML-Sitemaps Support

Hello,

could you please PM me your generator URL and an example URL that is not included in sitemap and how it can be reached starting from homepage?

info1728

#2
Oh -  I see you ran it.
I guess I'm not understanding the 'parse' term for this. I see you added "task=viewlink" to the box "do not parse urls". Can you explain what that does?

XML-Sitemaps Support

It means that those pages are added directly in sitemap, without fetching from server - that allows to complete crawling faster.

info1728


info1746

#5
Hope you can help me out here.
My website is [ External links are visible to forum administrators only ]
Generator is placed at [ External links are visible to forum administrators only ]
The generator follows the path from [ External links are visible to forum administrators only ]
(including others) it will come to --> [ External links are visible to forum administrators only ], there are a lot of links there, but the generator does not include any of them.
All links goto [ External links are visible to forum administrators only ] Where 420 is variable here.
This is the primairy content of the site and is the most inportant to be indexed.
Unfortunately it does not include it these links in te sitemap.xml

Kind regards
(I included them manualy now, but for testing you can overwrite it.)
Thanks for your answer.
John
username and password on request

XML-Sitemaps Support

Hello,

you should remove "SID" from generator configuration->"Advanced settings" -> "Remove session ID from URLs" option.

info1746

Wonderfull, that  did the trick

Thanks
John