• Welcome to Sitemap Generator Forum.
 

Generator only indexing index.html and a few other link

Started by admin92, July 17, 2007, 03:47:59 PM

Previous topic - Next topic

admin92

Everytime I try to generate It only generates the index.html and 5 other links on the page. But its ignoring every other folder and even other pages in the root directory? any suggestions?

XML-Sitemaps Support

Hello,

make sure that all links point to the same domain  - even [ External links are visible to logged in users only ] and [ External links are visible to logged in users only ] are assumed to be different domains and cannot be included in the same sitemap.

admin92

All links are pointing to same domain Im just not sure why it doesnt scan the entire site
Does this not can all folders for html files??

xml-sitemaps

Try putting your homepage URL into our robots simulator to see what links are found on your page.

https://www.xml-sitemaps.com/se-bot-simulator.html

Bear in mind that whatever the results are on this is likely to be the same for search engine robots. If you have links that aren't picked up by this then the search engine robots are not going to find them either.

Possible causes are links embeded in Flash or Javascript menus.

The following is taken from Google's Webmaster Guidelines

"Use a text browser such as Lynx to examine your site, because most search engine spiders see your site much as Lynx would. If fancy features such as JavaScript, cookies, session IDs, frames, DHTML, or Flash keep you from seeing all of your site in a text browser, then search engine spiders may have trouble crawling your site."
Philip Nicosia