over reaching search
« on: January 29, 2009, 01:35:11 AM »
Crawler had captured 4600+ pages from my site.  I have around 1300  content pages.  crawler must be looping through interface links(purchase cart ect)  I want to have deep clean sitemap of only my content pages (1300 html) My home page is PHP and rest of site is dynamic using PHP,Mysql, Ajax,Iframes(2). HTML pages(static) were created for search purposes only.  Site users access content through dynamic pages. 
How to configure to capture all my content with meta info. Should be around 1300 html pages with inner links to page content.

Warm Regards,
fenwick
classicalrecordtrader.com
Re: over reaching search
« Reply #1 on: January 29, 2009, 04:08:59 PM »
Hi Admin

Any help with  producing my site map?
Re: over reaching search
« Reply #2 on: January 29, 2009, 07:38:23 PM »
Hello,

yo ushould use "Eclude URLs" option to avoid crawling of unnecessary pages (enter a substring of URLs and generator will not include them).