Hi,
SG normaly indexed 1902 sites on my page. But there is a funny thing. If i activate Spider-friendly URLs in ReviewPost
[ External links are visible to forum administrators only ]
it seems to be SG hangs in a loop and after 4500 sites i stoped it.
Normaly a link looks like
reviews/showproduct.php/product/23/cat/7
But the SG buts other links behind this link like
reviews/showproduct.php/product/4/content/blogcategory/47/forum/
or
reviews/showproduct.php/product/23/content/blogcategory/47/galerie/
The directorys forum and galerie are different directorys on my server and not within the reviews directory.
I also got this problem on a friends shop. First i had a max_execution_time error and set it to 120. But still this error and SG keeps running and running and buts 3000 sites to the sitemap.
But in the shop i found no funny links at crawling...