in our shop [external links are visible to admins only] the online tool indexes all articles like it should be, i.e. 61 pages with Xlayer products. But the stand-alone generator only indexes a fraction of those (10 pages with Xlayer products). How many pages the stand-alone indexes varies little and I can not see why one product page is indexed and the other is not.
I already tried to add a pause of 1 second between each request (under "fine tuning") to make sure that the stand-alone generator is not faster than the apache-server, but this made no difference (except the crawling time of course).
What could be the reason?