I have a phpbb forum with 10K+ topics that I would like to index (urls are static format not dynamic)

I have created a script that lists links for those topics at 1000 per page

Assuming that url format for the script is [ External links are visible to forum administrators only ]
and at the bottom of the first listall.php page is a link to the next liastall.php page (such as [ External links are visible to forum administrators only ]) to get to the next 1000 topics links, and so on

My question is: is it possible to configure the generator to only traverse and index links shown on this script pages (i.e. visit and index: listall.php, listall.php?page=2, listall.php?page=3, etc)?
If yes, what configuration to set for that?

Thanks for any tips

assuming that you have "viewtopic.php?t=X" links on those pages, you can add "viewtopic.php" into "Do not parse" option. In this case the URLs will still be included in sitemap, but generator won't crawl them.
Oleg Ignatiuk
Send me a Private Message

For maximum exposure and traffic for your web site check out our additional SEO Services.