• Welcome to Sitemap Generator Forum.
 

site map eats memory + creats array on crawling page

Started by steven.morris, August 28, 2007, 05:51:15 PM

Previous topic - Next topic

steven.morris

Script problems

when i first loaded the script it worked great it crawled all 49K plus pages without a problem. it didnt show any info about memory the first time i ran the [ External links are visible to forum administrators only ] however it seems to eat memory like  alzheimer's  and screw up and create the array in the crawling page

the memory problem has been sorted but I dont understand how it can not have a problem on first install, but when trying to run it a second time has a problem. I have redownloaded and reinstalled the script
example of what im getting on the crewling page:

Links depth: 5
Current page: index.php?Title=PNY+flash+memory+card+-+512+MB+-+miniSD&pid=0000081282
Pages added to sitemap: 36680
Pages scanned: 36680 (6,788.5 Kb)
Pages left: 3733 (+ 5004 queued for the next depth level)
Time passed: 27:03
Time left: 2:45
Memory usage: 32,248.0 Kb
array ( 'url_ind' => 6268, 'urls_list' => array ( 0 => 'index.php?Title=Joe+Browns+Combat+Jacket&pid=0000054306', 1 => 'index.php?Title=Joe+Browns+Cote+D%27Azur+Cami&pid=0000054307', 2 => 'index.php?Title=Joe+Browns+Crazy+Crinkle+Cami&pid=0000054308', 3 => 'index.php?Title=Joe+Browns+Crochet+Dress&pid=0000054309', 4 => 'index.php?Title=Joe+Browns+Crop+Crush+Shirt&pid=0000054310', 5 => 'index.php?Title=Joe+Browns+Crop+Summer+Shirt&pid=0000054311', 6 => 'index.php?Title=Joe+Browns+Cross+Pebble+Necklace&pid=0000054312', 7 => 'index.php?Title=Joe+Browns+Crossover+Tee&pid=0000054313', 8 => 'index.php?Title=Joe+Browns+Crown+Belt&pid=0000054314', 9 => 'index.php?Title=Joe+Browns+Crucifix+T-


I would like to get it back to the state when i first installed it where it just did what it was supposed to do . any help would be greatly appriciated.

XML-Sitemaps Support

Hello,
Quotenow however it seems to eat memory like  alzheimer's  and screw up and create the array in the crawling page
did you change anything at that point?

steven.morris

no, the only thing that changed was i added more pages to the site

steven.morris

ive also reinstalled the script from scratch and still the same, the crawl_dump.log seems to be writern to, should there be any other files in the data directory ? theres the placeholder file and dump_log

XML-Sitemaps Support

Hello,

crawl_dump.log is a temporary file that sitemap generator creates and updated during crawling. Please let me know your generator URL in private message so that I can check it.

steven.morris

tried to send a PM to admin dont know if it worked as nothing in outbox