Full site not crawled - urgent help required!
« on: May 27, 2008, 04:21:55 PM »
Hi,

I have a domain at http://www.mysite.com and a forum at http://www.mysite.com/community and a news section at http://www.mysite.com/news

The sitemap is at the root domain and crawling the root and the forum perfectly. But unable to crawl the news section.

The news (word-press) section actual url is http://www.mysite.com/news/index.php but is redirected to http://www.mysite.com/news - so all the posts appear as http://www.mysite.com/news/post-url usinf .htaccess

Please advice me how to include the news section in the sitemap.

Regards.

Jack
Re: Full site not crawled - urgent help required!
« Reply #1 on: May 27, 2008, 08:03:27 PM »
Hello,

please PM me your generator URL so that I can have a detailed look.
Oleg Ignatiuk
www.xml-sitemaps.com
Send me a Private Message

For maximum exposure and traffic for your web site check out our additional SEO Services.
Re: Full site not crawled - urgent help required!
« Reply #2 on: May 27, 2008, 08:33:29 PM »
Sent you pm with generator url and details.

Thanks,

Jack.
Re: Full site not crawled - urgent help required!
« Reply #3 on: May 27, 2008, 08:58:46 PM »
Hello,

your news pages return 404 http code instead of code 200 (although page content is displayed), you can check that with http headers viewer.
« Last Edit: May 27, 2008, 09:01:15 PM by admin »
Oleg Ignatiuk
www.xml-sitemaps.com
Send me a Private Message

For maximum exposure and traffic for your web site check out our additional SEO Services.
Re: Full site not crawled - urgent help required!
« Reply #4 on: May 28, 2008, 09:12:33 AM »
I am using the permalink to change the news/index.php/topic.html  -> news/topic.html

Any help how this can be resolved...

Thnaks.
Re: Full site not crawled - urgent help required!
« Reply #5 on: May 29, 2008, 01:39:03 AM »
You should correct your news script to resolve the issue - currently it returns 404 http code. Even if those pages will be manually included in sitemap, search engines won't index them until you address the http code problem.
Oleg Ignatiuk
www.xml-sitemaps.com
Send me a Private Message

For maximum exposure and traffic for your web site check out our additional SEO Services.
Re: Full site not crawled - urgent help required!
« Reply #6 on: May 29, 2008, 08:41:11 AM »
Solved the http header issue which was coming from a mix of wordpress and .htaccess. Now the sitemap is crawling the pages.

Cheers.
Re: Full site not crawled - urgent help required!
« Reply #7 on: May 29, 2008, 11:06:22 PM »
Excellent!
Oleg Ignatiuk
www.xml-sitemaps.com
Send me a Private Message

For maximum exposure and traffic for your web site check out our additional SEO Services.
Re: Full site not crawled - urgent help required!
« Reply #8 on: May 30, 2008, 09:39:12 AM »
I believe I can use this script for my other sites also in the same server ... but when I uploaded it in another site in the same server - could not save the config details or even the login pass - if I do a direct edit of the config file it saves but when I try to start crawling from the sitemap page button - it redirects me back to the index page of config - permissions are all OK.

Please help.

Cheers.
Re: Full site not crawled - urgent help required!
« Reply #9 on: May 30, 2008, 07:23:35 PM »
Hello,

make sure that you have 0666 permissions set for config.inc.php file.
Oleg Ignatiuk
www.xml-sitemaps.com
Send me a Private Message

For maximum exposure and traffic for your web site check out our additional SEO Services.
Re: Full site not crawled - urgent help required!
« Reply #10 on: May 30, 2008, 08:15:21 PM »
Thats done - but some hoe for some reason its not writing the config file and also not crawling .

Regards.
Re: Full site not crawled - urgent help required!
« Reply #11 on: May 30, 2008, 11:42:25 PM »
Please try to reinstall sitemap generator on your another site (remove generator folder and reupload files)
Oleg Ignatiuk
www.xml-sitemaps.com
Send me a Private Message

For maximum exposure and traffic for your web site check out our additional SEO Services.
Re: Full site not crawled - urgent help required!
« Reply #12 on: May 31, 2008, 11:31:51 AM »
Do I need to remove the sitemap html/xml/txt/gz files generated earlier with your earlier version ? They are all writeable - if the new sitemaps are not generated then I will be left with nothing...

Regards.
Re: Full site not crawled - urgent help required!
« Reply #13 on: May 31, 2008, 02:24:12 PM »
You can rename them (or move to subfolder) and create empty sitemap files and set 0666 permissions for them.
Oleg Ignatiuk
www.xml-sitemaps.com
Send me a Private Message

For maximum exposure and traffic for your web site check out our additional SEO Services.
Re: Full site not crawled - urgent help required!
« Reply #14 on: May 31, 2008, 04:40:30 PM »
Done that no use - maybe ownership problem or something else... Sending you the genrator url by pm please check - the earlier sitemaps were 78 in number !

PHP execution time is also increased at 100 from 30 earlier.

Thanks.