XML Sitemaps Generator

Author Topic: Full site not crawled - urgent help required!  (Read 35245 times)

dd.mediaplus

  • Registered Customer
  • Jr. Member
  • *
  • Posts: 16
Full site not crawled - urgent help required!
« on: May 27, 2008, 03:21:55 PM »
Hi,

I have a domain at [external links are visible to admins only] and a forum at [external links are visible to admins only] and a news section at [external links are visible to admins only]

The sitemap is at the root domain and crawling the root and the forum perfectly. But unable to crawl the news section.

The news (word-press) section actual url is [external links are visible to admins only] but is redirected to [external links are visible to admins only] - so all the posts appear as [external links are visible to admins only] usinf .htaccess

Please advice me how to include the news section in the sitemap.

Regards.

Jack

XML-Sitemaps Support

  • Administrator
  • Hero Member
  • *****
  • Posts: 10625
Re: Full site not crawled - urgent help required!
« Reply #1 on: May 27, 2008, 07:03:27 PM »
Hello,

please PM me your generator URL so that I can have a detailed look.
Oleg Ignatiuk
www.xml-sitemaps.com
Send me a Private Message

For maximum exposure and traffic for your web site check out our additional SEO Services.

dd.mediaplus

  • Registered Customer
  • Jr. Member
  • *
  • Posts: 16
Re: Full site not crawled - urgent help required!
« Reply #2 on: May 27, 2008, 07:33:29 PM »
Sent you pm with generator url and details.

Thanks,

Jack.

XML-Sitemaps Support

  • Administrator
  • Hero Member
  • *****
  • Posts: 10625
Re: Full site not crawled - urgent help required!
« Reply #3 on: May 27, 2008, 07:58:46 PM »
Hello,

your news pages return 404 http code instead of code 200 (although page content is displayed), you can check that with http headers viewer.
« Last Edit: May 27, 2008, 08:01:15 PM by admin »
Oleg Ignatiuk
www.xml-sitemaps.com
Send me a Private Message

For maximum exposure and traffic for your web site check out our additional SEO Services.

dd.mediaplus

  • Registered Customer
  • Jr. Member
  • *
  • Posts: 16
Re: Full site not crawled - urgent help required!
« Reply #4 on: May 28, 2008, 08:12:33 AM »
I am using the permalink to change the news/index.php/topic.html  -> news/topic.html

Any help how this can be resolved...

Thnaks.

XML-Sitemaps Support

  • Administrator
  • Hero Member
  • *****
  • Posts: 10625
Re: Full site not crawled - urgent help required!
« Reply #5 on: May 29, 2008, 12:39:03 AM »
You should correct your news script to resolve the issue - currently it returns 404 http code. Even if those pages will be manually included in sitemap, search engines won't index them until you address the http code problem.
Oleg Ignatiuk
www.xml-sitemaps.com
Send me a Private Message

For maximum exposure and traffic for your web site check out our additional SEO Services.

dd.mediaplus

  • Registered Customer
  • Jr. Member
  • *
  • Posts: 16
Re: Full site not crawled - urgent help required!
« Reply #6 on: May 29, 2008, 07:41:11 AM »
Solved the http header issue which was coming from a mix of wordpress and .htaccess. Now the sitemap is crawling the pages.

Cheers.

XML-Sitemaps Support

  • Administrator
  • Hero Member
  • *****
  • Posts: 10625
Re: Full site not crawled - urgent help required!
« Reply #7 on: May 29, 2008, 10:06:22 PM »
Excellent!
Oleg Ignatiuk
www.xml-sitemaps.com
Send me a Private Message

For maximum exposure and traffic for your web site check out our additional SEO Services.

dd.mediaplus

  • Registered Customer
  • Jr. Member
  • *
  • Posts: 16
Re: Full site not crawled - urgent help required!
« Reply #8 on: May 30, 2008, 08:39:12 AM »
I believe I can use this script for my other sites also in the same server ... but when I uploaded it in another site in the same server - could not save the config details or even the login pass - if I do a direct edit of the config file it saves but when I try to start crawling from the sitemap page button - it redirects me back to the index page of config - permissions are all OK.

Please help.

Cheers.

XML-Sitemaps Support

  • Administrator
  • Hero Member
  • *****
  • Posts: 10625
Re: Full site not crawled - urgent help required!
« Reply #9 on: May 30, 2008, 06:23:35 PM »
Hello,

make sure that you have 0666 permissions set for config.inc.php file.
Oleg Ignatiuk
www.xml-sitemaps.com
Send me a Private Message

For maximum exposure and traffic for your web site check out our additional SEO Services.

dd.mediaplus

  • Registered Customer
  • Jr. Member
  • *
  • Posts: 16
Re: Full site not crawled - urgent help required!
« Reply #10 on: May 30, 2008, 07:15:21 PM »
Thats done - but some hoe for some reason its not writing the config file and also not crawling .

Regards.

XML-Sitemaps Support

  • Administrator
  • Hero Member
  • *****
  • Posts: 10625
Re: Full site not crawled - urgent help required!
« Reply #11 on: May 30, 2008, 10:42:25 PM »
Please try to reinstall sitemap generator on your another site (remove generator folder and reupload files)
Oleg Ignatiuk
www.xml-sitemaps.com
Send me a Private Message

For maximum exposure and traffic for your web site check out our additional SEO Services.

dd.mediaplus

  • Registered Customer
  • Jr. Member
  • *
  • Posts: 16
Re: Full site not crawled - urgent help required!
« Reply #12 on: May 31, 2008, 10:31:51 AM »
Do I need to remove the sitemap html/xml/txt/gz files generated earlier with your earlier version ? They are all writeable - if the new sitemaps are not generated then I will be left with nothing...

Regards.

XML-Sitemaps Support

  • Administrator
  • Hero Member
  • *****
  • Posts: 10625
Re: Full site not crawled - urgent help required!
« Reply #13 on: May 31, 2008, 01:24:12 PM »
You can rename them (or move to subfolder) and create empty sitemap files and set 0666 permissions for them.
Oleg Ignatiuk
www.xml-sitemaps.com
Send me a Private Message

For maximum exposure and traffic for your web site check out our additional SEO Services.

dd.mediaplus

  • Registered Customer
  • Jr. Member
  • *
  • Posts: 16
Re: Full site not crawled - urgent help required!
« Reply #14 on: May 31, 2008, 03:40:30 PM »
Done that no use - maybe ownership problem or something else... Sending you the genrator url by pm please check - the earlier sitemaps were 78 in number !

PHP execution time is also increased at 100 from 30 earlier.

Thanks.

 

SMF 2.0.12 | SMF © 2014, Simple Machines
XHTML RSS WAP2