XML Sitemaps Generator

Author Topic: Standalone Sitemap Generator Indexing 0 Pages  (Read 2543 times)

debbie2

  • Registered Customer
  • Approved member
  • *
  • Posts: 4
Standalone Sitemap Generator Indexing 0 Pages
« on: June 02, 2017, 07:22:11 PM »
Just as the title suggests, I am having trouble indexing my website. Accessing the page [external links are visible to admins only] isn't an issue, simply getting the crawler to see my pages appears to be an issue.

I am sure this is a simple issue but need direction.

debbie2

  • Registered Customer
  • Approved member
  • *
  • Posts: 4
Re: Standalone Sitemap Generator Indexing 0 Pages
« Reply #1 on: June 02, 2017, 07:27:31 PM »
I should mention that I have checked with the link [external links are visible to admins only] and the website, both http and https can be crawled.

XML-Sitemaps Support

  • Administrator
  • Hero Member
  • *****
  • Posts: 10917
Re: Standalone Sitemap Generator Indexing 0 Pages
« Reply #2 on: June 03, 2017, 06:58:05 AM »
Hello,

please let me know your generator URL/login in private message to check this.
Oleg Ignatiuk
www.xml-sitemaps.com
Send me a Private Message

For maximum exposure and traffic for your web site check out our additional SEO Services.

XML-Sitemaps Support

  • Administrator
  • Hero Member
  • *****
  • Posts: 10917
Re: Standalone Sitemap Generator Indexing 0 Pages
« Reply #3 on: June 03, 2017, 06:59:53 AM »
Update: all pages are blocked with robots.txt :
Disallow: /
Oleg Ignatiuk
www.xml-sitemaps.com
Send me a Private Message

For maximum exposure and traffic for your web site check out our additional SEO Services.

debbie2

  • Registered Customer
  • Approved member
  • *
  • Posts: 4
Re: Standalone Sitemap Generator Indexing 0 Pages
« Reply #4 on: June 05, 2017, 10:26:22 PM »
Thank you very much, we had recently migrated servers and it had not occurred to me to check our robots.txt file to make sure we updated it once the dust had settled.

My next issue has to do with an empty sitemap.xml file.

I have started the crawling for a small set of links so that testing is easier.

Currently it is retrieving 3 URLS: [external links are visible to admins only]

However, none of those are saved in the sitemap.xml file in the public_html/ directory: [external links are visible to admins only]


One thing I have noticed is that I change the permissions for sitemap.xml to 666, only to have it change to 644 once the crawler has been used. Is there any reason for this, and could that be where some of the issues are stemming from?

debbie2

  • Registered Customer
  • Approved member
  • *
  • Posts: 4
Re: Standalone Sitemap Generator Indexing 0 Pages
« Reply #5 on: June 06, 2017, 07:43:43 PM »
I just reinstalled everything and now it works.

-shrug-

Thank you for the help on the previous issue.

 

SMF 2.0.14 | SMF © 2017, Simple Machines
XHTML RSS WAP2