XML Sitemaps Generator

Author Topic: Not crawl links on pages with Meta noindex, follow  (Read 10521 times)

spbstyle

  • Registered Customer
  • Approved member
  • *
  • Posts: 1
Not crawl links on pages with Meta noindex, follow
« on: May 26, 2015, 11:08:19 AM »
Hello,

The problem is in subject. This pages not crawl even if I put it into "Crawl, but do not include URLs:"

Thanx

XML-Sitemaps Support

  • Administrator
  • Hero Member
  • *****
  • Posts: 10624
Re: Not crawl links on pages with Meta noindex, follow
« Reply #1 on: May 27, 2015, 05:15:49 AM »
Hello,

could you please PM me your generator URL and an example URL that is not included in sitemap and how it can be reached starting from homepage?
 
Oleg Ignatiuk
www.xml-sitemaps.com
Send me a Private Message

For maximum exposure and traffic for your web site check out our additional SEO Services.

 

SMF 2.0.12 | SMF © 2014, Simple Machines
XHTML RSS WAP2