• Welcome to Sitemap Generator Forum.
 

why my new pages aren't being index

Started by Judy114, March 27, 2006, 06:29:20 PM

Previous topic - Next topic

Judy114

I'm new to sitemaps and need of help.

I have an old site I'm updating from asp to aspx.  I've made a sitemap and did notice google crawling my site but new pages haven't been index.  I was concerned about the old pages so in reading up I used the following code to redirect to new pages:

<%@ Language=VBScript %>
<%
response.status="301 moved permanently"
Response.AddHeader "Location", "[ External links are visible to forum administrators only ]"
%>

AGAIN MY PAGES DON'T SHOW UP, so I made this robot.txt file to ignore old pages but this robot.txt validator shows nothing but errors  [ External links are visible to forum administrators only ]  this is my robot.txt file, the pages I don't want google to index:

# robots.txt for [ External links are visible to forum administrators only ]
User-agent: *
Disallow:
Disallow: /cgi-bin/
Disallow: /Inspections/home_inspection.asp
Disallow: /Inspections/home_inspection_Air_Quality_Testing.asp
Disallow: /Inspections/home_inspection_Asbestos.asp
Disallow: /Inspections/home_inspection_Benefit_Gas_Fireplace.asp
Disallow: /Inspections/home_inspection_Buy_Selling.asp
Disallow: /Inspections/home_inspection_CheckList.asp
Disallow: /Inspections/home_inspection_Cooling_Alternatives.asp
Disallow: /Inspections/home_inspection_Franchise.asp
Disallow: /Inspections/home_inspection_Hiring_Inspector.asp
Disallow: /Inspections/home_inspection_Lead_Paint.asp
Disallow: /Inspections/home_inspection_NewHome.asp
Disallow: /Inspections/home_inspection_Oil_Tanks.asp
Disallow: /Inspections/home_inspection_Radon_Gas.asp
Disallow: /Inspections/home_inspection_reduce_basement_himidity.asp
Disallow: /Inspections/home_inspection_Roofing_Maintenance.asp
Disallow: /Inspections/home_inspection_Safety_security.asp
Disallow: /Inspections/home_inspection_Saving_Energy.asp
Disallow: /Inspections/home_inspection_Summer_Comfort.asp
Disallow: /solutions/asbestos.asp
Disallow: /solutions/become_home_inspector.asp
Disallow: /solutions/benefit_gas_fireplace.asp
Disallow: /solutions/cooling_alternatives.asp
Disallow: /solutions/gas_fireplace.asp
Disallow: /solutions/home_buying_selling.asp
Disallow: /solutions/home_buying_selling1.asp
Disallow: /solutions/home_inspections.asp
Disallow: /solutions/home_inspections_01.asp
Disallow: /solutions/home_inspections_02.asp
Disallow: /solutions/home_inspections_br.asp
Disallow: /solutions/indoor_air_quality_testing_toronto.asp
Disallow: /solutions/lead_paint.asp
Disallow: /solutions/mold_removal.asp
Disallow: /solutions/radon_gas.asp
Disallow: /solutions/reduce_basement_humidity.asp
Disallow: /solutions/roofing_maintenance.asp
Disallow: /solutions/safety_security.asp
Disallow: /solutions/saving_energy.asp
Disallow: /solutions/summer_comfort.asp
Disallow: /solutions/underground_oil_tanks.asp

WHAT ELSE SHOULD I BE DOING TO GET MY NEW ASPX PAGES TO BE INDEX?

Judy

XML-Sitemaps Support

Hello Judy,

you should remove this line from robots.txt:
QuoteDisallow:
because empty value will disallow ALL pages from being indexed.

Judy114

thanks, THAT I SUPPOSE WAS HUGE.  the validator seems to have fixed all my errors except for line one, is there a problem with the first line of my robot.txt? 

any ideas as to why my new pages aren't being indexed?  I don't think google has used my robot.txt file yet, just worked on that this morning.  I was thinking it won't index because it's to similar to old pages. 

Could it possibly take a long time to index new pages even with a sitemap?

XML-Sitemaps Support

Hello,

do you mean indexing pages in Search Engines (google)? If so, of course it may take quite some time until they appear in the listings.