~GW~

*
  • *
  • 3
  • If you don't ask.......
Error on crawler execution
« on: January 28, 2011, 09:03:43 AM »
Hi Guys

I am trying to work out why I am getting an error with our sitemap crawler.

This is a fresh NEW install of the sitemap generator with version 5.

There was an error while retrieving the URL specified: [ External links are visible to forum administrators only ]
Error message: Error opening socket to southwestphotography.com.au
HTTP headers follow:

HTTP output:

It may be something to do with the htaccess, I am not sure, here it is for your info.  The page loads fine, the sitemap works on other sites, so it is a specific site issue, but cannot work out what with exception to maybe the htaccess fie in use?

Code: [Select]
# modified php.ini values
suPHP_ConfigPath /home/swpname/public_html
<Files php.ini>
order allow,deny
deny from all
</Files>

# Compression
<IfModule mod_deflate.c>
  <FilesMatch "\.(js|css)$">
  SetOutputFilter DEFLATE
  </FilesMatch>
</IfModule>

# Cache
<IfModule mod_headers.c>
  <FilesMatch "\.(ico|pdf|flv|jpg|jpeg|png|gif|swf|mp3|mp4)$">
    ExpiresDefault A7257600
    Header set Cache-Control "max-age=7257600"
    Header unset Last-Modified
  </FilesMatch>
  <FilesMatch "\.(pdf)$">
    ExpiresDefault A2419200
    Header set Cache-Control "max-age=2419200"
  </FilesMatch>
  <FilesMatch "\.(js|css)$">
    ExpiresDefault A14400
    Header set Cache-Control "max-age=14400"
  </FilesMatch>
  <FilesMatch "\.(html|htm|txt)$">
    ExpiresDefault A432000
    Header set Cache-Control "max-age=432000, must-revalidate"
  </FilesMatch>
  <FilesMatch "\.(pl|php|cgi|spl|scgi|fcgi)$">
    ExpiresDefault A0
    Header set Cache-Control "no-store, no-cache, must-revalidate, max-age=0"
    Header set Pragma "no-cache"
  </FilesMatch>
</IfModule>

# Zenphoto htaccess
# htaccess file version 1.4.0;

<IfModule mod_rewrite.c>
  RewriteEngine On
 
  RewriteBase /
 
  RewriteRule ^admin/?$                       zp-core/admin.php [R,L]
 
  RewriteCond %{REQUEST_FILENAME} -d
  RewriteRule ^albums/?(.+/?)?$ $1 [R=301,L]

  RewriteCond %{REQUEST_FILENAME} -f [OR]
  RewriteCond %{REQUEST_FILENAME} -d
  RewriteRule ^.*$ - [L]
 
  ##### put no rules before this line #######
 
 
  #######################
  #Rewrite rules for sitemap.php
  RewriteRule ^sitemap\.php                  index.php?sitemap [L,R=301]
  RewriteRule ^sitemap1\.php                  index.php?sitemap1 [L,R=301]
  RewriteRule ^sitemap2\.php                  index.php?sitemap2 [L,R=301]
  RewriteRule ^sitemap3\.php                  index.php?sitemap3 [L,R=301]
 
  #Rewrite rules for removed RSS files
RewriteCond %{QUERY_STRING} albumtitle=(.*)&albumname=(.*)&lang=".$lang
RewriteRule ^rss\.php/?$                index.php?rss&albumtitle=%1&albumname=%2&lang=%3 [L,R=301]

RewriteCond %{QUERY_STRING} albumtitle=(.*)&folder=(.*)&lang=(.*)
RewriteRule ^rss\.php/?$                index.php?rss&albumtitle=%1&folder=%2&lang=%3 [L,R=301]

RewriteCond %{QUERY_STRING} albumtitle=(.*)&albumname=(.*)&lang=(.*)
RewriteRule ^rss\.php/?$                index.php?rss&albumtitle=%1&albumname=%2&lang=%3 [L,R=301]

RewriteCond %{QUERY_STRING} folder=(.*)&lang=(.*)&albumsmode
RewriteRule ^rss\.php/?$                index.php?rss&folder%1&lang=%2&albumsmode [L,R=301]

  RewriteCond %{QUERY_STRING} withimages&lang=(.*)
RewriteRule ^rss-news\.php/?$             index.php?rss-news&withimages&lang=%1 [L,R=301]

RewriteCond %{QUERY_STRING} id=(.*)&title=(.*)&type=(.*)&lang=(.*)
RewriteRule ^rss-comments\.php/?$         index.php?rss-comments&id=%1&title=%2&type=%3&lang=%4 [L,R=301]

  RewriteCond %{QUERY_STRING} type=(.*)&lang=(.*)
RewriteRule ^rss-comments\.php/?$         index.php?rss-comments&type=%1&lang=%2 [L,R=301]
 
  RewriteCond %{QUERY_STRING} lang=(.*)&albumsmode
RewriteRule ^rss\.php/?$                index.php?rss&lang=%1&albumsmode [L,R=301]

  RewriteCond %{QUERY_STRING} lang=(.*)&category=(.*)
RewriteRule ^rss-news\.php/?$             index.php?rss-news&lang=%1&category=%2 [L,R=301]
 
  RewriteCond %{QUERY_STRING} lang=(.*)
RewriteRule ^rss\.php/?$                index.php?rss&lang=%1 [L,R=301]
RewriteRule ^rss-news\.php/?$             index.php?rss-news&lang=%1 [L,R=301]
  #######################
 
  RewriteRule index\.php$                     index.php [L,QSA]
  RewriteRule ^page/([0-9]+)/?$               index.php?page=$1 [L,QSA]
  RewriteRule ^page/([A-Za-z0-9_\-]+)/?$      index.php?p=$1 [L,QSA]
  RewriteRule ^page/([A-Za-z0-9_\-]+)/([0-9]+)/?$      index.php?p=$1&page=$2 [L,QSA]
  RewriteRule ^(.*)/page/([0-9]+)/?$          index.php?album=$1&page=$2 [L,QSA]
  RewriteRule ^(.*)/page/([A-Za-z0-9_\-]+)/?$      index.php?album=$1&p=$2 [L,QSA]
   
  #### Rewrite rule addition for search
  RewriteRule ^page/search/fields([0-9]+)/(.*)/([0-9]+)/?$      index.php?p=search&searchfields=$1&words=$2&page=$3 [L,QSA]
  RewriteRule ^page/search/fields([0-9]+)/(.*)/?$        index.php?p=search&searchfields=$1&words=$2 [L,QSA]
  RewriteRule ^page/search/archive/(.*)/([0-9]+)/?$      index.php?p=search&date=$1&page=$2 [L,QSA]
  RewriteRule ^page/search/archive/(.*)/?$         index.php?p=search&date=$1 [L,QSA]
  RewriteRule ^page/search/tags/(.*)/([0-9]+)/?$   index.php?p=search&searchfields=tags&words=$1&page=$2 [L,QSA]
  RewriteRule ^page/search/tags/(.*)/?$ index.php?p=search&searchfields=tags&words=$1 [L,QSA]
  RewriteRule ^page/search/(.*)/([0-9]+)/?$      index.php?p=search&words=$1&page=$2 [L,QSA]
  RewriteRule ^page/search/(.*)/?$             index.php?p=search&words=$1 [L,QSA]
  #### Rewrite additions for zenpage
  RewriteRule ^pages/?$                        index.php?p=pages [L,QSA]
  RewriteRule ^pages/(.*)/?$                   index.php?p=pages&title=$1 [L,QSA]
  RewriteRule ^news/?$                     index.php?p=news [L,QSA]
  RewriteRule ^news/([0-9]+)/?$                index.php?p=news&page=$1 [L,QSA]
  RewriteRule ^news/category/(.*)/([0-9]+)/?$  index.php?p=news&category=$1&page=$2 [L,QSA]
  RewriteRule ^news/category/(.*)/?$           index.php?p=news&category=$1 [L,QSA]
  RewriteRule ^news/archive/(.*)/([0-9]+)/?$   index.php?p=news&date=$1&page=$2 [L,QSA]
  RewriteRule ^news/archive/(.*)/?$            index.php?p=news&date=$1 [L,QSA]
  RewriteRule ^news/(.*)/?$                    index.php?p=news&title=$1 [L,QSA]
  ####
  RewriteRule ^(.*)/image/(thumb|[0-9]{1,4})/([^/\\]+)$    zp-core/i.php?a=$1&i=$3&s=$2 [L,QSA]
  RewriteRule ^(.*)/image/([^/\\]+)$                       zp-core/i.php?a=$1&i=$2 [L,QSA]
  RewriteRule ^(.*)/album/(thumb|[0-9]{1,4})/([^/\\]+)$    zp-core/i.php?a=$1&i=$3&s=$2&album=true [L,QSA]
  ####
   
  # Catch-all - everything else gets handled in PHP for compatibility.
  RewriteRule ^(.*)/?$                        index.php?album=$1 [L,QSA]
</IfModule>


Any ideas?

Thanks

GW
« Last Edit: January 28, 2011, 09:30:44 AM by ~GW~ »
Re: Error on crawler execution
« Reply #1 on: January 28, 2011, 11:10:39 AM »
Hello,

please try to specify server IP address in generator configuration.

~GW~

*
  • *
  • 3
  • If you don't ask.......
Re: Error on crawler execution
« Reply #2 on: January 28, 2011, 04:47:27 PM »
Do you mean in place of the site domain?  Or a different setting to be added to the file?

Ok found it, but no luck, still get the same error.

Thanks

GW
« Last Edit: January 28, 2011, 04:53:34 PM by ~GW~ »
Re: Error on crawler execution
« Reply #3 on: January 28, 2011, 11:11:50 PM »
Hello,

please let me know your generator URL/login in private message to check this.

~GW~

*
  • *
  • 3
  • If you don't ask.......
Re: Error on crawler execution
« Reply #4 on: January 29, 2011, 12:56:07 AM »
PM sent :)