crawling doesn't follow Hebrew links
« on: June 17, 2009, 07:13:52 AM »
I have page with partly Hebrew URL,
the generator crawled the page (but write the link in gibberish letters,
but doesn't follow to the other links that goes out from this page.

and it's really important for me! (those pages were the reason I bought the generator!)

other option should be :
those URLs  with the hebrew letters are all virtuals, and are translated by isap_modrewrite,
could be the generator doesn't know to follow virtual URL?

Re: crawling doesn't follow Hebrew links
« Reply #2 on: June 18, 2009, 06:34:22 AM »


[ External links are visible to forum administrators only ]