Robots

Topic

New Reply

I am trying to get a robot to index my page, but it keeps coming up with a “fobidden” error. See attached image file.

I have deleted the robots.txt file and even remarked out any and all meta tags related to robots. Shouldn’t this open it wide open?

Web URL is: http://www.grace-efc.org.

Thanks
–KZ

Reply | Quote

Replies

Hi there

There is a lot of confusion around about the usage of the robots.txt file I have had this link stored away in an old file which is a great tutorial which may help you

Robots tutorial

Reply | Quote

Thanks. I’ve read through that and even created a brand new robots.txt file and still is shows as forbidden. Hmmm

–KZ

Reply | Quote

Ken

Your robots.txt states:

User-agent: *
Disallow:

How about if you just delete this text and make it an empty file?

Reply | Quote

I’ll try that. It didn’t work when I didn’t even have a robots.txt file though.

Reply | Quote

Same thing with it blank too.

Reply | Quote

Right time for a new analyser. Place this code in yout index page and then try analysing it using this link

Meta tag Analyser Tool

Comes up with some interesting info about your site

Reply | Quote

Interesting information! Still doesn’t allow that index to run though, not sure why.

–KZ

Reply | Quote

I am running out of ideas so will hand over the baton…but I have a feeling that you ISP server administrator does not allow this

Reply | Quote

That’s probably it. I’ll check on that. Thanks!

Reply | Quote

The IP of that company was blocked in my config. Works now. Thanks!

Reply | Quote

I thought I was going mad….glad it is fixed

Reply | Quote

ME TOO.

Check it out:

http://www.grace-efc.org/

Type in Youth or something in the search.

–KZ

Reply | Quote

I typed Jezza Bear and got two hits…I was impressed until I realised it picked up the word bearing

Reply | Quote

That’s funny. You were thinking that YOU would be listed on my site somewhere???

Reply | Quote

Right let’s go back….where are you getting the error message …in your browser?

Reply | Quote

No, I’m trying to use http://www.websidestory.com. When I run their indexing feature it shows forbidden in the main page and in the robots.txt file and stops.

–KZ

Reply | Quote