-2

So I just opened up my Nginx error log as all my pages are throwing error code 500.

It's enormous, my site is still in development and their are well over 100k errors for resources Nginx was unable to find all with spammy names, it's like someone is using some kind of dictionary to try to connect to masses of random pages on my website.

Is this a DDOs attack or am I missing something and how can I prevent this person from connecting again? They are all coming from an IP in the range of 66.249.73.*

Improve this question

2 Answers 2

Reset to default
1

Isn't that one of the ranges the Googlebot comes from?

Sounds like someone has set up a link farm pointing at your domain for some reason. (Probably spam but the details elude me.)

You should be able to control the rate of scanning and which pages get crawled by using the Google webmasters tools and robots.txt.

My first concern is why are you returning 500 responses when you should be returning 404s?

Why is your still-under-development site available to the public?

Improve this answer
1
  • How can I test the site if it is not available to the public? The pages do throw 404, the pages that actually exist are throwing 500 which was the reason I checked the error logs. It does indeed seem to be a googlebot. The rate isn't more than my server can handle it just seems strange, I don't understand what benefit could be had from it.
    – George Reith
    Commented Jun 5, 2012 at 23:41
0

Knowing what the actual error message is would probably help; also, check the access.log and see which requests match up with the errors - then try visiting those URLs by hand, if you get an error yourself, that suggests that your site has bugs :P

Improve this answer
1
  • You misunderstood the question, I am not having a problem with the error codes, they are just the reason I checked the logs in the first place, I can handle that. I am asking about the massive number of attempted GET requests for tons of random non-existent pages.
    – George Reith
    Commented Jun 6, 2012 at 10:24

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .