Image may be NSFW.
Clik here to view.
This is a 28 day graph of my website’s traffic through Feb. 5. Normally my site transfers between 5 and 50MB per day. A distributed bot called “80 legs” (or user-agent: 008) crawled the site on the 5th, sucking up 391MB of my bandwidth to God knows what end. Since this is a distributed bot for-hire, it could conceivably crawl my site like this every damned day or even multiple times per day. No thanks! I immediately blocked it in my robots.txt, we’ll see if it obeys or not.
I found this thread about “80 legs” to be informative.
