mousapelli: (making me stupid)
[personal profile] mousapelli
Thanks to [livejournal.com profile] iibnf for answering my roommate question, and to [livejournal.com profile] mctabby for pointing her my way. hugs to both of you!

So i got one of those "you have exceeded your bandwidth" emails again. ARGH. Last time this happened, it was a webcrawler problem and i ended up creating the robots.txt thing that sends them packing. They've tried to load my site 250 times this month alone! 250 TIMES!

So I'm looking at my webstats, and there is a suspicious leap in the wallpaper page's stats, like 400 more visits than any other page...further investigation reveals...GOOGLE AGAIN! This time the google image server was having an illicit affair with my wallpaper page, and the only solution is to delete the page, I suppose.

It's not really like I mind, the website is really for my fiction and the email anyway, but CHRIST on a POGO STICK, I'm so frustrated with the search engines and their effing crawlers. This isn't some shitty free website either, I'm paying for my 1048576000 bytes a month. There is NO REASON I should be exceeding that. NONE.

So, there's a lot less images floating around sprunkers.com now. I think this is all Ellen's fault somehow, her and her stupid porno ring...

Date: 2004-02-20 08:28 am (UTC)
From: [identity profile] thistle-chaser.livejournal.com
Wow, I could have written this post! I got that same letter last month, and it's all Google's fault. They crawled my site hundreds of times a week, and the image.google section just murdered me through my screenshots page.

You can edit your robots.txt file to just exclude their image-robot if you like. (If you want it, I can give you the lines from my file.) And if you do edit it to block all/some of Google, you can look in their help section for webmasters -- they have a place you can put an "emergency call" in to be crawled again (so your robots.txt will be read and you'll be removed) within 24 hours.

Good luck!

January 2025

S M T W T F S
   1234
567891011
12131415161718
19202122232425
262728293031 

Style Credit

Expand Cut Tags

No cut tags
Page generated Jan. 21st, 2026 08:39 am
Powered by Dreamwidth Studios