A followup to last year’s post about DDoSes. In that post, I covered what was, at the time, fairly new and novel (to me) attacks on our wiki, The Cutting Room Floor. In the time since, I’ve had to observe and defend against various new and ongoing attacks.
Some background
Our wiki is hosted on an 8GB Linode (with a few extras, like daily backups), running a tight stack of nginx, php, and mariadb. We don’t utilize any kind of edge providers, CDNs, cloud scaling, whatever. What you see is what you get.
We don’t rely on Cloudflare (part of the problem), Anubis (itself developed with AI), or the like. It is, for better and worse, an entirely single-person operation: yours truly.
Part of this is because that’s how it’s been since I began operating the wiki in 2009, first as shared hosting, then as increasingly large VPSes. Part of this is because it means our site is entirely independent: Cloudflare having a sad? Not our problem. AWS US-East-2 down? Our shit works. (Obviously, Linode is our failure point, but that’s a much smaller surface area.) Part of it is because it means we’ll never be scrambling to migrate off of (whatever service is getting retired today) and onto (fresh, new expensive service). Don’t fix what ain’t broke.
Unfortunately, this means we can’t take advantage of things like Cloudflare’s anti-DDoS protections or the like. Hosting the wiki and supporting services is effectively my full time job right now (at about $2.60/hour, before costs).
For now, it’s more than capable of dealing with this (after the mitigations I’ve added).
Junk Traffic
A common trait of most of this traffic is that it uses residential proxies. In short, these are often compromised devices on home networks, that work to funnel malicious traffic from an outside source through the home’s connection. Imagine if you had an app installed on your phone that secretly let anyone make outgoing calls from your number. Same thing.
Unfortunately, this makes them hard to block automatically, and can cause false positives… such as when someone who’s infected actually visits the site legitimately. Most “corporate” crawlers tend to come from registered IP blocks, often belonging to data centers or their own companies, but blocking a whole residential range often excludes a lot of real users.
For individual types of junk traffic…
Generic crawler spam
This is, by far, the most common type I see, and I suspect a major reason several independent wikis struggle to stay online. These crawlers power through everything, and seem intentionally designed to target “expensive” pages. A common feature is that URL parameters get sorted: a URL that is normally “title=A&prev=B&cur=C” will instead be crawled as “cur=C&prev=B&title=A”. These URLs often include malformed HTML entities like “<”/”>”/”&” or URL encoded characters (frequently extra parenthesis).
The main defense against these has just been disallowing guests from visiting certain expensive pages, namely WhatLinksHere, RecentChangesLinked, UserContributions, etc. At times, we’ve had to disable viewing history, too, for similar reasons.
As far as behavior goes, it comes and goes in waves. It is the typical “background radiation” and contributes about 1-4 requests/second, but sometimes will surge to 50 or even more, with huge floods of requests from hundreds to thousands of unique IPs. (Depending on how long the attack goes on, some IPs might get used multiple times.)
The bots frequently present as outdated browsers, using a mishmash of collected user-agents.
Known malicious scrapers are sent a randomly-sized zip bomb (a bunch of repeated nonsense with a few links into itself), intended to gum up their operations and slow them down.
Targeted denial-of-service attacks
These are less frequent, but crop up every now and then. Unfortunately, due to our site’s policies (queer-positive and bigot-free, anti-AI), we’re frequently targeted directly by people with no hobbies.
The exact vector these attacks take tends to vary the most. As far as it appears, someone basically puts a few quarters in the “nuke the website” button, and it putters away for a few hours. The more common ones are mass requests of the same page across thousands and thousands of unique IPs, or a handful of unique networks (e.g. 5-10 IPs) sending a continual flood of trash traffic.
Attacks like this can vary from 50 to over 150+ requests/second, with peaks of up to 400-500. I’m not sure if that limit is theirs or ours, as the traffic is usually so “obvious” that it is trivial to block at the application level without impacting anything else.
Fake search engine referrers
One botnet I’ve seen will rotate between a handful of fake “.net” versions of common search engines (bing, google, yandex, etc…) as a referrer. It’s mostly notable because it comes in waves and then vanishes again after a while.
Blank “Mozilla/5.0” agent
This botnet’s main trait is that, before every “real” request, a different IP will request “/robots.txt” with a blank “Mozilla/5.0” user agent. Real requests that come after that always use a “modern” user agent.
For a while, this was a very persistent DDoS vector, and maintained a hundred requests/minute rate consistently for days. It wasn’t sophisticated, but it used tens of thousands of unique IP addresses across pretty much every country.
Fake Googlebot plague
Fake “Googlebot” crawlers have always been a thing, and it’s a common trick to try to bypass paywalls. However, in the last month or so, the problem has gotten much worse. These waves of scrapers will use entire /24 subnets, sequentially by number, requesting page after page after page. If you give them a 403 with the Googlebot UA, they’ll immediately retry with a “real” user-agent.
This bot does not use residential proxies; all of its IPs observed thus far have been from various datacenters or “business” networks. My solution to these has been simple: if you say you’re a Googlebot, you’d better hope that Google agrees.
Unfortunately, I’ve seen a few people definitely false-positive (well, “false,” they very much are a fake bot…) this because of clueless “get around this captcha” extensions. To which I say, well, maybe don’t hand your fake Elvis Presley ID to the bartender next time.
VPNs
I would love to allow VPNs again, and I’m still working on a better solution to allow them. The main problem is that VPNs are also frequently (ab)used, and we’ve had a few… really persistent problems with them. One of the things that led to e.g. ProtonVPN getting blocked finally was someone had set up something to spam the front page through it, and at that point, with multiple IP endpoints, it’s effectively a DDoS all on its own.
People like that are why we can’t have nice things.
LLM scrapers
I’m not sure if it’s just that we’ve blocked most of the major cloud providers that run them (Alibaba, Tencent, AWS, Azure, Google, etc, etc, etc…), or that most of them are nice enough to put it in their user-agent (the major providers), but these are all just simply filtered at the user-agent / network level. If you report as one of the major AI scrapers, you’re fed a junk page.
I add new LLM tools and crawlers to the user agent ban list when I notice them in the logs. This relies on the bots sending a real user agent, but ones that don’t are malicious bots anyway.
Claude (and other AI agents)
I’ll admit, in the grand scheme of things, Claude traffic does not meaningfully impact our server. Their “training” scraper comes from the same subnet every time, presents as Claude, and otherwise “follows the rules,” so it’s given the junk page.
The agent, on the other hand, does not follow those rules. Multiple times, I have observed behavior similar to this, sometimes trying more steps:
- The agent requests a page and is given the anti-LLM page instead
- The agent tries again, often trying to use the API to get the raw text
- The agent, still foiled, will try different user agents, masquerading as both fake browsers (see the “research” one above), or hollow copies of real ones (no real headers, just the UA).
- Occasionally hallucinating other pages or endpoints, e.g. “/w/”-type URLs that are used on other Mediawiki sites, but not ours
In many cases, these are not even cases where the operator has explicitly requested our site! The agent has simply gone searching for whatever the operator has asked for, slurping up data as it does. The person behind it will likely never actually see the page that the agent digested, if they even know what pages it accessed.
Claude actively attempts to evade blocks. This is new behavior to me. In the past, I have seen ChatGPT suggest finding an archived copy of a website when faced with a block, but hadn’t seen evidence of it evading one like this.
This recently came to a head: Claude Code is, like all LLM user agents, served our anti-AI page. One operator of it received a “prompt injection” warning. Nothing happened, but the tool threw up a scary warning that it had blocked a malicious prompt. Our anti-AI page — at the time, containing plain-English instructions on zeroing data (no actual code) — was called “malware.”
As it turns out, the bot had never even been explicitly instructed to visit our site. Its operator wanted “research” on a game, so the bot dutifully set out, accessing websites intended for humans. It stumbled across ours, and was configured as such that it nearly interpreted the robot equivalent of “delete your account” as literal instructions. All of this, apparently, on “auto mode.” Imagine if it ran into an actual prompt injection and not a shitpost!
On “Malware”
I want to stress that the anti-AI page did not actually contain any code. It was not tested, or obfuscated, or hidden in any way. You can view the version at the time right here, in your browser. See if you can spot the typo (and it isn’t the date). The intended outcome was achieved: the operator was annoyed.
Regardless, it was replaced with a more whimsical (but hopefully just as disruptive “prompt injection”) new version. I’ll let you know if I ever get a brazillion dollars.
That said, two days ago, a blog post revealed that TIME is serving AI bots with a different version of its website. Normal visitors get a webpage for humans. AI crawlers, for human operators e.g. Claude, instead get a Markdown version of the page, full of unrelated marketing copy. Is that malware?
I could easily detect the Claude user-agent, and wrap the wiki in a function that replaces every number on the page with a random number instead. That’ll break styles and assets, but the bot won’t care (it’s only interested in the text, remember!), and now it’s being fed deliberate misinformation that’s nigh-impossible to track down. Is that malware?
Is software that automatically attempts to bypass blocks specifically designed to keep it out, malware?
Final thoughts
It’s rough. If you aren’t willing to give in and use tools like Anubis, or divert all your traffic through Cloudflare, it’s extremely cheap to harass a website. There’s a higher level of background noise constantly now, beyond the old-style exploit scanners.
There’s a lot of time I would like to spend on maintenance, on research, on documentation, but instead I have been having to spend it on this. Which, it’s part of the job, but lately it is so much of the job.
I worry about the future of the independent web. We’re already starting to see it affect places like Sudomemo, which recently posted about their own challenges with AI bots. Much of their post echoes what we’ve done and experienced: monitoring new traffic patterns, deploying new defenses in response, crawlers increasingly pretending to be “real.” It’s a constant game of walls-and-ladders, all while trying to keep legitimate users out of it. It is, frankly, exhausting.
We do not keep stats on traffic, beyond a basic count of “blocks” and “not blocked” requests. Even by this metric, many days, more than half of the requests to our site are blocked. On days where we’re facing targeted harassment, that can jump much higher, with over a million requests/day being blocked.
Our server is currently more than capable of handling it, but the constant game of cat-and-mouse requires constant effort and vigilance. It wasn’t always like this. 🙁
Self-promotion
If you like what I write, or you like our work hosting The Cutting Room Floor, consider following us on Bluesky, supporting us via Patreon, or donating through Ko-fi.
Comments are also appreciated, but any attempt to debate the merits/uses/whatever of AI will be moderated.