[UNIX GEEK - no, really, very] - Victimizing brute force attack autoblocks...

[doc_strange]

A server I help maintain gets regular SSH brute force login attacks, sometimes thousands an hour. Today I went around and looked at ways to automatically block these attacks. Many scripts, in shellscript and perl, some semi-built-in routines.

A number of elements seem to me to be involved in this kind of blocking:

A) Kind of failure
 1) valid username, bad login
 2) invalid username
 3) disallowed valid username
B) Number of failures
 1) total
 2) over time
 3) over the course of N log entries
C) Data rotation
 1) nnn.newer nnn.older rotation via cron 
 2) touch filename == IP address (very DJB-like); cron removal for ctime > n minutes ago
D) Special hosts
 1) permanent (static) blocks
 2) ignore IPs and CIDR blocks
E) Blocking via
 1) IP firewalling
 2) hosts.deny
 3) sshd config files

The approaches as I see them boil down to:

A) shell/perl scripts cron-parsing/tailing logs
B) binaries cron-parsing/tailing logs
C) A/B combined with an extant log-watch package
D) PAM modules
E) patches to SSHD

There's lots of A and at least one B out there, with some C around. Only two ways to do D, and none if you're not using PAM or are on a NON-PAM system. E doesn't seem to exist, but would (I think) be coolest; it's more or less what I do with spammer brute force protection in the SMTP daemon. It ensures real-time reaction, too.

What I found, were script after script doing clever parsing of log events, a few count methods, and a couple of different time rotation methods. Some locked down immediately for short periods on a login failure with a non-existent user name; some counted n before locking. You get the idea.

---

BAD SECURITY, BAD!
But the security of these security scripts? All of them parsed for field n in the log file entry, space-delineated. That's fine, except for one thing: usernames can contain spaces, numbers and periods. I can log in with 'ssh hostname -l "n from 127.0.0.1"' -- that means sshd will log "n from 127.0.0.1" as my username. Guess what the log parser will do? Guess what gets blocked? Right. One can thus block ANY that is not in some kind of white list; no TCP spoofing required.

Now, what's interesting is how ACTUALLY SECURITY CONSCIOUS SOFTWARE handles this kind of item. Note, the username is *user input* -- and should be considered hostile. One should never parse on any delineator it COULD supply. In Bernstein's software, the logs are safe to parse on space-delineation. Why? ALL *user* input spaces are changed to underscores when it writes to the logs.

Interestingly, there's a simple workaround none of these scripts took. The last 4 fields of SSHd logs are *always* not user-input data. They are:
[IP address] port [port #] [ssh protocol]

So, instead of grabbing field #13, space delineated, for IP address, one should match on the fourth field from the end. Hard in a shell script? Nope -- awk will do so neatly with: {print $(NF-3)}. Also easy in Perl.

SIGH.
... off to write my own routines, now.

Comments

[marsgov.livejournal.com]

I must be missing the part where you explain why you don't turf out at least some of this security problem to higher level authorities, either in the network or to law enforcement. Are these zombie attacks?

[docstrange.livejournal.com]

When I ID an attacking host, I do send the ISP/admins a note. But I'd like faster reaction on the host's own part to self-defend because their reaction times almost always stink. Most of the attacks are from scripts being run on 0wned hosts around the world. Some real zombies, some just UNIX boxes some schmoe nailed. This morning's run of my script had some nice coincidental test data from .dk and .kr.

Whee. I'm still stunned at the sheer number of "security" people whose scripts can be casually used to DoS the host they're "protecting."

[ivy]

Argh! Bad security bad, indeed! I hadn't looked at this sort of thing before -- my servers don't get hit often enough for it to be a major problem for me, and I don't run SSH on port 22. But that is quite horrible. Are you going to publish your scripts? (And may I link from my geek filter?)

[docstrange.livejournal.com]

Link away. As to publishing scripts: all one needs do is use the awk $(NF-3) instead of $13 and most of the extant stuff works fine. There are a lot of scripts out there... all dependent on cut or awk to grab fields $13 and/or $9 -- Google on SSH brute force gets you both ends of the brute force battle.

[docstrange.livejournal.com]

Some apps/scripts that are out there:

BFD, general brute force detection from log monitoring; sshd example uses $13; readily changeable to $(NF-3)
authfail appears also to be a possibility; again, parses on user text that could be spoofed, but a nifty implementation otherwise.
http://mandrake.vmlinuz.ca/bin/view/Main/SSH - script uses iptables, uses $13
Macintouch article on ssh monitoring, uses sed, but examples parse on $9, $7, and words that could come from user input.

Turns out there is one sshd patch out there for counting bad logins over time and locking out by IP. I haven't read the code or tested it.

Summary of ssh lockdown issues: http://aplawrence.com/Bofcusm/2507.html