hupden/ projects
blogtools
← blog

fail2ban kept banning my own git pushes

#fail2ban#nginx#git#self-hosting

I self-host a Forgejo instance behind SWAG (nginx + fail2ban in one container). Four private repos, one user, regular pushes. Twice in two days, my own IP stopped being able to reach the server at all — not just git, everything.

The jail that did it was nginx-unauthorized, which ships with SWAG and bans an IP after enough HTTP 401s. That sounds like exactly the jail you want. The problem is what a 401 actually means when git is on the other end of the connection.

Git asks for a 401 on purpose

Git's smart-HTTP transport does not send credentials up front. Every fetch, pull, and push starts with a probe:

GET /user/project-repo.git/info/refs?service=git-upload-pack

No Authorization header. The server answers 401, git reads the WWW-Authenticate challenge, and then it retries the same request with credentials. That handshake is the protocol working correctly. It is not a failed login — nobody typed a wrong password, and the very next request from that same IP succeeds.

To fail2ban, though, a 401 in the access log is a 401. It has no idea the line above it is one half of a two-step handshake. So every single git operation I performed contributed one "auth failure" toward my own ban.

With one repo and occasional pushes, you never notice. With four repos, a CI runner, and a habit of pushing in small commits, you cross maxretry in a couple of minutes.

Why the obvious fixes are wrong

The first instinct is to raise maxretry or shorten findtime until the problem stops. That works right up until it doesn't, and it weakens the jail against the traffic it actually exists to stop.

The second instinct is ignoreip with my own address. That fixes it for me and leaves the jail broken for anyone else who ever pushes to this server — and it means a real credential-stuffing run from my own network gets a free pass.

Neither addresses the actual defect, which is narrow and specific: one exact request path produces a 401 that is not an auth failure.

The fix

fail2ban filters support ignoreregex, evaluated after failregex — a match there means "saw it, don't count it." So the fix is a filter-only override that exempts the probe path and nothing else.

Because the vendor filter may be regenerated when the SWAG image updates, this goes in its own .local file rather than being edited in place:

# /config/fail2ban/filter.d/nginx-unauthorized.local
[Definition]
ignoreregex = ^<HOST> .*"(GET|POST) [^"]*/info/refs\?service=git-(upload|receive)-pack[^"]*" 401

That exempts the unauthenticated probe for git-upload-pack (fetch/pull) and git-receive-pack (push). Every other 401 anywhere on the server still counts normally: a wrong password on an admin login elsewhere, a scanner walking basic-auth endpoints, a brute-force run against any other vhost. The jail keeps doing its job. It just stops counting a protocol handshake as an attack.

One gotcha that cost me a confusing half hour: on a LinuxServer image, config under /config/fail2ban/filter.d/ is copied into the container's real /etc/fail2ban/ at container startup, not continuously. Dropping a new filter file in while the container is running does nothing, and fail2ban-client reload will not pick it up either. You need a full docker restart. Check your work from inside the container:

docker exec swag fail2ban-regex \
  /config/log/nginx/access.log \
  /etc/fail2ban/filter.d/nginx-unauthorized.conf \
  /etc/fail2ban/filter.d/nginx-unauthorized.local

If the ignoreregex is live, the git probe lines show up under "Ignored lines" rather than "Matched".

The general lesson

A 401 is a status code, not a verdict. Plenty of perfectly normal protocols use one as a negotiation step — git is the one that bit me, but WebDAV and some API clients behave the same way. A jail that counts status codes is really counting your assumption about what those codes mean, and that assumption is worth checking against your own logs before it locks you out of your own box.

Which, in the end, is the cheap version of this lesson. The expensive version is the jail that doesn't fire when it should.