Last week I needed to migrate a specific FTP host to a new VPC as part of some related infrastructure improvements that we are doing. Setting up the new server meant I could party with Ansible again 🙄 I’ll spare the details of all the hoops I needed to jump to get our outdated Ansible roles back working, but I will share how we configured our ProFTPD container and protect it with fail2ban running on the host.
ProFTPD Docker Log Driver
As I mentioned, we’ve set up ProFTPD to run in a Docker container using the following Ansible task, I’ve omitted some of the details for brevity.
- name: proftpd container
become: yes
docker_container:
name: proftpd
image: eu.gcr.io/bnl-blendle/sftp-server
....
log_driver: syslog
log_options:
tag: proftpd
There are two important parts, first the log_driver: syslog param. This will configure the output of the daemonized
ProFTPD process to be logged to the rsyslogd process running on the host. Since we don’t configure the syslog-address param, Docker will automatically pipe it to unixgram:///dev/log, I assume 😅.
The second configuration param which is important is log_options.tag: proftpd, this option will enable us to
filter the logs from this specific container with rsyslog and write it to a specific file
where we can point fail2ban to monitor and ban eventual unwanted visitors.
Configure rsyslog
This is an example log line of what you can find in /var/log/syslog before you configure the rsyslog filter
Oct 17 08:34:32 host proftpd[25415]: 2019-10-17 08:34:32,274 host proftpd[1] host.internal: ProFTPD 1.3.5 (stable) (built Wed May 6 2015 13:31:16 UTC) standalone mode STARTUP
The important log_options.tag: proftpd makes sure the loglines are annotated with the proftpd tag. In the example
line, it’s this proftpd[25415]: string. The next step is to make sure these loglines are saved to a file for fail2ban to monitor. To make that happen, you need to create the following property based filter
1
You need to create a file 50-proftpd.conf in the rsyslog configuration directory /etc/rsyslog.d/ with the following
filter:
:programname, isequal, "proftpd" -/var/log/proftpd.log
& stop
The first param programname is the property we want to filter on, rsyslog will make sure it’s only the tag we
configured. It will strip everything after the first [, see the documentation for more information
Second is the compare operations a simple isequal, we want it to match with our proftpd tag (3rd param), and finally, the fourth param we instruct it to write these logs to /var/log/proftpd.log.
The second line & stop makes sure the logs aren’t processed any further down the line.
Once you configured rsyslog make sure to restart it sudo service rsyslog restart and afterward to check the status
to see it didn’t encounter any issues during the restart (sudo service rsyslog service).
Then tail the logs and try to login to the FTP server, use a wrong password, or a missing user to generate some failed
login attempts. You should be able to see something like the following lines
Oct 17 13:27:26 ... proftpd[25415]: ...: SSH2 session opened.
Oct 17 13:27:29 ... proftpd[25415]: ...: USER felix (Login failed): Incorrect password
Oct 17 13:27:32 ... proftpd[25415]: ...: SSH2 session closed.
Oct 17 19:07:14 ... proftpd[25415]: ...: SSH2 session opened.
Oct 17 19:07:22 ... proftpd[25415]: ...: USER felix (Login failed): Incorrect password
Oct 17 19:07:25 ... proftpd[25415]: ...: SSH2 session closed.
Oct 17 19:07:30 ... proftpd[25415]: ...: SSH2 session opened.
Oct 17 19:07:30 ... proftpd[25415]: ...: USER unknown: no such user found from 10.24.13.24
Oct 17 19:07:32 ... proftpd[25415]: ...: USER unknown: no such user found from 10.24.13.24
Finally configure fail2ban
If you reached this, great success 🎉 You’re almost finished. The final step in protecting our FTP service using fail2ban. You will need to configure a custom fail2ban filter for ProFTPD and enable the jail in the
fail2ban configuration file. Fail2ban comes with default installed ProFTPD filter, but I’ve tried it first on the logs I produced, and it didn’t work. You can try it yourself. Fail2ban comes with fail2ban-regex installed, which helps you
to test your filters. This blog helped me during my search: https://www.the-art-of-web.com/system/fail2ban-filters/
fail2ban-regex /var/log/proftpd.log /etc/fail2ban/filter.d/proftpd.conf --print-all-matched
I didn’t get any matches => Lines: 11 lines, 0 ignored, 0 matched, 11 missed, so we constructed our “custom” filter.
Create a file called /etc/fail2ban/filter.d/custom_proftpd.conf with the following instructions
# fail2ban filter for the ProFTPD FTP daemon
[INCLUDES]
before = common.conf
[Definition]
_daemon = proftpd
failregex = \(\S+\[<HOST>\]\)[: -]+ USER \S+: no such user found from \S+ \[[0-9.]+\] to \S+:\S+\s*$
\(\S+\[<HOST>\]\)[: -]+ USER \S+ \(Login failed\):.*\s+$
\(\S+\[<HOST>\]\)[: -]+ Maximum login attempts \([0-9]+\) exceeded, connection refused.*\s+$
\(\S+\[<HOST>\]\)[: -]+ SECURITY VIOLATION: \S+ login attempted\.\s+$
\(\S+\[<HOST>\]\)[: -]+ Maximum login attempts \(\d+\) exceeded\s+$
ignoreregex =
These regexps work better with the logs we captured when we test it we got Lines: 11 lines, 0 ignored, 4 matched, 7 missed 🥳
Now you’ve prepared the filter it’s time to instruct fail2ban to use it. Edit the fail2ban jail config file
/etc/fail2ban/jail.conf or place a new config file in /etc/fail2ban/jail.d/ with the following configuration
[proftpd]
enabled = true
port = 22
filter = custom_proftpd
logpath = /var/log/proftpd.log
maxretry = 6
Make sure you restart fail2ban (sudo service fail2ban restart) and check the status (sudo service fail2ban status) to make sure it’s running. Next, check if the correct jails are running using sudo fail2ban-client status. If you want
to see more details of a specific jail pass it along as 3rd params sudo fail2ban-client status proftpd, you can see
how many bans are applied for this jail.
So when you try out to log in a couple of times with a wrong password or missing users, you should be able to see the following logs from fail2ban.
fail2ban.jail [24605]: INFO Jail 'proftpd' started
fail2ban.filter [24605]: INFO [proftpd] Found 10.24.13.24 - 2019-10-17 13:31:29
fail2ban.filter [24605]: INFO [proftpd] Found 10.24.13.24 - 2019-10-17 13:31:42
fail2ban.filter [24605]: INFO [proftpd] Found 10.24.13.24 - 2019-10-17 13:31:49
fail2ban.filter [24605]: INFO [proftpd] Found 10.24.13.24 - 2019-10-17 13:32:03
fail2ban.filter [24605]: INFO [proftpd] Found 10.24.13.24 - 2019-10-17 13:32:07
fail2ban.filter [24605]: INFO [proftpd] Found 10.24.13.24 - 2019-10-17 13:32:12
fail2ban.actions [24605]: NOTICE [proftpd] Ban 10.24.13.24
fail2ban.actions [24605]: NOTICE [proftpd] Unban 10.24.13.24
ps: Make sure you have an existing /var/log/proftpd.log file when you start fail2ban otherwise it crashes when is
missing see fail2ban#1593
The setup is done on ubuntu 18.04 Bionic Beaver with Docker 19.03.3, rsyslog 8.32.0, fail2ban 0.10.2 and ProFTPD 1.3.5e
Disclaimer
I know this isn’t the best solution, but we were in a hurry to get things working again. The reason we needed to fix it
in the first place was we found out our original approach wasn’t logging the logs we expected. We’ve used the ExtendLog config option from ProFTPD and mounted a volume from the host to make the logs accessible for fail2ban.
ExtendedLog /var/log/proftpd/auth.log AUTH auth
The logs looked like
Debian [18/Oct/2019:17:49:19 +0000] "USER felix" [10.24.13.24] 331
Debian [18/Oct/2019:17:49:19 +0000] "PASS (hidden)" [10.24.13.24] 530
And were completely missing the information which we rely on in the fail2ban filter to work. The reason why we’ve
switched to use syslog was we did see the stdout log of ProFTPD did contain all the right information we were looking for.
We tried several different options to get ExtendLog work, but without any results, so we looked for an alternative.
Using Syslog log driver was an alternative approach, in the future I hope we can run our FTP server on Kubernetes as
we do for 95% of the rest of our infrastructure.
Hopefully, I can save somebody the trouble in the future
