Fail2ban

Ship your Fail2ban logs using Filebeat to your Logit.io Stack

Follow the steps below to send your observability data to Logit.io

Logs

Configure Fail2ban to ship logs via Filebeat to your Logit.io stacks via Logstash.

Install Integration

Please click on the Install Integration button to configure your stack for this source.

Install Filebeat

To get started you will need to install filebeat. To do this you have two main options:

Choose the AMD / Intel file (x86_64) or
Choose the ARM file (arm64)

You can tell if you have a Linux PC with an AMD / Intel CPU (kernel) architecture by opening a terminal and running the uname -m command. If it displays x86_64 you have AMD / Intel architecture.

To successfully install filebeat you will need to have root access.

If you have an x86_64 system download and extract the contents of the file using the following commands:

curl -L -O https://artifacts.elastic.co/downloads/beats/filebeat/filebeat-8.15.2-linux-x86_64.tar.gz
tar xzvf filebeat-8.15.2-linux-x86_64.tar.gz

If you have an arm64 system download and extract the contents of the file using the following commands:

curl -L -O https://artifacts.elastic.co/downloads/beats/filebeat/filebeat-8.15.2-linux-arm64.tar.gz
tar xzvf filebeat-8.15.2-linux-arm64.tar.gz

Configure Filebeat.yml

The configuration file below is pre-configured to send data to your Logit.io Stack.

Copy the configuration file below and overwrite the contents of the Filebeat configuration file typically located at /etc/filebeat/filebeat.yml

filebeat.yml

####################### Logit.io Filebeat Configuration #########################
# ============================== Filebeat inputs ===============================
filebeat.inputs:
- type: filestream
  enabled: true
  paths:
  - "/var/log/fail2ban.log"
  
  fields:
    type: "fail2ban"
  fields_under_root: true
  encoding: utf-8
  ignore_older: 12h
 
# ================================== Outputs ===================================
output.logstash:
  hosts: ["@logstash.host:@logstash.sslPort"]
  loadbalance: true
  ssl.enabled: true

Validate Configuration

sudo ./@beatname test config -c @beatname.yml --strict.perms=false

You'll be running as root, so you need to change ownership of the configuration file and any configurations enabled in the modules.d directory, or run with --strict.perms=false as shown above. Read more about how to change ownership.

If the yml file is invalid, @beatname will print a description of the error. For example, if the output.logstash section was missing, @beatname would print no outputs are defined, please define one under the output section

Start Filebeat

To start Filebeat, run in Powershell:

Start-Service filebeat

Launch OpenSearch Dashboards to View Your Data

Launch OpenSearch Dashboards

How to diagnose no data in Stack

If you don't see data appearing in your stack after following this integration, take a look at the troubleshooting guide for steps to diagnose and resolve the problem or contact our support team and we'll be happy to assist.

Metrics

Configure Telegraf to ship Fail2ban metrics to your Logit.io stacks via Logstash.

Install Integration

Please click on the Install Integration button to configure your stack for this source.

Install Telegraf

This integration allows you to configure a Telegraf agent to send your metrics to Logit.io.

Choose the installation method for your operating system:

When you paste the command below into Powershell it will download the Telegraf zip file. Once that is complete, press Enter again and the zip file will be extracted into C:\Program Files\InfluxData\telegraf\telegraf-1.34.1.

wget https://dl.influxdata.com/telegraf/releases/telegraf-1.34.1_windows_amd64.zip -UseBasicParsing -OutFile telegraf-1.34.1_windows_amd64.zip 
Expand-Archive .\telegraf-1.34.1_windows_amd64.zip -DestinationPath 'C:\Program Files\InfluxData\telegraf'

or in Powershell 7 use:

# Download the Telegraf ZIP file
Invoke-WebRequest -Uri "https://dl.influxdata.com/telegraf/releases/telegraf-1.34.1_windows_amd64.zip" `
                -OutFile "telegraf-1.34.1_windows_amd64.zip" `
                -UseBasicParsing
 
# Extract the contents of the ZIP file
Expand-Archive -Path ".\telegraf-1.34.1_windows_amd64.zip" `
            -DestinationPath "C:\Program Files\InfluxData\telegraf"

Configure Telegraf

The configuration file below is pre-configured to scrape the system metrics from your hosts, add the following code to the configuration file telegraf.conf from the previous step.

### Read metrics from fail2ban.
[[inputs.fail2ban]]
  ## Use sudo to run fail2ban-client
  use_sudo = false
 
### System metrics
[[inputs.disk]]
[[inputs.net]]
[[inputs.mem]]
[[inputs.system]]
[[inputs.cpu]]
  percpu = false
  totalcpu = true
  collect_cpu_time = true
  report_active = true
 
### Output
[[outputs.http]]
  url = "https://@metricsUsername:@metricsPassword@@metrics_id-vm.logit.io:@vmAgentPort/api/v1/write"
  data_format = "prometheusremotewrite"
 
  [outputs.http.headers]
    Content-Type = "application/x-protobuf"
    Content-Encoding = "snappy"

Read more about how to configure data scraping and configuration options for Fail2ban (opens in a new tab)

Start Telegraf

From the location where Telegraf was installed (C:\Program Files\InfluxData\telegraf\telegraf-1.34.1) run the program providing the chosen configuration file as a parameter:

.\telegraf.exe --config telegraf.conf

Once Telegraf is running you should see output similar to the following, which confirms the inputs, output and basic configuration the application has been started with:

Powershell Telegraf information

Launch Grafana to View Your Data

Launch Grafana

How to diagnose no data in Stack

Fail2ban Overview

Fail2ban is an open-source security monitoring solution that provides real-time threat detection, integrity monitoring, and compliance management capabilities. To effectively monitor and analyze security events, it's crucial to have a reliable and efficient log management solution. Fail2ban leverages the Elastic Stack, including Elasticsearch, Logstash, and OpenSearch, to provide a centralized platform for collecting, processing, and visualizing security-related data.

Organizations can use Filebeat, an open-source log shipper, to send data from Fail2ban to Logstash for processing and analysis. By configuring Filebeat to monitor the Fail2ban logs directory, any new logs can be automatically sent to Logstash, where they can be processed and analyzed in real-time for threat detection and response.

Filebeat ensures that logs are securely transferred to Logstash, where they can be filtered, transformed, and enhanced using various plugins and modules. Logstash can also perform correlation and analysis on the data to identify patterns and anomalies that may indicate security threats.

Using Fail2ban with Filebeat and Logstash provides organizations with a powerful security monitoring solution that can detect and respond to security incidents in real-time, while also maintaining compliance with industry regulations and standards. By leveraging Logit.io, organizations can collect and analyze data from various sources, gain valuable insights into their security posture, and make informed decisions to improve their overall security stance.

If you need any further assistance with migrating your log data to OpenSearch we're here to help you get started. Feel free to get in contact with our support team by sending us a message via live chat and we'll be happy to assist.

etcd Falco