Send data via Filebeat to your Logstash instance provided by Logit.io

Filebeat

A log shipper designed for files.

Filebeat is an open source shipping agent that lets you ship logs from local files to one or more destinations, including Logstash.

Step 1 - Install Filebeat

deb (Debian/Ubuntu/Mint)

sudo apt-get install apt-transport-https
wget -qO - https://artifacts.elastic.co/GPG-KEY-elasticsearch | sudo apt-key add -
echo 'deb https://artifacts.elastic.co/packages/oss-6.x/apt stable main' | sudo tee /etc/apt/sources.list.d/beats.list

sudo apt-get update && sudo apt-get install filebeat

rpm (CentOS/RHEL/Fedora)

sudo rpm --import https://artifacts.elastic.co/GPG-KEY-elasticsearch
echo "[elastic-6.x]
name=Elastic repository for 6.x packages
baseurl=https://artifacts.elastic.co/packages/oss-6.x/yum
gpgcheck=1
gpgkey=https://artifacts.elastic.co/GPG-KEY-elasticsearch
enabled=1
autorefresh=1
type=rpm-md" | sudo tee /etc/yum.repos.d/elastic-beats.repo

sudo yum install filebeat

macOS

curl -L -O https://artifacts.elastic.co/downloads/beats/filebeat/filebeat-oss-6.7.1-darwin-x86_64.tar.gz 
tar xzvf filebeat-oss-6.7.1-darwin-x86_64.tar.gz

Windows

  • Download the Filebeat Windows zip file from the official downloads page.
  • Extract the contents of the zip file into C:\Program Files.
  • Rename the filebeat-<version>-windows directory to Filebeat.
  • Open a PowerShell prompt as an Administrator (right-click the PowerShell icon and select Run As Administrator). If you are running Windows XP, you may need to download and install PowerShell.
  • Run the following commands to install Filebeat as a Windows service:
PS > cd 'C:\Program Files\Filebeat'
PS C:\Program Files\Filebeat> .\install-service-filebeat.ps1`
If script execution is disabled on your system, you need to set the execution policy for the current session to allow the script to run. For example: PowerShell.exe -ExecutionPolicy UnRestricted -File .\install-service-filebeat.ps1.
My OS isn't here! Don't see your system? Check out the official downloads page for more options (including 32-bit versions).

Step 2 - Locate the configuration file

deb/rpm /etc/filebeat/filebeat.yml
mac/win <EXTRACTED_ARCHIVE>/filebeat.yml

Step 3 - Configure the inputs

Setup the data you wish to send us, by editing the input path variables.
These fully support wildcards. You can also add a document type.
An example with nginx logs might look like

filebeat.inputs:

- type: log
#change to true to enable this input configuration
  enabled: true
  paths:
    - /var/log/nginx/*.log
  fields:
    type: nginx-access
  fields_under_root: true
  encoding: utf-8
  exclude_files: [".gz"]
  ignore_older: 3h
There's also a full example configuration file called filebeat.reference.yml that shows all the possible options.

Step 4 - Configure the inputs

Filebeat also has modules that can be displayed, enabled or disabled using

deb/rpm

sudo filebeat modules list
sudo filebeat modules enable <module name>
sudo filebeat modules disable <module name>

macOS

cd <EXTRACTED_ARCHIVE>
./filebeat modules list
./filebeat modules enable <module name>
./filebeat modules disable <module name>

Windows

cd <EXTRACTED_ARCHIVE>
filebeat.exe modules list
filebeat.exe modules enable <module name>
filebeat.exe modules disable <module name>

Additionally module configuration can be done using the per module config files located in the modules.d folder, most commonly this would be to read logs from a non-default location

deb/rpm /etc/filebeat/modules.d/
mac/win <EXTRACTED_ARCHIVE>/modules.d/

Step 5 - Configure output

We'll be shipping to Logstash so that we have the option to run filters before the data is indexed.
Comment out the elasticsearch output block.

## Comment out elasticsearch output
#output.elasticsearch:
#  hosts: ["localhost:9200"]

Uncomment and change the logstash output to match below.

output.logstash:
    hosts: ["your-logstash-host:your-port"]
    loadbalance: true
    ssl.enabled: true

Step 6 - Validate configuration

Let's check the configuration file is syntactically correct.

deb/rpm

sudo filebeat -e -c /etc/filebeat/filebeat.yml

macOS

cd <EXTRACTED_ARCHIVE>
./filebeat -e -c filebeat.yml

Windows

cd <EXTRACTED_ARCHIVE>
filebeat.exe -e -c filebeat.yml

Step 7 - Start filebeat

Ok, time to start ingesting data!

deb/rpm

sudo systemctl enable filebeat
sudo systemctl start filebeat

mac

./filebeat

Windows

Start-Service filebeat
expand view

Expand View

compact view

Compact View

Return to Search
Sign Up