Step 1 - About
We recommend using Filebeat to upload logs locally from Debian to Logstash.
Step 2 - Install
Filebeat isn't in the main debian repos, so first we need to add the elastic beats repo
apt-get install apt-transport-https wget -qO - https://artifacts.elastic.co/GPG-KEY-elasticsearch | sudo apt-key add - echo 'deb https://artifacts.elastic.co/packages/oss-6.x/apt stable main' > /etc/apt/sources.list.d/beats.list
Use APT to download and update filebeat:
sudo apt-get update && sudo apt-get install filebeat-oss
Step 3 - Configuration
Locate the filebeat YAML config file:
Open the file and change the following:
- Line 24: Set the log input to enabled:
# Change to true to enable this input configuration. enabled: true
- Line 28: Specify the directory to scan for new logs. For example:
# Paths that should be crawled and fetched. Glob based paths. paths: - /var/log/*
Note: * means any text, so this will pick up any file inside the filebeat_logs folder.
- Line 143: The elasticsearch output will be enabled/configured by default. Disable this by commenting it out:
#output.elasticsearch: # Array of hosts to connect to. #hosts: ["localhost:9200"]
- Line 153: Enable the logstash output and the load balancer:
output.logstash: # The Logstash hosts hosts: ["your-logstash-host:your-port"] loadbalance: true ssl.enabled: true
Save and close the file.
Step 4 - Start Logging
Start filebeat (starting filebeat using this method will display live activity inside the terminal. It will also display any validation errors found in the YAML file):
sudo filebeat -e -c /etc/filebeat/filebeat.yml
Any logs found inside the previously specified directory will be harvested by filebeat (this activity will be displayed in the terminal) and logged to logstash.
You can also start the service without using the
filebeat command but this will not
display the activity in the terminal:
sudo systemctl enable filebeat sudo systemctl start filebeat