Already have an account? Sign In
Filebeat is a lightweight shipper that enables you to send your Docker container application logs to Logstash and Elasticsearch. Configure Filebeat using the pre-defined examples below to start sending and analysing your Docker application logs.
curl -L -O https://artifacts.elastic.co/downloads/beats/filebeat/filebeat-oss-7.8.1-amd64.deb sudo dpkg -i filebeat-oss-7.8.1-amd64.deb
curl -L -O https://artifacts.elastic.co/downloads/beats/filebeat/filebeat-oss-7.8.1-x86_64.rpm sudo rpm -vi filebeat-oss-7.8.1-x86_64.rpm
curl -L -O https://artifacts.elastic.co/downloads/beats/filebeat/filebeat-oss-7.8.1-darwin-x86_64.tar.gz tar xzvf filebeat-oss-7.8.1-darwin-x86_64.tar.gz
- Download the filebeat Windows zip file from the official downloads page.
- Extract the contents of the zip file into C:\Program Files.
- Rename the
- Open a PowerShell prompt as an Administrator (right-click the PowerShell icon and select Run As Administrator). If you are running Windows XP, you may need to download and install PowerShell.
- Run the following commands to install filebeat as a Windows service:
cd 'C:\Program Files\filebeat' .\install-service-filebeat.ps1
PowerShell.exe -ExecutionPolicy UnRestricted -File .\install-service-filebeat.ps1.
Change the owner of the filebeat.yml file to root to allow access to the docker container logs.
sudo chown root:root filebeat.yml ls -la
Skip this step if you are using windows.
On Linux we want filebeat to read the container logs from
/var/lib/docker/containers/*/*.log which is where docker's container logs are stored, this is handled by default.
Add to your filebeat.inputs section the docker type.
filebeat.inputs: - type: docker containers.ids: - '*'
On Windows we want filebeat to read the container logs from
filebeat.inputs: - type: docker containers: ids: - '*' path: C:\ProgramData\docker\containers\
If you do not want to read all container logs then you can specify the continer ID to logs from specific containers.
We'll be shipping to Logstash so that we have the option to run filters before the data is indexed.
Comment out the elasticsearch output block.
## Comment out elasticsearch output #output.elasticsearch: # hosts: ["localhost:9200"]
Uncomment and change the logstash output to match below.
output.logstash: hosts: ["your-logstash-host:your-ssl-port"] loadbalance: true ssl.enabled: true
Let's check the configuration file is syntactically correct by running filebeat directly inside the terminal.
If the file is invalid, filebeat will print an
error loading config file error message with details on how to correct the problem.
sudo filebeat -e -c /etc/filebeat/filebeat.yml
cd <EXTRACTED_ARCHIVE> ./filebeat -e -c filebeat.yml
cd <EXTRACTED_ARCHIVE> .\filebeat.exe -e -c filebeat.yml
Ok, time to start ingesting data!
sudo systemctl enable filebeat sudo systemctl start filebeat
Docker is a platform as a service (PaaS) tool created for building & deploying applications by using containers. Developers use these isolated containers to package an application with all of its required dependencies for streamlined deployment. Docker was first created in 2013 and offers both a free open source solution and paid offering. The platform has been instrumental in the development of cloud-native applications.
Thanks to Docker’s widespread adoption, the trend towards using microservices and containerization has become a must for developers launching applications in the cloud.
Despite this, effective log analysis using Docker’s container logs can easily spiral and run into many complications when trying to keep up with the scaling required for your growing infrastructure. Due to their isolated & distributed nature, centralising Docker log messages is often overly problematic when log data is required for further analysis.
The Logit platform provides a single source of truth for container monitoring and log management. We enable your teams to have complete observability across containers, enabling your Developers & IT leaders with the ability to investigate and fix issues faster.
If you need any more help with migrating your Docker log data to Filebeat our engineers are here to help. Feel free to get in contact with our support team by sending us a message via live chat & we'll be happy to assist.