Send data via Logstash to your Logstash instance provided by Logit.io

Logstash

Ship logs to your hosted Logstash instance at Logit

All Logit ELK Stacks include highly available hosted Logstash instances, removing the need for installing and maintaining your own Logstash server. Logit recommends using Filebeat to ship your logs and metrics to your hosted Logstash instance on Logit, you then benefit from predefined filters that can be customised from your dashboard.

Step 1 - Before you begin

Logit recommends using Filebeat to ship logs and metrics to your hosted Logstash instance on Logit, you then benefit from high availability and predefined filters that can be customised via your dashboard.

We understand that some customers might have a specific requirement to use Logstash locally to ship logs and so provide steps below to configure this integration.

Step 2 - Logstash to Logstash

One option for how to send your data from your local Logstash to your Logit ELK stack is to send it via your hosted Logstash instance. To do this you can configure the output on your local Logstash to utilise the tcp-ssl port of your hosted Logstash. Configure the local Logstash output to ship your data to the hosted Logstash as shown below, the data you're sending will need to be valid json content.

output {
  tcp {
    codec => json_lines
    host => "your-logstash-host"
    port => your-port
    ssl_enable => true
    }
  }

Step 3 - Sending directly to Elasticsearch

Another option for how to send data from your local Logstash instance is to send it directly to Elasticsearch. In order to do this you will need your Stack in Basic Authentication mode. To enable this choose Stack Settings > Elasticsearch and switch authentication mode to basic authentication. Once you have done this edit the output on your local Logstash to look like the below.

output {
  elasticsearch {
    hosts => ["your-elasticsearch-endpoint-address:443"]
    user => "your-elasticsearch-username"
    password => "your-elasticsearh-password"
    manage_template => false
    index => "%{[@metadata][index]}-%{+YYYY.MM.dd}"
    }
  }

While not required it may be worthwhile adding the following filter before the output. This will add metadeta to your logs so it gives the index name the following format logstash-YYYY.MM.DD

filter {
  if ! [@metadata][beat] {
    mutate { add_field => { "[@metadata][index]" => "logstash" } }
    } 
  else {
    mutate { add_field => { "[@metadata][index]" => "%{[@metadata][beat]}" }}
    }
  }
expand view

Expand View

compact view

Compact View

Return to Search
Sign Up