Ready to get going? Start your 14 days free trial today

Start free trial

Have an account? Sign in

Send data via Google Kubernetes Engine to your Logstash instance provided by Logit.io

Google Kubernetes Engine

Ship Google Kubernetes Engine Metrics to Logstash

Metricbeat is an open source shipping agent that lets you ship Google Kubernetes Engine (GKE) Metrics to one or more destinations, including Logstash.

Step 1 - Configure GKECopy

Open Google Kubernetes Engine in Google Cloud. And choose to create a cluster, give it a suitable name and choose a region for your cluster.

Step 2 - Connecting to the clusterCopy

Once the cluster is created you will see a similar screen to the below. Go ahead and hit Connect.

You'll have two options, we're going to choose to Run in Cloud Shell.

This will open a terminal window in your browser. (It can take a few seconds to load)

Step 3 - Setting up the terminalCopy

You'll have to run the initial command that comes pre-populated in the terminal to authenticate. Wait a few seconds for this to process and then you're going to download the following Logit Metricbeat Kubernetes deployment manifest.

To download the Logit deployment manifest for GKE, in the GKE terminal run:

curl -L -O cdn.logit.io/metricbeat-kubernetes-7.9.3.yaml

Step 4 - Deploy MetricbeatCopy

Now you have the manifest we need to add your Stack Logstash endpoint details. Open the file in terminal editor using vim or nano command. Alternatively you can choose Open Editor from the console toolbar to make your changes.

vi metricbeat-kubernetes-7.9.3.yaml

or

nano metricbeat-kubernetes-7.9.3.yaml

Update the following lines in the yaml with your Stack Logstash endpoint and Beats-SSL port.

Note: The code snippet occurs two times in the yaml file and needs updating in both.
env:
- name: LOGSTASH_HOST
  value: "guid-ls.logit.io"
- name: BEATS_PORT
  value: "00000"

After updating the code should look as below.

env:
- name: LOGSTASH_HOST
  value: ["your-logstash-host"]
- name: BEATS_PORT
  value: ["your-ssl-port"]

Exit and save the file.

Step 5 - Apply your updatesCopy

Now we're going to apply the file to the cluster.

kubectl apply -f metricbeat-kubernetes-7.9.3.yaml

You should see output similar to that below confirming that the file has been created.

If you need to apply further updates after running the apply command you may need to remove the yaml file, make your changes and then apply again.

Step 6 - Confirm DeploymentCopy

Confirm your pod has deployed, you should see output similar to that below.

kubectl --namespace=kube-system get ds/metricbeat

Browse to Kibana and you should see Metrics arriving in your Stack.

In the GKE console you can view the log file to confirm that Metrics are being sent to your Logit Stack using the following command.

kubectl logs ["podname"] --namespace=kube-system

Metricbeat requires kube-state-metrics to be deployed and running to gather the cluster metrics. Follow the Kuberenetes deployment docs to get this set-up.

Step 7 - GKE Kibana dashboardsCopy

Kibana includes some pre-built dashboards for Google Kubernetes Engine (GKE). To view your dashboards for any of your Logit Stacks, launch Kibana, choose Dashboards and filter for Kubernetes.

Toggle View

Expand View

Return to Search