Start your 14-day free trial today
No Credit Card Required
Try Logit.io FreeAlready have an account? Sign In
Kubernetes Metrics
Collect and ship Kubernetes container service metrics to Logstash and Elasticsearch.
Metricbeat is a lightweight shipper that helps you monitor Kubernetes by collecting metrics from the containers running on the host system. Configure Metricbeat using the pre-defined examples below to collect and ship Kubernetes container service metrics and statistics to Logstash or Elasticsearch.
Step 2 - Insert stack details
Open the manifest and look for the environment variables controlling the logging destination. (Lines 132 and 261) We need to change environment variables on both lines 138 and 267 because Kubernetes runs a metricbeat instance on each node in the cluster. And also an additional single instance of Metricbeat to retrieve metrics for the cluster itself.
Step 3 - Configure module
Each module has its own configuration file where different metricsets can be enabled / disabled. Locate the metricbeatbeat-kubernetes.yaml configuration file you downloaded in step 1.
These are the default metricsets available for the kubernetes module. To enable or disable a metric simply comment or uncomment the line out.
- module: kubernetes
metricsets:
- node
- system
- pod
- container
- volume
period: 10s
host: ${NODE_NAME}
hosts: ["localhost:10255"]
- module: kubernetes
enabled: true
metricsets:
- apiserver
hosts: ["https://${KUBERNETES_SERVICE_HOST}:${KUBERNETES_SERVICE_PORT}"]
- module: kubernetes
enabled: true
metricsets:
- proxy
hosts: ["localhost:10249"]
period: 10s
- module: kubernetes
enabled: true
metricsets:
- controllermanager
hosts: ["http://localhost:10252"]
period: 10s
- module: kubernetes
enabled: true
metricsets:
- scheduler
hosts: ["localhost:10251"]
period: 10s
There is also a system module enabled for the Kubernetes node, these are the default metricsets enabled. To enable or disable a metric simply comment or uncomment the line out.
- module: system
period: 10s
metricsets:
- cpu
- load
- memory
- network
- process
- process_summary
#- core
#- diskio
#- socket
processes: ['.*']
process.include_top_n:
by_cpu: 5 # include top 5 processes by CPU
by_memory: 5 # include top 5 processes by memory
The Kubernetes cluster also has some default metricsets available. Metricbeat requires kube-state-metrics to be deployed and running to gather the cluster metrics. Follow the Kubernetes deployment docs to get this set-up.
- module: kubernetes
metricsets:
- state_node
- state_deployment
- state_replicaset
- state_pod
- state_container
# Uncomment this to get k8s events:
#- event
period: 10s
host: ${NODE_NAME}
hosts: ["kube-state-metrics:8080"]
Step 4 - Deploy pod
Now your deployment manifest is updated, you can deploy it using.
kubectl apply -f metricbeat-kubernetes.yaml
Step 5 - Confirm deployment completed
Confirm that your pod has successfully been deployed using:
kubectl --namespace=kube-system get ds/metricbeat
kubectl --namespace=kube-system get pods
You should see a pod for each kubernetes node with a name similar to metricbeat-abcde listed. The pods should work though from Pending to Running within a couple of minutes as the containers are downloaded and started.
Step 6 - how to diagnose no data in Stack
If you don't see data appearing in your Stack after following the steps, visit the Help Centre guide for steps to diagnose no data appearing in your Stack or Chat to support now.