Get a DemoStart Free TrialSign In

Amazon CloudWatch

Ship logs from Amazon AWS to logstash using the logstash Cloudwatch plugin

Send Your DataLogsAWSAmazon CloudWatch Guide

Follow this step by step guide to get 'logs' from your system to Logit.io:

Step 1 - Create IAM Policy

In the top left corner of your aws console you will notice a services drop down arrow. Open it and from that menu choose IAM.

AWS Services

Next, in the left hand menu, select policies. Once you have reached the policies page you want to hit the Create Policy that appears towards the top of the page.

Go to policies page

Creating a policy is essentially defining permissions that you can then assign to an identity (user, group or roles.)

On the create policy screen choose the json tab and enter the following:

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "Stmt1444715676000",
            "Effect": "Allow",
            "Action": [
                "cloudwatch:GetMetricStatistics",
                "cloudwatch:ListMetrics"
            ],
            "Resource": "*"
        },
        {
            "Sid": "Stmt1444716576170",
            "Effect": "Allow",
            "Action": [
                "ec2:DescribeInstances"
            ],
        "Resource": "*"
        }
    ]
}

Create policy editor

At the bottom of the page select review policy and finally on the next page you need to give your policy a name, now hit create policy.

Review new policy

Step 2 - Create IAM User

You are now going to need to create a new user so that you can attach the policy you created in step one, this will outline the responsibilities of this user.

While on the IAM page, in the left hand menu choose users.

Now at the top of the page select Add User.

View users and press Add user button

Enter a username and make sure to check Programmatic Access, continue onto the next page of creation.

Set user details in add user form

In the next section you want to attach an exisiting policy. Highlight attach exisiting policies and search the policy list below for your newly created policy.

Attach policies to the new user

Continue onwards to the next step, you can choose to set any Tags here but they aren't necessary. Continue onto the User Review, check all settings are correct and select create user.

On the next screen you will be given your Access Key ID and Secret Access Key. You will need to make a note of these or alternatively download the .csv file provided.

Success, and view your access key ID

Step 3 - Confirm EC2 Instance (Optional)

If you already have an EC2 Instance running you can skip this step.

The easiest way to for the new user to generate some data is to create an EC2 instance so that it can gather the metrics. Select the services drop down and look for EC2, it should come under the heading Compute. On the left hand menu select instances and choose to create a new instance and follow the steps provided.

Once you've created your instance you can move onto the next step.

Step 4 - Configuring Logstash

To finishing setting up your Amazon Cloudwatch input Logit.io will need the following details.

namespace => "AWS/EC2"
region => The Region of your EC2 Instance.
secret_access_key => The Secret Access Key of the user you created.
access_key_id => The Access Key of the user you created.

Logit.io will verify your input before it is applied this should be actioned in less than 24 hours, contact our support team by visiting our dedicated Help Centre or via live chat & we'll be happy to assist.

Step 5 - Check Logit.io for your logs

Now you should view your data:

View my data

If you don't see logs take a look at How to diagnose no data in Stack below for how to diagnose common issues.

Step 6 - CloudWatch Logging Overview

Amazon CloudWatch tracks metrics published to the platform by various AWS services. If we look at AWS EC2 logs as an example we can note that CloudWatch tracks the following metrics; instance, traffic mirroring, CPU credit & status check metrics.

With CloudWatch you can also send custom AWS metrics that allow you to add data points at any rate and order you specify.

Logit.io offers a centralised logging platform that offers detailed monitoring & analysis for a wide range of AWS logs & metrics data including but not limited to Amazon EBS, EC2, CloudWatch, CloudFront, S3, VPC & Amazon RDS.

Our platform provides a solution for monitoring not only AWS metrics but also centralises any other vital log files including Docker, Kubernetes, Azure Event Hub & Google Cloud Platform. This provides you with a single view of your applications across your infrastructure.

If you require any further assistance with analysing your Amazon CloudWatch metrics we're here to help. Feel free to reach out & get in contact with our team via live chat & we'll be happy to assist.

Return to Search
Sign Up

© 2023 Logit.io Ltd, All rights reserved.