Ready to get going? Start your 14 days free trial today

Start free trial

Have an account? Sign in

Send data via Amazon CloudWatch to your Logstash instance provided by Logit.io

Amazon CloudWatch

Ship logs from Amazon AWS to logstash using the logstash Cloudwatch plugin

Step 1 - Create IAM PolicyCopy

In the top left corner of your aws console you will notice a services drop down arrow. Open it and from that menu choose IAM.

Now in the left hand menu you want to select policies. Once you have reached the policies page you want to hit the Create Policy that appears towards the top of the page.

Creating a policy is essentially defining permissions that you can then assign to an identity (user, group or roles.)

On the create policy screen choose the json tab and enter the following:

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "Stmt1444715676000",
            "Effect": "Allow",
            "Action": [
                "cloudwatch:GetMetricStatistics",
                "cloudwatch:ListMetrics"
            ],
            "Resource": "*"
        },
        {
            "Sid": "Stmt1444716576170",
            "Effect": "Allow",
            "Action": [
                "ec2:DescribeInstances"
            ],
        "Resource": "*"
        }
    ]
}

At the bottom of the page select review policy and finally on the next page you need to give your policy a name, now hit create policy.

Step 2 - Create IAM UserCopy

You are now going to need to create a new user so that you can attach the policy you created in step one, this will outline the responsibilities of this user.

While on the IAM page, in the left hand menu choose users.

Now at the top of the page select Add User.

Enter a username and make sure to check Programmatic Access, continue onto the next page of creation.

In the next section you want to attach an exisiting policy. Highlight attach exisiting policies and search the policy list below for your newly created policy.

Continue onwards to the next step, you can choose to set any Tags here but they aren't necessary. Continue onto the User Review, check all settings are correct and select create user.

On the next screen you will be given your Access Key ID and Secret Access Key. You will need to make a note of these or alternatively download the .csv file provided.

Step 3 - Confirm EC2 Instance (Optional)Copy

If you already have an EC2 Instance running you can skip this step.

The easiest way to for the new user to generate some data is to create an EC2 instance so that it can gather the metrics. Select the services drop down and look for EC2, it should come under the heading Compute. On the left hand menu select instances and choose to create a new instance and follow the steps provided.

Once you've created your instance you can move onto the next step.

Step 4 - Configuring LogstashCopy

To finishing setting up your Amazon Cloudwatch input Logit.io will need the following details.

namespace => "AWS/EC2"
region => The Region of your EC2 Instance.
secret_access_key => The Secret Access Key of the user you created.
access_key_id => The Access Key of the user you created.

Logit.io will verify your input before it is applied this should be actioned in less than 24 hours, contact our support team by visiting our dedicated Help Centre or via live chat & we'll be happy to assist.

Step 5 - CloudWatch Logging OverviewCopy

Amazon CloudWatch tracks metrics published to the platform by various AWS services. If we look at AWS EC2 logs as an example we can note that CloudWatch tracks the following metrics; instance, traffic mirroring, CPU credit & status check metrics.

With CloudWatch you can also send custom AWS metrics that allow you to add data points at any rate and order you specify.

Logit offers an auto scaling logging platform that offers detailed monitoring & analysis for a wide range of AWS logs & metrics data including but not limited to Amazon EBS, EC2, CloudWatch, CloudFront, S3, VPC & Amazon RDS.

Our platform provides a solution for monitoring not only AWS metrics but also centralises any other vital log files including Docker, Kubernetes, Azure Event Hub & Google Cloud Platform. This provides you with a single view of your applications across your infrastructure.

If you require any further assistance with analysing your Amazon CloudWatch metrics we're here to help. Feel free to reach out & get in contact with our team via live chat & we'll be happy to assist.

Toggle View

Expand View

Return to Search