Get a DemoStart Free TrialSign In

Google Cloud Operations

Ship logs from Google Operations Platform to logstash

Send Your DataLogsGoogle CloudGoogle Cloud Operations Guide

Follow this step by step guide to get 'logs' from your system to Logit.io:

Step 1 - Export Logs to Google Pub/Sub

Ensure that you have Cloud Operations logs configured in GCP.

Now we can filter the logs as needed, by clicking on the service name and choosing Show matching entries as shown below.

Filter the logs that we want to export

Read more about how to configure and export logs using the Google Operations Console

Step 2 - Export Logs to Google Pub/Sub

To export the logs choose Create sink as shown below.

Select sink service

You will be then be prompted for:

  • Sink details (enter a name)
  • Sink destination (choose Cloud Pub/Sub topic creating a new topic if needed)
  • Choose logs to include in sink (this will have a filter applied already if you filtered the logs above)

Choose Create sink to continue.

Fill in edit sink form

Step 3 - Create Google Pub/Sub

Locate your newly created Pub/Sub topic and choose Create subscription.

Find topic in Topics menu

Enter a suitable Subscription ID and leave the delivery type as pull.

We don't need to change anything further at this stage, scroll to the bottom and choose CREATE to create the Subscription.

Step 4 - Create Service Account

Now in service accounts, choose the project you're working with.

Then select CREATE SERVICE ACCOUNT, this account will be used to retrieve logs from the Pub/Sub and send them to Logit.io.

Service accounts screenshot

Name the service account and in step 2 add the role Pub/Sub Subscriber as shown below, then choose DONE.

View new service account

Now choose Manage keys as shown below.

Add key using dropdown

Then choose ADD KEY > Create new key, choosing the recommended JSON format. This will download the JSON key to your machine ready to use in the next step.

Add key using dropdown

Step 5 - Configuring Logstash

To start pulling logs and metrics from the Google Cloud Platform to your Stack you need to configure a Google Cloud Platform Input on your Logit.io Stack.

Go to Dashboard

Logit.io will verify your input before it is applied, we will contact you to confirm when this has been completed.

Step 6 - Check Logit.io for your logs

Data should now have been sent to your Stack.

View my data

If you don't see logs take a look at How to diagnose no data in Stack below for how to diagnose common issues.

Step 7 - Google Operations Logging Overview

Sending and analyzing logs and metrics efficiently is crucial for maintaining the health and performance of your applications and services on the Google Cloud Platform (GCP). One powerful option for log management and analysis is integrating your GCP environment with Logit.io.

Logit.io provides a seamless way to collect, centralize, and gain insights from your logs and metrics. To get started with sending data to Logit.io from GCP, you can utilize various methods and tools, including Logstash, Fluentd, or even direct integration with popular GCP services like Cloud Logging. Learn more about setting up this essential connection by exploring our guide on sending GCP logs to Logit.io. However, before you embark on this journey of log management, it's essential to ensure that your GCP infrastructure is optimized for performance and reliability. Google Cloud Load Balancing plays a pivotal role in distributing incoming traffic and ensuring high availability for your applications. Additionally, efficient data storage is key when dealing with logs and metrics. Google Cloud Storage provides a secure and scalable solution for storing your log data for long-term analysis.

Logit.io's GCP logging incorporates integrations that function seamlessly.

Return to Search
Sign Up

© 2024 Logit.io Ltd, All rights reserved.