Google Compute Engine

First, we need to ensure that you have Cloud Compute logs configured in Google Operations log viewer.

Read more about how to configure and export logs using the Google Operations Console

Having located the logs you wish to export, choose the option to CREATE SINK. You will be prompted with the following options. Choose the Cloud PubSub topic option.

You will be then be prompted for:

• Sink name
• Sink Service (choose Pub/Sub)
• Sink Destination (choose create new Cloud Pub/Sub topic)

Once you've entered these details choose create sink. This will ensure that the logs you have specified are directed to the Pub/Sub sink

Locate your newly created Pub/Sub topic.

On your topic, click the ellipsis to the right.

Enter a suitable subscription ID and make sure delivery type is set to pull.

Scroll to the bottom of the page and press create.

Visit service accounts and choose the project you want to add the service account to. This will be used to retrieve logs from the GooglePubSub and send to Logstash.

Select create a new service account.

Give your service account a name and add the relevant Pub/Sub permissions for the service account.

Now we need to create a key for the service account, this will be used by Logstash to access the service account in order to retrieve logs.

Click into the newly created service account.

Choose the option to add a key and then select create a key option, remember to download the service key as json when prompted.

To start pulling Logs from the Google Cloud Platform pub/sub to your Stack a Logit.io engineer will configure the Logstash input for you. You can share the following details with our engineers securely over intercom using the button below.

project_id => The complete ID of your project
topic => The name of the topic you created
subscription => The name of the subscription you created
json_key_file => The json key file for your service account.

request input

Now you should view your data:

Launch Dashboard

If you don't see logs take a look at How to diagnose no data in Stack below for how to diagnose common issues.

Cloud Compute Engine is an Infrastructure-as-a-service (IaaS) offering from Google Cloud Platform (GCP). This IaaS platform enables users to launch virtual machines on demand through Google’s data centers and extensive global network.

Cloud Compute is a comparable platform to Microsoft Azure & Amazon EC2 and is often used as it is relatively inexpensive when compared to the two previously cited service providers. Some of the drawbacks of using GCP can include limited access to virtual machines based upon billing reliability of the organisation’s account as if monthly spends are exceeded instances may be unavailable for use.

Compute Engine creates activity, usage, billing & audit logs that allow you track a variety of events affecting your deployment. For example; activity logs record the occurrence of API calls and system events as log messages. It is recommended that you prioritise activity logs over audit messages to follow through log analysis best practises.

With the amount of log messages and formats being generated by a single Infrastructure service it can be hard to know where to look first when it comes to debugging your applications and services running on virtual machines.

You can use Google Cloud Compute logs & metrics to visualise the performance of your services in Hosted Kibana for intelligent reporting & by using our log analysis platform you can correlate the performance of your Compute Engines with the rest of your applications (even those based outside of Google services).

Our Cloud Compute log file analyser can be used to centralise your data, alert on preconfigured rules & monitor the health of all your virtual machine (VM) deployments.

If you need any assistance with analysing your Google Cloud Compute logs we're here to help. Feel free to reach out by getting in contact with us via live chat & we'll be happy to help you start analysing your Cloud Compute data.