Google Cloud Storage

First, confirm the logs that you want to ship are located in your project by accessing the Google Operations log viewer.

Google recommends configuring cloud audit logs with cloud storage, read more about how to configure Cloud Audit Logs

Having located the logs you wish to export, choose the option to CREATE SINK. You will be prompted with the following options. Choose the Cloud PubSub topic option.

You will be then be prompted for:

• Sink name
• Sink Service (choose Pub/Sub)
• Sink Destination (choose create new Cloud Pub/Sub topic)

Once you've entered these details choose create sink. This will ensure that the logs you have specified are directed to the Pub/Sub sink

Locate your newly created Pub/Sub topic.

On your topic, click the ellipsis to the right.

Enter a suitable subscription ID and make sure delivery type is set to pull.

Scroll to the bottom of the page and press create.

Visit service accounts and choose the project you want to add the service account to. This will be used to retrieve logs from the GooglePubSub and send to Logstash.

Select create a new service account.

Give your service account a name and add the relevant Pub/Sub permissions for the service account.

Now we need to create a key for the service account, this will be used by Logstash to access the service account in order to retrieve logs.

Click into the newly created service account.

Choose the option to add a key and then select create a key option, remember to download the service key as json when prompted.

To start pulling Logs from the Google Cloud Platform pub/sub to your Stack a Logit.io engineer will configure the Logstash input for you. You can share the following details with our engineers securely over intercom using the button below.

project_id => The complete ID of your project
topic => The name of the topic you created
subscription => The name of the subscription you created
json_key_file => The json key file for your service account.

request input

Now you should view your data:

View my data

If you don't see logs take a look at How to diagnose no data in Stack below for how to diagnose common issues.

Google Cloud Storage is an enterprise public cloud storage platform for storing and accessing data on Google Cloud Platform (GCP). It’s RESTful API allows users to seamlessly integrate with a vast array of apps and services.

Google Cloud Storage is well known for its ability to scale in line with GCP’s security and sharing functionality. Many users also find the service useful due to it’s out of the box features allowing fine-grain permissions and object version control.

There are three main log types created by this Google Cloud Storage, these include access, daily storage and cloud audit logs. You may wish to export and analyse these logs outside of the Google Cloud Platform especially in the event that you need to centralise your log data from a number of external sources.

The Logit.io log management platform can be used to gather Google Cloud Storage logs and metrics for visualising and correlating the performance of the application alongside your various Google Cloud services.

Our built in Google Cloud storage log analyser can be used to centralise and manage your data. Set up alerts to monitor your cloud environments with Logit.io & report on key metrics with our Hosted Kibana dashboards.

If you need any further assistance with analysing your Google Cloud Storage logs we're here to help. Feel free to reach out by getting in contact with us via live chat & we'll be happy to help you get started with managing your logs.