Ready to get going? Start your 14 days free trial today

Start free trial

Have an account? Sign in

Send data via Google Cloud Storage to your Logstash instance provided by

Google Cloud Storage

Ship audit logs from Google Cloud Storage to Logstash

Step 1 - Confirm Google Cloud Storage Audit LogsCopy

First, confirm the logs that you want to ship are located in your project by accessing the Google Operations log viewer.

Google recommends configuring cloud audit logs with cloud storage, read more about how to configure Cloud Audit Logs

Step 2 - Export Logs to Google Pub/SubCopy

After having located the logs you wish to export, choose the option to create a sink. You will be prompted with the following menu. Choose the Cloud PubSub Topic.

You will be then be prompted for:

  • Sink name
  • Sink Service (choose Pub/Sub)
  • Sink Destination (choose create new Cloud Pub/Sub topic)

Once you've entered these details press create sink.

Step 3 - Create Google Pub/SubCopy

Locate your newly created Pub/Sub topic.

Click the 3 dots icon to the right and select create subscription.

Enter a suitable subscription ID and make sure delivery type is set to pull.

Scroll to the bottom of the page and press create.

Step 4 - Create Service AccountCopy

Visit service accounts and choose the project you want to add the service account to. This will be used to retrieve logs from the GooglePubSub and send to Logstash.

Select create a new service account.

Give your service account a name and add the relevant Pub/Sub permissions for the service account.

Now we need to create a key for the service account, this will be used by Logstash to access the service account in order to retrieve logs.

Click into the newly created service account.

Choose the option to a add key and then select create a key option, remember to download the service key as json when prompted.

Step 5 - Configuring LogstashCopy

To finishing setting up your Google Cloud Platform pub/sub input will need the following details of your Google Cloud Project.

project_id => The complete ID of your project
topic => The name of the topic you created
subscription => The name of the subscription you created
json_key_file => The json key file for your service account. will verify your input before it is applied. This should be actioned in less than 24 hours, contact our support team by visiting our dedicated Help Centre or via live chat & we'll be happy to assist.

Toggle View

Expand View

Return to Search