CODEWITHSUNDEEP
logginggoogle-cloud-platformlogstash
Ajay Gupta
Ajay Gupta

Reputation: 157

How to send GCS audit logs to logstash?

I am very new to GCS. kindly help me to pull complete audit logs and send it to logstash so that i can parse it and send it to elasticsearch.

Upvotes: 1

Views: 793

Answers (1)

Serhii
Serhii

Reputation: 4461

Have a look at the documentation Cloud Audit Logs section Exporting audit logs:

To export audit log entries outside of Logging, create a logs sink. Give the sink a query that specifies the audit log types you want to export; for query examples, go to Security logging queries.

If you want to export audit log entries for a Google Cloud organization, folder, or billing account, review Aggregated sinks.

then move to the section Overview of logs exports:

All logs, including audit logs, platform logs, and user logs, are sent to the Cloud Logging API where they pass through the Logs Router. The Logs Router checks each log entry against existing rules to determine which log entries to ingest (store), which log entries to include in exports, and which log entries to discard. For more details, see Logs Router overview.

Exporting involves writing a query that selects the log entries you want to export, and choosing a destination in Cloud Storage, BigQuery, or Pub/Sub. The query and destination are held in an object called a sink. Sinks can be created in Google Cloud projects, organizations, folders, and billing accounts.

and

Every time a log entry arrives in a project, folder, billing account, or organization resource, Logging compares the log entry to the sinks in that resource. Each sink whose query matches the log entry writes a copy of the log entry to the sink's export destination.

after that, follow instructions at the Scenarios for exporting Cloud Logging: Elasticsearch:

This scenario shows how to export selected logs from Logging to an Elasticsearch cluster.

(Elasticsearch cluster could be installed from Marketplace)

as an alternative you can follow community tutorial Exporting Stackdriver logs to Elastic Cloud:

This tutorial explains how to export Stackdriver logs to the Elastic Cloud Elasticsearch SaaS platform to perform log analytics. Elastic Cloud is a SaaS offering, which saves time by not needing to build and manage the Elasticsearch infrastructure.

Upvotes: 1

Related Questions