CODEWITHSUNDEEP
stackdrivergoogle-cloud-stackdriver
MHS
MHS

Reputation: 41

Stackdriver Logging as Data Pipeline for Analytics

Is it appropriate to use Stackdriver Logging as a means of recording events for analytics such as ad impressions?

Stackdriver Logging can be used to record events (expressed in JSON) that can then be written to GCS, PubSub, and/or BigQuery. Is it appropriate to use this as a means of recording events such ad ad impressions for use in OLAP processes? e.g. are its reliability and throughput adequate for such use cases?

Upvotes: 0

Views: 67

Answers (1)

Kolban
Kolban

Reputation: 15276

I can't see any reason why Stackdriver Logging wouldn't technically work. It appears that you could use the logging API to write the records to Stackdriver and then have an export to pump them out-bound to BigQuery, GCS or PubSub. Combine that with a Stackdriver logging exclude and the written records need not be written to the actual log and hence apparently not be included in your logging utilization.

A possible down-side would be that as you pass through Stackdriver, you are likely increasing the path length of writes and hence it may be longer before a write actually makes its way to the final destination. However, for historical analytics, this should not pose a problem.

I'd also suggest that you encapsulate your record writes in a function where your initial implementation might be to use Stackdriver but if you change the implementation in the future, you will only have to modify the function body.

Upvotes: 2

Related Questions