Google Cloud Storage Usage Pricing from byte_hours

Question

I recently setup the Google Cloud Storage Access Logs & Storage Data and the logs are getting logged but I could see 4 logs at the same time.

For example:

• usage_2017_02_14_07_00_00_00b564_v0
• usage_2017_02_14_07_00_00_01b564_v0
• usage_2017_02_14_07_00_00_02b564_v0
• usage_2017_02_14_07_00_00_03b564_v0

So there are 4 usage logs logged for every hour, what's the different between them.

I connected all the logs to big query to query the table - and all 4 of them have different values.

Also analysing on storage logs - I could see storage_byte_hours to 43423002260.

How to calculate the cost from storage_byte_hours?

Answer 1

It is normal for GCS to sometimes produce more than one logfiles for the same hour. From Downloading logs (emphasis mine):

Note:

• Any log processing of usage logs should take into account the possibility that they may be delivered later than 15 minutes after the end of an hour.
• Usually, hourly usage log object(s) contain records for all usage that occurred during that hour. Occasionally, an hourly usage log object contains records for an earlier hour, but never for a later hour.
• Cloud Storage may write multiple log objects for the same hour.
• Occasionally, a single record may appear twice in the usage logs. While we make our best effort to remove duplicate records, your log processing should be able to remove them if it is critical to your log analysis. You can use the s_request_id field to detect duplicates.

You calculate the bucket size from storage_byte_hours. From Access and storage log format:

Storage data fields:

Field                 Type        Description
storage_byte_hours    integer     Average size in byte-hours over a 24 hour period of the bucket. 

To get the total size of the bucket, divide byte-hours by 24.

In your case 43423002260 byte-hours / 24 hours = 1809291760 bytes

You can use the bucket size to estimate the cost for the storage itself:

1809291760 bytes = 1809291760 / 2^^30 GB ~= 1.685 GB

Assuming Multi Regional Storage (per GB per Month ) $0.026 your storage cost be:

1.685 GB x $0.026 = $0.04381 / month ~= $0.00146033333333 / day (w/ 30 days month)

But a pile of other data (network, ops, etc) is needed to compute additional related costs, see Google Cloud Storage Pricing.