Persistent disk reliability and backups in GCE

Question

(My apologies for this being a slightly off-SO question, but it seems that the GCE questions tend to be slightly loess tightly connected to programming.)

I am designing a data acquisition program to run on GCE. The data is collected onto a persistent disk. As the data is something I cannot afford to lose, I need to know something about the reliability of the persistent disks. I have been able to find three pieces of information:

• Persistent Disks have built-in redundancy to protect your data against equipment failure and to remain available through datacenter maintenance events. Your instances, free of local storage, can be moved by Google Live Migration to newer hardware without your intervention. This allows Google datacenters to be maintained at the highest level; software, hardware, and facilities can be continually updated to ensure excellent performance and reliability for your cloud-based services. [Google]

• Google Compute Engine uses redundant, industry-standard mechanisms to protect persistent disk users from data corruption and from sophisticated attacks against data integrity. [Google]

• Google Persistent Disk will never return erroneous data, instead there will be an IO error. [I cannot find this one right now, but remember reading this from some of Google's docs, so take this with a pinch a salt]

The cloud storage comes with some reliability numbers, but is there some information for persistent disks? Without knowing any reliability estimates it is difficult to choose the backup regime. The opposite ends of the continuum are using hot backups with real time synchronization and taking regular deltas of the data (the data is append-only). In the latter case recovery takes much longer and will most probably involve manual bit-stitching. That can be afforded, if the MTBF is high enough.

I am not worried about brief downtimes, but I am worried about data corruption. The system is running on linux+ext4, so it should be resilient against unplanned downtime.

Answer 1

You can go through the following blog post which gives more info about Persistence disk https://cloud.google.com/developers/articles/compute-engine-disks-price-performance-and-persistence