CODEWITHSUNDEEP
pythongoogle-app-enginegoogle-cloud-datastore
Nando
Nando

Reputation: 11

Is Google Cloud Datastore a Column Oriented NoSQL database?

From my understanding BigTable is a Column Oriented NoSQL database. Although Google Cloud Datastore is built on top of Google’s BigTable infrastructure I have yet to see documentation that expressively says that Datastore itself is a Column Oriented database. The fact that names reserved by the Python API are enforced in the API, but not in the Datastore itself makes me question the extent Datastore mirrors the internal workings of BigTable. For example, validation features in the ndb.Model class are enforced in the application code but not the datastore. An entity saved using the ndb.Model class can be retrieved someplace else in the app that doesn't use the Model class, modified, properties added, and then saved to datastore without raising an error until loaded into a new instance of the Model class. With that said, is it safe to say Google Cloud Datastore is a Column Oriented NoSQL database? If not, then what is it?

Upvotes: 1

Views: 512

Answers (1)

pgiecek
pgiecek

Reputation: 8200

Strictly speaking, Google Cloud Datastore is distributed multi-dimensional sorted map. As you mentioned it is based on Google BigTable, however, it is only a foundation.

From high level point of view Datastore actually consists of three layers.

BigTable

This is a necessary base for Datastore. Maps row key, column key and timestamp (three-dimensional mapping) to an array of bytes. Data is stored in lexicographic order by row key.

  • High scalability and availability
  • Strong consistency for single row
  • Eventual consistency for multi-row level

Megastore

This layer adds transactions on top of the BigTable.

Datastore

A layer above Megastore. Enables to run queries as index scans on BigTable. Here index is not used for performance improvement but is required for queries to return results.

Furthermore, it optionally adds strong consistency for multi-row level via ancestor queries. Such queries force the respective indexes to update before executing actual scan.

Upvotes: 3

Related Questions