Universal RESTful Data API framework

Question

we are implementing an operational data store for some key data sets ("single view of customer", "single view of employee") to provide meaningful integrated data primarily to front office applications (primarily B2E....i.e. both provider and consumer are controlled internally, no external exposure).

All the "single views" will be centered around a slow changing master business entity ("customer", "employee", "asset", "product") which has related child/satellite entities which are more transactional/fast changing in nature (i.e. bookings, orders, payments etc.). The different "single views" will be overlapping and interconnected.

So this ODS would become a data abstraction layer between disparate "systems of record" and vertical systems of engagement, providing a traversable data universe decoupling clients from producers

Of course, an ODS is pointless if there is no way to access the data. Therefore, I am looking for some kind of elegant way to implement a resource based data services layer on top of the ODS with some of the following characteristics

• Stateless
• REST Level 3 (HATEOS) but most likely limited to GET's (i.e. create, update, delete would be executed against the actual "system of record" in order to apply business logic)
• abstraction between enterprise data model (=domain entities are exposed as representations) and physical data model (=actual data is stored in physical tables)...no leakage of physical data model
• support for common query features, such as order by, paging etc.
• easy to use for developers, decoupled from physical data model and backend system changes
• Ideally not constrained to relational db's even though this could be optional for now.

The key standard I came across for that would be OData, but there is a couple of concerns though

• It is very MS centric both from a standard (now OASIS) but more importantly from an implementation perspective (.NET, WCF Data services). We are an enterprise java shop.
• A couple of frontrunners (netflix, ebay) have sunsetted their OData services which is always a concerning thing
• OData has been touted as a magic blackbox which exposes the underlying datamodel more or less automatically. That means that there is no "human translation"/modelling in between, and the service layer therefore doesn't provide the abstraction that is one of the key requirements.

Now, some of these disadvantages might not be that critical as we don't plan to expose the data services layer to the external world, but rather use it within our own environment (i.e. rather small number of consumers which can be controlled).But then the question is how much value OData adds.

I do know that there are no free lunches out there :-)

Are there any other approaches of how to implement a generic data access layer ?

thx a lot, Nick

Answer 1

To answer your concerns about OData:

• The statement of OData being MS centric is not true anymore. And there's a very good news to you especially when you're using Java to write services or clients. The Apache Olingo project is currently maintained and developed under an open source manner and Microsoft is just one of the contributors of it. The project is based on an OData V2 version and will also support OData V4.
• It's true that Netflix and Ebay don't expose their OData services anymore. But according to the monitoring of the OData ecosystem on OData.org, there are new OData services and clients published constantly. (The OData.org has started to accept contribution to the ecosystem)
• I'm not sure if I correctly understood this item. But if I did, the OData vocabularies (a part in the OData standard) may be able to resolve your concern. As with the OData model annotated with terms, you can add more "human specification" to the capability of the service and more control to the data and the model. The result will be, that the client will be able to intelligently "human translate" the data and the model to better interact with the service. What is even better about vocabularies is, although there are canonical namespaces of annotations that are reserved by the protocol, you can also define your own vocabulary and have it accepted by whoever wants to consume your service as the vocabularies of OData is extensible as is defined in the OData protocol.

In addition, although your plan is to only expose a data service for limited access. OData natures such as queryablility, RESTful data API, and new OData V4 compelling features such as delta responses, asynchronous requests, server side aggregation will definitely help you write a more efficient and powerful data publishing and consumption story.