CODEWITHSUNDEEP
data-warehousedata-vault
Blue Clouds
Blue Clouds

Reputation: 8161

Adding Load date in real time from message queue DV2.0

In real time load we do not use Staging tables. And it is in staging that we usually add LDTS. So when taking data directly from messages to Raw Vault in real time loading, how do we get LDTS?

Upvotes: 0

Views: 43

Answers (1)

Blue Clouds
Blue Clouds

Reputation: 8161

This is taken from the offical Data Vault Book (building scalable..) Page 473:

Note that the loading patterns use the load date from the staging area. However, it is also possible to create a new load date within the loading processes (for example, using GETDATE() or a similar function). The key is to control the load date within the data warehouse and not rely on a third-party system such as an operational source application.

So the answer is, we can add LDTS directly in the Raw Vault in real-time scenario. Which ever is the entrypoint to the Data Vault, there we can add. And as stated above, we even have the freedom to change it as it moves to another storage area if needed.

Hard rules in realtime are executed on the way in to the vault.

Upvotes: 0

Related Questions