CODEWITHSUNDEEP
google-cloud-platformgoogle-bigquerydouble-click-advertising
Blessy
Blessy

Reputation: 520

Move BigQuery Data Transfer Service(DCM) data to another project

I have BigQuery Data Transfer Service for Campaign Manager setup in dataset A in GCP project A. I would like to move this to dataset B located in project B. How can I move the existing data and setup the BigQuery Transfer with any loss of data and duplicates?

Upvotes: 0

Views: 522

Answers (2)

Blessy
Blessy

Reputation: 520

Here's how I migrated the transfer service:

  1. The transfer service was enabled in the project B.
  2. Once the data started to arrive at dataset B, the historical data(from the starting till MIN(partition_date)-1) was copied from dataset A to appropriate partitions in dataset B.
  3. The transfer service in project A was stopped after verifying the partition counts and row counts.

Upvotes: 0

marcyb5st
marcyb5st

Reputation: 252

I'm afraid you would have to:

  1. Copy the relevant tables from dataset A to dataset B
  2. Set up the transfer service again for dataset B (assuming it can be done if the tables already exist in the target dataset)
  3. De-dup the data yourself.

A workaround, that achieve something similar but not exactly what you asked, is to create views of relevant tables in dataset A into dataset B.

By doing so, these views will behave like proper tables in dataset B and you don't have to worry about de-deplication, data loss, and setting up again the data transfer. The downside is that you will have to keep dataset A around.

Upvotes: 2

Related Questions