CODEWITHSUNDEEP
google-cloud-platform
Rostislav V
Rostislav V

Reputation: 1798

How to control task deduplication feature in Google Cloud Tasks

I'm looking at the documentation here:
https://cloud.google.com/tasks/docs/dual-overview
And in quotas here:
https://cloud.google.com/tasks/docs/quotas
I still not sure how I can enable or disable this deduplication feature. I wonder if I can set Task de-duplication window

Upvotes: 8

Views: 5081

Answers (2)

Andres S
Andres S

Reputation: 1247

Task deduplication refers to making sure you don't have duplicate names of tasks.

Per the doc it works like this:

Explicitly specifying a task ID enables task de-duplication. If a task's ID is identical to that of an existing task or a task that was deleted or executed recently then the call will fail with google.rpc.Code.ALREADY_EXISTS. If the task's queue was created using Cloud Tasks, then another task with the same name can't be created for ~4hours (previously 1hr) after the original task was deleted or executed.
If the task's queue was created using queue.yaml or queue.xml, then another task with the same name can't be created for ~9days after the original task was deleted or executed.

Because there is an extra lookup cost to identify duplicate task names, these tasks.create calls have significantly increased latency.

Upvotes: 8

Vitalii Kyrychenko
Vitalii Kyrychenko

Reputation: 316

We had the same problem.

To not spam users, we've been looking for a mechanism that we could use to control the de-duplication feature for certain notification types.

Since they check TASK_ID - we can use it to control de-duplication by adding something unique. For example a timestamp.

TASK_ID is the last part of the task name. Task name looks like this - projects/PROJECT_ID/locations/LOCATION_ID/queues/QUEUE_ID/tasks/TASK_ID.

If we need to create a task that will be executed once per 10 seconds - we can generate a name like this - projects/PROJECT_ID/locations/LOCATION_ID/queues/QUEUE_ID/tasks/notification-like-${timestamp}.

Other tasks created within these 10 seconds will be rejected with an error.

JavaScript snippet that generates a task name 

/**
 * Round time up to given coefficient.
 * @param {number} coefficient - Coefficient in milliseconds.
 */
const roundTime = (coefficient) => {
    const date = new Date();
    return new Date(Math.ceil(date.getTime() / coefficient) * coefficient).getTime();
};

const coefficient = 1000 * 10; // Trigger once per 10 seconds
const taskId = `notification-like-${roundTime(coefficient)}`;
const taskName = `projects/PROJECT_ID/locations/LOCATION_ID/queues/QUEUE_ID/tasks/${taskId}`;

console.log(taskName);
// projects/PROJECT_ID/locations/LOCATION_ID/queues/QUEUE_ID/tasks/notification-like-1643898560000

Upvotes: 6

Related Questions