CODEWITHSUNDEEP
aws-dms
Leigh Mathieson
Leigh Mathieson

Reputation: 2038

AWS DMS CDC replication task failure error: load_common_wrapper( common_load_locker ) failed (at_common.c:69)

We have an AWS DMS CDC replication task replicating data from an EC2 running MongoDB to Postgres. It has been running fine, DMS replication instance resources (RAM, CPU etc) all seem normal, however the task has started to fail, with:

Last Error The task stopped abnormally Stop Reason RECOVERABLE_ERROR Error Level RECOVERABLE

And error from Cloudwatch logs:

at_common_is_supported: load_common_wrapper( common_load_locker ) failed (at_common.c:69)

It is also possible to improve logging output further in the task settings JSON editor (additionally to the 5 fields that you can set in the console UI) by opening the task JSON editor and amending all the additional fields to: "Severity": "LOGGER_SEVERITY_DETAILED_DEBUG"

With the detailed_debug error logging enabled:

sqlite_execute_statement:361: Failed (at_sqlite.c:367)

sqlStatement=ALTER TABLE cdc_status ADD COLUMN read_volume_rollback INTEGER; (at_sqlite.c:325)

object already exists (at_sqlite.c:331)

Has anyone seen/ resolved this previously? Thanks in advance

Upvotes: 1

Views: 1815

Answers (1)

ikoif
ikoif

Reputation: 66

In order to debug this you can try two things:

  1. If you have access to the mongod logs, you can check those. You will be able to see issues that DMS can not report back to you. The mongod logs are the system logs of the MongoDB process outputs.

  2. If you don't have access to the MongoDB logs, make sure that the user that you are using for the MongoDB source in DMS has the following roles:

First, you need to create a new role in the admin database:

use admin

db.createRole({
  role: "changestreamrole",
  privileges: [
    { resource: { db: "", collection: "" }, actions: ["find", "changeStream"] },
  ],
  roles: [{ role: "read", db: "local" }],
});

Second, you need to make sure that as an end result, the user you use needs to have the following roles:

[
  { role: "changestreamrole", db: "admin" },
  { role: "backup", db: "admin" },
  { role: "readWrite", db: "wur_university_rankings" },
]

readWrite and backup are MongoDB Built-in roles. The reason why we use backup is for the listDatabases privellage.

I hope this helps! Good luck!

Upvotes: 1

Related Questions