CODEWITHSUNDEEP
mongodbazure-cosmosdb
Dr.YSG
Dr.YSG

Reputation: 7591

CosmosDB (mongo/mongoose) Bulk Write Error

I am trying to use insertMany() to put 50 documents in a collection. On the local emulator this works fine. When I try the same code in Azure with a CosmosDB I get the following error:

2019-12-05T00:12:51.320 [Information] Connected to Cosmsos DB
2019-12-05T00:12:51.576 [Error] failed to create users, err: BulkWriteError: Error=16500, RetryAfterMs=72, Details='
2019-12-05T00:12:51.320 [Information] Connected to Cosmsos DB
2019-12-05T00:12

Here is the connection mechanism, schema, and the code that populates the collection initially.

const personSchema = new mongoose.Schema({
    firstName: String,
    lastName: String,
    addressNumber: Number,
    streetName: String,
    city: String,
    email: String
})

async open() {
    let host = this.conn
    this.log.info(`Host: ${JSON.stringify(host)}`)
    const url = `mongodb://${host.host}:${host.port}/${host.db}?ssl=true&replicaSet=globaldb&retryWrites=false`
    const options = {
        auth: {
            user: host.user,
            password: host.password
        }
    }
    try {
        await mongoose.connect(url, options)
        this.log.info('Connected to Cosmsos DB')
        this.db = mongoose
    } catch (err) {
        this.log.error('Unable to connect to DB', err)
    }
    this.users = mongoose.model('person', personSchema)
}

async populate(logger) {
    await this.connect(logger)
    try {
        await this.users.deleteMany({})
    } catch (err) {
        this.log.error(`cannot empty collection: ${err}`)
    }
    const uList = userData.map(u => this.users(u))
    try {
        const result = await this.users.collection.insertMany(uList)
        this.log.info(`users created: ${result}`)
    } catch (err) {
        this.log.error(`failed to create users, err: ${err}`)
    } finally {
        await this.close()
    }
}

Upvotes: 4

Views: 4305

Answers (1)

David Makogon
David Makogon

Reputation: 71130

The error you're getting, 16500, is "Too Many Requests." TL;DR you've hit your RU limit and are being throttled. Each insert is costing some level of RU, and your per-second RU rate isn't set high enough to handle a rapid-insert scenario like what you've set up.

You can work around this by either increasing your RU capacity or reducing the number of items you're trying to insert at the same time.

FYI I just recreated this error condition by calling db.collection.insertMany() from the mongo shell with a large array of documents (in this case, about 150 small documents), combined with a very low RU count (400 in my case).

Upvotes: 6

Related Questions