Upsert one or more documents, but only update if the existing document meets a condition

Question

After reading a bunch of C++ driver documentation and examples (e. g. 1, 2) I can't piece together a way of achieving my goal with the C++ driver.

I have a collection of documents with the following structure:

{
   _id : int64_t // Supplied by me manually
   url : string
   status : int
   date : int
}

I want to insert a new document. However, if the document with the same _id already exists (which means its url is the same because my _id is a hash of url), I want to update it as follows. Let existing_doc be the document with the same _id already in the db, and new_doc be the one I'm submitting to MongoDB:

1. Update the date field of the existing_doc only if existing_doc[status] was x (some integer constant).
2. Update the status field of the existing_doc only if new_doc[status] was y (some other constant).

Bonus points if it can be made a bulk operation (a bunch of different new_docs), but any tips on how to achieve this logic will be appreciated.

Answer 1

I couldn't find an easy way using MongoDB's field update operators or in other answers.

But one possibility is to perform the bulk_write with two operations:

1. One upsert operation to update existing_doc[date] when existing_doc[status] is x or to insert new documents (condition 1)
2. One update operation to update existing_doc[status] to be executed when new_doc[status] is y (condition 2)

For condition 1 we can perform a single upsert operation querying by id and status=x:

1. If a document is retrieved, the date gets updated ($set operator) since it meant that existing_doc[status] was x
2. If no document is retrieved, an insert is attempted ($setOnInsert operator) and two things can happen:
   • The document gets correctly inserted -> it was a new document
   • A duplicate key error is thrown -> a document with that id already existed but its status was not x, so there's nothing to update

Condition 2 is simpler since the input data already tells us if we need to perform the update or if we can skip it.

The following code is using the bulk_write to perform the above operations:

#include <iostream>

#include <bsoncxx/builder/stream/document.hpp>
#include <bsoncxx/json.hpp>

#include <mongocxx/client.hpp>
#include <mongocxx/exception/bulk_write_exception.hpp>
#include <mongocxx/instance.hpp>
#include <mongocxx/uri.hpp>

using bsoncxx::builder::basic::kvp;
using bsoncxx::builder::basic::make_document;
using bsoncxx::builder::stream::document;
using bsoncxx::builder::stream::finalize;

// Status constants
int kStatus_X = 1234;
int kStatus_Y = 6789;

// Helper method to retrieve the document (in json format) if exists
std::string retrieveJsonDocById(mongocxx::collection& coll, const std::string& id)
{
    bsoncxx::stdx::optional<bsoncxx::document::value> maybe_result =
        coll.find_one(document{} << "_id" << id << finalize);

    if (maybe_result)   { return bsoncxx::to_json(*maybe_result); }
    else                { return "Nothing retrieved for id: " + id; }
}


// Inserts a new document {id,url,status,date}, or updates the existing one
void upsertUrl(mongocxx::collection& coll,
               std::string id, std::string url, int status, int date)
{
    std::cout << ">> Before insert/update: " << retrieveJsonDocById(coll, id) << std::endl;

    // Bulk write ordered=false to force performing all operations
    mongocxx::options::bulk_write bulkWriteOption;
    bulkWriteOption.ordered(false);
    auto bulk = coll.create_bulk_write(bulkWriteOption);

    // If document exists and has status='x', update the date field
    // If document exists but status!='x', nothing will be inserted (duplicate key thrown)
    // If document is new, perform insert
    mongocxx::model::update_one upsert_op{
        make_document(kvp("_id", id), kvp("status", kStatus_X)),
        make_document(
                kvp("$set", make_document(kvp("date", date))),
                kvp("$setOnInsert", make_document(kvp("status", status), kvp("url", url))))
    };
    upsert_op.upsert(true);
    bulk.append(upsert_op);

    // If new_doc[status] is 'y', attempt to perform status update
    if (status == kStatus_Y) {
        mongocxx::model::update_one update_op{
            make_document(kvp("_id", id)),
            make_document(kvp("$set", make_document(kvp("status", status))))
        };
        bulk.append(update_op);
    }

    try {
        auto result = bulk.execute();
    }
    catch (const mongocxx::bulk_write_exception& e) {
        if (e.code().value() == 11000) {
            std::cout << "Duplicate key error expected when id exists but the status!=x: ";
            std::cout << std::endl << e.what() << std::endl;
        }
    }
    std::cout << ">> After insert/update:  " << retrieveJsonDocById(coll, id) << std::endl << std::endl;
}

Which with these test scenarios:

int main(int, char**) {

    std::cout << "Starting program, x=" << kStatus_X << ", y=" << kStatus_Y << std::endl;

    mongocxx::instance instance{};
    mongocxx::client client{ mongocxx::uri{} };

    mongocxx::database db = client["stack"];
    mongocxx::collection coll = db["urls"];

    std::cout << "Inserting Doc #1 (status=x):" << std::endl;
    upsertUrl(coll, "1", "1_url.com", kStatus_X, 101010);
    std::cout << "Inserting Doc #2 (status=x):" << std::endl;
    upsertUrl(coll, "2", "2_url.com", kStatus_X, 202020);
    std::cout << "Inserting Doc #3 (status!=x):" << std::endl;
    upsertUrl(coll, "3", "3_url.com", 3, 303030);
    std::cout << "Inserting Doc #4 (status!=x):" << std::endl;
    upsertUrl(coll, "4", "4_url.com", 4, 404040);

    std::cout << "Inserting again Doc #1 (existing.status=x, new.status=y) -> should update the date and status:" << std::endl;
    upsertUrl(coll, "1", "1_url.com", kStatus_Y, 505050);
    std::cout << "Inserting again Doc #2 (existing.status=x, new.status!=y) -> should update date:" << std::endl;
    upsertUrl(coll, "2", "2_url.com", 6, 606060);
    std::cout << "Inserting again Doc #3 (existing.status!=x, new.status=y) -> should update status:" << std::endl;
    upsertUrl(coll, "3", "3_url.com", kStatus_Y, 707070);
    std::cout << "Inserting again Doc #4 (existing.status!=x, new.status!=y) -> should update nothing:" << std::endl;
    upsertUrl(coll, "4", "4_url.com", 8, 808080);

    std::cout << "End program" << std::endl;
}

generates the following output:

Starting program, x=1234, y=6789
Inserting Doc #1 (status=x):
>> Before insert/update: Nothing retrieved for id: 1
>> After insert/update:  { "_id" : "1", "status" : 1234, "date" : 101010, "url" : "1_url.com" }

Inserting Doc #2 (status=x):
>> Before insert/update: Nothing retrieved for id: 2
>> After insert/update:  { "_id" : "2", "status" : 1234, "date" : 202020, "url" : "2_url.com" }

Inserting Doc #3 (status!=x):
>> Before insert/update: Nothing retrieved for id: 3
>> After insert/update:  { "_id" : "3", "status" : 3, "date" : 303030, "url" : "3_url.com" }

Inserting Doc #4 (status!=x):
>> Before insert/update: Nothing retrieved for id: 4
>> After insert/update:  { "_id" : "4", "status" : 4, "date" : 404040, "url" : "4_url.com" }

Inserting again Doc #1 (existing.status=x, new.status=y) -> should update the date and status:
>> Before insert/update: { "_id" : "1", "status" : 1234, "date" : 101010, "url" : "1_url.com" }
>> After insert/update:  { "_id" : "1", "status" : 6789, "date" : 505050, "url" : "1_url.com" }

Inserting again Doc #2 (existing.status=x, new.status!=y) -> should update date:
>> Before insert/update: { "_id" : "2", "status" : 1234, "date" : 202020, "url" : "2_url.com" }
>> After insert/update:  { "_id" : "2", "status" : 1234, "date" : 606060, "url" : "2_url.com" }

Inserting again Doc #3 (existing.status!=x, new.status=y) -> should update status:
>> Before insert/update: { "_id" : "3", "status" : 3, "date" : 303030, "url" : "3_url.com" }
Duplicate key error expected when id exists but the status!=x:
E11000 duplicate key error collection: stack.urls index: _id_ dup key: { : "3" }: generic server error
>> After insert/update:  { "_id" : "3", "status" : 6789, "date" : 303030, "url" : "3_url.com" }

Inserting again Doc #4 (existing.status!=x, new.status!=y) -> should update nothing:
>> Before insert/update: { "_id" : "4", "status" : 4, "date" : 404040, "url" : "4_url.com" }
Duplicate key error expected when id exists but the status!=x:
E11000 duplicate key error collection: stack.urls index: _id_ dup key: { : "4" }: generic server error
>> After insert/update:  { "_id" : "4", "status" : 4, "date" : 404040, "url" : "4_url.com" }

End program