CODEWITHSUNDEEP
azureazure-cosmosdb
Zubair
Zubair

Reputation: 6233

Azure Cosmos Bulk-Executor Library - {\"Errors\":[\"The input content is invalid because the required properties - 'id; ' - are missing\"]}

I am trying to bulk insert documents/Json using Azure Bulk Executor library. I used the approach as described in Azure Documentation here: https://learn.microsoft.com/en-us/azure/cosmos-db/bulk-executor-java

Below is the DocumentClient creation:

  public static DocumentClient documentClientFrom(BulkExecutionConfiguration cfg) throws DocumentClientException {
    
    ConnectionPolicy policy = new ConnectionPolicy();
    RetryOptions retryOptions = new RetryOptions();
    retryOptions.setMaxRetryAttemptsOnThrottledRequests(0);
    policy.setRetryOptions(retryOptions);
    policy.setConnectionMode(cfg.getConnectionMode());
    policy.setMaxPoolSize(cfg.getMaxConnectionPoolSize());
    
    return new DocumentClient(cfg.getServiceEndpoint(), cfg.getMasterKey(), policy, cfg.getConsistencyLevel());
}

When I try to do a bulk import as below:

BulkImportResponse bulkImportResponse = bulkExecutor.importAll( documents, false, true, null );

I am getting the Error:

com.microsoft.azure.documentdb.DocumentClientException: Message: {"Errors":["Encountered exception 
while executing function. Exception = Error: {\"Errors\":[\"The input content is invalid because the 
required properties - 'id; ' - are missing\"]}\r\nStack trace: Error: {\"Errors\":[\"The input 
content is invalid because the required properties - 'id; ' - are missing\"]}\n   at createCallback 
(script.js:6223:26)\n   at Anonymous function (script.js:686:29)"]}
ActivityId: 3b6ca789-508d-40a2-bda1-b4e424dac88f, Request URI: /apps/6ec7164c-b528-494b-b40e- 
249832f34bb1/services/221058a8-0299-4341-ad5a-0ea3d0a49cc1/partitions/ebae0619-6126-453c-b6a1- 
774739a52a71/replicas/132378420031633670p/, RequestStats: 
RequestStartTime: 2020-07-08T10:55:11.7899079Z, RequestEndTime: 2020-07-08T10:55:11.7999008Z,  Number 
of regions attempted:1
ResponseTime: 2020-07-08T10:55:11.7999008Z, StoreResult: StorePhysicalAddress: rntbd://cdb-ms-prod- 
southcentralus1-fd11.documents.azure.com:14397/apps/6ec7164c-b528-494b-b40e- 
249832f34bb1/services/221058a8-0299-4341-ad5a-0ea3d0a49cc1/partitions/ebae0619-6126-453c-b6a1- 
774739a52a71/replicas/132378420031633670p/, LSN: 105, GlobalCommittedLsn: 105, PartitionKeyRangeId: 
0, IsValid: True, StatusCode: 400, SubStatusCode: 400, RequestCharge: 4.38, ItemLSN: -1, 
SessionToken: -1#105, UsingLocalLSN: False, TransportException: null, ResourceType: StoredProcedure, 
OperationType: ExecuteJavaScript
, SDK: Microsoft.Azure.Documents.Common/2.11.0, StatusCode: BadRequest
at com.microsoft.azure.documentdb.internal.ErrorUtils.maybeThrowException(ErrorUtils.java:74)
at com.microsoft.azure.documentdb.internal.GatewayProxy.performPostRequest(GatewayProxy.java:284)
at com.microsoft.azure.documentdb.internal.GatewayProxy.doExecute(GatewayProxy.java:108)
at com.microsoft.azure.documentdb.internal.GatewayProxy.processMessage(GatewayProxy.java:360)
at com.microsoft.azure.documentdb.DocumentClient$5.apply(DocumentClient.java:3037)
at com.microsoft.azure.documentdb.internal.RetryUtility.executeDocumentClientRequest(RetryUtility.java:73)
at com.microsoft.azure.documentdb.DocumentClient.doCreate(DocumentClient.java:3042)
at com.microsoft.azure.documentdb.DocumentClient.executeStoredProcedure(DocumentClient.java:1571)
at com.microsoft.azure.documentdb.bulkexecutor.internal.BatchInserter$1.call(BatchInserter.java:186)
at com.microsoft.azure.documentdb.bulkexecutor.internal.BatchInserter$1.call(BatchInserter.java:158)
at com.microsoft.azure.documentdb.repackaged.com.google.common.util.concurrent.TrustedListenableFutureTask$TrustedFutureInterruptibleTask.runInterruptibly(TrustedListenableFutureTask.java:125)
at com.microsoft.azure.documentdb.repackaged.com.google.common.util.concurrent.InterruptibleTask.run(InterruptibleTask.java:57)
at com.microsoft.azure.documentdb.repackaged.com.google.common.util.concurrent.TrustedListenableFutureTask.run(TrustedListenableFutureTask.java:78)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)

Here is json for the object I am trying to insert:

.........

"grocinvoiceInd": "N",
"_id": {
    "transactionId": {
        "$numberLong": "<xxxxxx>"
    },
    "txnSeqNbr": 0
},
"sourceInfo": "<xxxxxxx>"
}

Apparently it looks like it has a _id field but its not inserting the record. Any help is highly appreciated.

Upvotes: 0

Views: 1341

Answers (1)

4c74356b41
4c74356b41

Reputation: 72191

it is necessary for the document to have an id field, because you've created a collection with a partition key, if you create a collection with a partition key, all the documents in that collection must have that property

https://learn.microsoft.com/en-us/azure/cosmos-db/partitioning-overview

Also as described in Azure documentation https://learn.microsoft.com/en-us/azure/cosmos-db/bulk-executor-java "Currently, the bulk executor library is supported only by Azure Cosmos DB SQL API and Gremlin API accounts." Considering this we bulk executor may not work with MongoDB storage accounts at this time.

Upvotes: 1

Related Questions