CODEWITHSUNDEEP
node.jsazureazure-cosmosdb
Lech Migdal
Lech Migdal

Reputation: 3978

CosmosDB - query for consumed RU/s programmatically?

I am trying to get information about consumption of RU/second on the collection level, unfortunately when I query for collection information with node.js documentdb module, with code like 

var client = new DocumentDBClient(host, { masterKey: masterKey });
client.readCollection(collection._self, function(err, item) {
   if (err) {
      console.log(err);
    } else {
      console.log(item);
    }
});

I get only basic information like below

{ id: 'myCollection',
  indexingPolicy:
   { indexingMode: 'consistent',
     automatic: true,
     includedPaths: [ [Object] ],
     excludedPaths: [] },
  _rid: 'ku8SANRCfwA=',
  _ts: 1505295711,
  _self: 'dbs/ku8SAA==/colls/ku8SANRCfwA=/',
  _etag: '"00008b00-0000-0000-0000-59b8fd5f0000"',
  _docs: 'docs/',
  _sprocs: 'sprocs/',
  _triggers: 'triggers/',
  _udfs: 'udfs/',
  _conflicts: 'conflicts/' }

Is there any other way to get RU consumption information per collection? This data is available in the portal in Metrics -> Throughput blade, but how to get it through API is a mystery to me. Azure Monitor provides only a metric averaged on the whole database level, so using Azure Monitor API also isn't a way to go.

Upvotes: 3

Views: 1434

Answers (2)

Domo
Domo

Reputation: 46

You can query the consumed RU/s using the @azure/arm-monitor and @azure/ms-rest-nodeauth packages. You'll need to provide the caller (function app running in Azure if using MSI auth or a service principal) with 'API Management Service Reader' and 'Monitoring Reader' privileges on the resource group in the Azure portal. The code looks something like this in TypeScript:

import * as moment from 'moment'
import {
  loginWithAppServiceMSI
} from '@azure/ms-rest-nodeauth'
import {
  MonitorManagementClient, MonitorManagementModels
} from '@azure/arm-monitor'
import { MetricsListResponse } from '@azure/arm-monitor/esm/models'

// alternatively, you can use a service principal creds, @azure/ms-rest-nodeauth -> loginWithServicePrincipalSecretWithAuthResponse()
const credentials = await loginWithAppServiceMSI({
  msiEndpoint: process.env.MSI_ENDPOINT,
  msiSecret: process.env.MSI_SECRET,
  resource: 'https://management.azure.com/'
})
const now = moment()
const end = now.toISOString()
const start = now.subtract(1, 'minute').toISOString()
const client = new MonitorManagementClient(credentials, SUBSCRIPTION_ID)

const options: MonitorManagementModels.MetricsListOptionalParams = {
  metricnames: 'TotalRequestUnits',
  metricnamespace: 'Microsoft.DocumentDB/databaseAccounts',
  filter: `CollectionName eq '${CONTAINER_NAME}'`,
  interval: 'PT1M',
  timespan: `${start}/${end}`
}
const url = `subscriptions/${SUBSCRIPTION_ID}/resourceGroups/${RESOURCE_GROUP_NAME}/providers/Microsoft.DocumentDB/databaseAccounts/${DATABASE_NAME}`
const response: MetricsListResponse = (await client.metrics.list(
  url,
  options
))

Upvotes: 0

Jesse Carter
Jesse Carter

Reputation: 21147

This may not be feasible in your scenario, but you could definitely roll your own solution with the use of some other Azure offerings. I agree that a native API for pulling RU consumption would be much more ideal.

If you wanted to roll your own you could do so through the use of Event Hubs and Stream Analytics. Write a small wrapper around your Cosmos queries that publishes their RU cost to your event hub (RU cost is returned from Cosmos with every operation you perform). From there, Stream Analytics has built in integration for ingesting messages from your Hub. It would be a matter of creating a simple query that aggregates RU cost in one second buckets. From there you could do all kinds of cool stuff including sending alerts at given thresholds or even triggering automatic scale operations in response to your realtime usage to optimize your Cosmos costs.

Depending on how heavily you leverage Cosmos and if you observe spikey workloads the cost savings of doing intelligent scaling would more than offset the additional spend for Event Hubs + Stream Analytics.

Upvotes: 2

Related Questions