How can I get access to multiple user's OneDrive files through a custom daemon and the graph REST API?

Question

We are working on a daemon service that will periodically automatically connect to the Microsoft Graph API to list any files in all user's drives with sensitive content. We have setup a custom app in our Azure/Office365 tenant account that has many privileges enabled (all Graph and Sharepoint privs (plus some others), for the sake of testing).

Using the Graph Explorer tool and my personal login account, I am able to list files in my own drive account using both the /me/drive/root/children endpoint and the /users('<user-id>')/drive/root/children endpoint (when the user-id is my own). When I try to connect using curl and a grant_type of client_credentials, using the client_id and client_secret from our custom app in Azure, /users('<user-id>')/drive returns the correct drive id, but /users('<user-id>')/drive/root/children just returns an empty list of children.

Is there some permission that I am missing that we need to set somewhere?

Is this a limitation of the current state of the Graph API?

Is this a limitation of the client_credentials grant type?

Answer 1

This is possible today (with application permissions) by using the new Microsoft App Dev Portal and by following the instructions here. Or if you created (registered) your app within the Azure Portal you have to use a X509 certificate instead of a shared secret (client secret). The most helpful resources, at least for me, to get that working are:

• https://msdn.microsoft.com/en-us/office/office365/howto/building-service-apps-in-office-365#acquiring-an-access-token-with-the-client-credential-flow
• https://blogs.msdn.microsoft.com/richard_dizeregas_blog/2015/05/03/performing-app-only-operations-on-sharepoint-online-through-azure-ad/

Here some python code (for the second case) that generates a url for the user to visit, so she can authorize your app, and for requesting the access token:

import calendar
from cryptography import x509
from cryptography.hazmat.backends import default_backend
from cryptography.hazmat.primitives import serialization
from datetime import datetime, timedelta
import jwt
from jwt.exceptions import InvalidTokenError
from oauthlib.common import generate_nonce, generate_token
from oauthlib.oauth2 import BackendApplicationClient
import requests
from requests_oauthlib import OAuth2Session
import uuid

def to_unix(obj):
    if isinstance(obj, datetime):
        if obj.utcoffset() is not None:
            obj = obj - obj.utcoffset()
    millis = calendar.timegm(obj.timetuple()) + obj.microsecond / 1e6
    return millis

def validate_id_token(token):
    '''Validates the given id token.

    Args:
        token (str): An encoded ID token.
    Returns:
        The decoded token which is a dict.
    '''
    # Extract kid from token header
    try:
        header = jwt.get_unverified_header(token)
    except InvalidTokenError as e:
        raise Exception('No valid id token provided.')
        })
    else:
        kid = header.get('kid', '')

    if not kid:
        raise Exception("Unable to find 'kid' claim in token header.")

    # Fetch public key info
    url = 'https://login.microsoftonline.com/common/discovery/keys'
    try:
        response = requests.get(url)
    except RequestException as e:
        raise Exception('Failed to get public key info: %s' % e)
    else:
        if not response.ok:
            raise Exception('Failed to get public key info: %s' %
                              response.content)
        else:
            public_keys = response.json().get('keys', [])

    # Find public key, used to sign id token
    public_key = None
    for k in public_keys:
        if kid == k['kid']:
            public_key = k['x5c'][0]
            break
    if not public_key:
        raise Exception("Unable to find public key for given kid '%s'" % kid)

    # Verify id token signature
    # NOTE: The x5c value is actually a X509 certificate. The public key
    # could also be generated from the n (modulos) and e (exponent) values.
    # But that's more involved.
    cert_string = ('-----BEGIN CERTIFICATE-----\n' +
                   public_key +
                   '\n-----END CERTIFICATE-----').encode('UTF-8')
    try:
        cert = x509.load_pem_x509_certificate(
            cert_string, default_backend())
    except ValueError as e:
        raise Exception('Failed to load certificate for token signature'
                          'verification: %s' % e)
    else:
        public_key = cert.public_key()

    try:
        decoded = jwt.decode(token, public_key, audience=self.key)
    except InvalidTokenError as e:
        raise Exception('Failed to decode token: %s' % e)
    else:
        return decoded

def generate_client_assertion(tenant_id, fp_hash, private_key, private_key_passphrase):
    """Generate a client assertion (jwt token).

    This token is required to fetch an oauth app-only access token.

    Args:
        fp_hash (str): Base64 encoded SHA1 has of certificate fingerprint
        private_key (str): Private key used to sign the jwt token
        tenant_id (str): The tenant to which this token is bound.
    Returns:
        On success a tuple of the client assertion and the token type
        indicator.
    """
    valid_from = str(int(ts.to_unix(datetime.utcnow() - timedelta(0, 1))))
    expires_at = str(int(ts.to_unix(datetime.utcnow() + timedelta(7))))
    jwt_payload = {
        'aud': ('https://login.microsoftonline.com/%s/'
                'oauth2/token' % tenant_id),
        'iss': client_id,
        'sub': client_id,
        'jti': str(uuid.uuid1()),
        'nbf': valid_from,
        'exp': expires_at,
    }
    headers = {
        'x5t': fp_hash
    }

    if not private_key_passphrase:
        secret = private_key
    else:
        try:
            secret = serialization.load_pem_private_key(
                str(private_key), password=str(private_key_passphrase),
                backend=default_backend())
        except Exception as e:
            raise Exception('Failed to load private key: %s' % e)

    try:
        client_assertion = jwt.encode(jwt_payload, secret,
                                      algorithm='RS256', headers=headers)
    except ValueError as e:
        raise Exception('Failed to encode jwt_payload: %s' % e)

    client_assertion_type = ('urn:ietf:params:oauth:client-assertion-type:'
                             'jwt-bearer')

    return client_assertion, client_assertion_type

def generate_auth_url(client_id, redirect_uri):
    nonce = generate_nonce()
    state = generate_token()
    query_params = {
        'client_id': client_id,
        'nonce': nonce,
        'prompt': 'admin_consent',
        'redirect_uri': redirect_uri,
        'response_mode': 'fragment',
        'response_type': 'id_token',
        'scope': 'openid',
        'state': state
    }
    tenant = 'common'
    auth_url = ('https://login.microsoftonline.com/%s'
                '/oauth2/authorize?%s') % (tenant, urllib.urlencode(query_params))

    return nonce, auth_url

def get_access_token(client_id, id_token, nonce=None):
    '''id_token is returned w/ the url after the user authorized the app'''
    decoded_id_token = validate_id_token(id_token)

    # Compare the nonce values, to mitigate token replay attacks
    if not nonce:
        raise Exception("No nonce value provided.")
    elif nonce != decoded_id_token['nonce']:
        raise Exception("Nonce values don't match!")

    # Prepare the JWT token for fetching an access token
    tenant_id = decoded_id_token['tid']
    client_assertion, client_assertion_type = generate_client_assertion(tenant_id)

    # Fetch the access token
    client = BackendApplicationClient(self.key)
    oauth = OAuth2Session(client=client)
    resource = 'https://graph.microsoft.com/'
    url = https://login.microsoftonline.com/common/oauth2/token
    query_params = {
        'client_id': client_id,
        'client_assertion': client_assertion,
        'client_assertion_type': client_assertion_type,
        'resource': resource
    }
    try:
        fetch_token_response = oauth.fetch_token(url, **query_params)
    except Exception as e:
        raise Exception('Failed to obtain access token: %s' % e)
    else:
        return fetch_token_response

Answer 2

This is a limitation of the current state of the Graph API - there doesn't exist an app-only permission scope, to be used with the client credentials flow, which would allow an app to access drive/files of any user. The Files.* scopes can only be used as delegated permissions - see https://graph.microsoft.io/en-us/docs/authorization/permission_scopes.