SharePoint Online REST by using access_token still getting 403

Question

following this article http://www.ktskumar.com/2017/01/access-sharepoint-online-using-postman/ I was able to register an app and get a client_id as well as a security token.

Now if I follow the article, I'm able to get an access token by using Postman, SOAP UI as well as by using a REST client in browser. I'm also able to fetch data from SharePoint using this token.

However, I need to do this from a unix based middleware, which is able to do HTTP calls as well. I tried everything but I can't get it work.

Preparation that has been done before:

1. register new app by using https://.sharepoint.com/sites//_layouts/15/appregnew.aspx
2. add app and permission to site collection to grant access by using https://.sharepoint.com/sites//_layouts/15/appinv.aspx

After this, I do some webservice calls like this

1. I try to get an access token by calling https://accounts.accesscontrol.windows.net/<mytenant_id>/tokens/OAuth/2 and got one. I can use this token in every REST client as well as in Postman. So I assume it is a valid one.

2. Now I try to retrieve the Title of web by calling this URL https://<my_tenant>.sharepoint.com/sites/<site_collection>/_api/web?$select=Title

This always returns a 403 but only when using middleware system. If I do the same from any other client, it works.

Could someone please enlight me what is going wrong here?

This is how the request header looks like (I've shorten some things)

cookie'='fpc=...some other stuff; domain=.accounts.accesscontrol.windows.net; path=/; secure; HttpOnly; SameSite=None
x-ms-gateway-slice=prod; path=/; SameSite=None; secure; HttpOnly
stsservicecookie=ests; path=/; SameSite=None; secure; HttpOnly'

  'User-Agent'='Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:47.0) Gecko/20100101 Firefox/47.0'
  'accept'='application/json;odata=verbose'
  'Authorization'='Bearer eyJ0eXAiOiJKV1QiLCJhbG... lot following here, but only value of access_token'

This is what the response looks like:

'RESPONSE_HTTP_HEADER_X-ASPNET-VERSION'='4.0.30319'
  'RESPONSE_HTTP_HEADER_LAST-MODIFIED'='Tue, 23 Jun 2020 08:10:42 GMT'
  'RESPONSE_HTTP_HEADER_X-SHAREPOINTHEALTHSCORE'='1'
  'RESPONSE_HTTP_HEADER_X-FORMS_BASED_AUTH_RETURN_URL'='https://<mytenant>.sharepoint.com/_layouts/15/error.aspx'
  'RESPONSE_HTTP_HEADER_CACHE-CONTROL'='private, max-age=0'
  'RESPONSE_HTTP_DATA'='<?xml version="1.0" encoding="utf-8"?><m:error xmlns:m="http://schemas.microsoft.com/ado/2007/08/dataservices/metadata"><m:code>-2147024891, System.UnauthorizedAccessException</m:code><m:message xml:lang="en-US">Access denied. You do not have permission to perform this action or access this resource.</m:message></m:error>'
  'RESPONSE_HTTP_HEADER_X-POWERED-BY'='ASP.NET'
  'RESPONSE_HTTP_HEADER_DATE'='Tue, 23 Jun 2020 08:10:42 GMT'
  'RESPONSE_HTTP_STATUSLINE'='Forbidden'
  'RESPONSE_HTTP_HEADER_EXPIRES'='Mon, 08 Jun 2020 08:10:42 GMT'
  'RESPONSE_HTTP_HEADER_CONTENT-SECURITY-POLICY'='frame-ancestors 'self' teams.microsoft.com *.teams.microsoft.com *.skype.com *.teams.microsoft.us local.teams.office.com;'
  'RESPONSE_HTTP_HEADER_MICROSOFTSHAREPOINTTEAMSERVICES'='16.0.0.20203'
  'RESPONSE_HTTP_HEADER_X-MSDAVEXT_ERROR'='917656; Access+denied.+Before+opening+files+in+this+location%2c+you+must+first+browse+to+the+web+site+and+select+the+option+to+login+automatically.'
  'RESPONSE_HTTP_HEADER_SPREQUESTGUID'='78265f9f-40b3-b000-f2bb-2df685280534'
  'RESPONSE_HTTP_HEADER_STRICT-TRANSPORT-SECURITY'='max-age=31536000'
  'RESPONSE_HTTP_HEADER_TRANSFER-ENCODING'='chunked'
  'RESPONSE_HTTP_HEADER_MS-CV'='n18meLNAALDyuy32hSgFNA.0'
  'RESPONSE_HTTP_HEADER_CONTENT-TYPE'='application/xml;charset=utf-8'
  'RESPONSE_HTTP_HEADER_P3P'='CP="ALL IND DSP COR ADM CONo CUR CUSo IVAo IVDo PSA PSD TAI TELo OUR SAMo CNT COM INT NAV ONL PHY PRE PUR UNI"'
  'RESPONSE_HTTP_HEADER_X-FRAME-OPTIONS'='SAMEORIGIN'
  'RESPONSE_HTTP_HEADER_X-IDCRL_AUTH_PARAMS_V1'='IDCRL Type="BPOSIDCRL", EndPoint="/sites/<sitecollection>/_vti_bin/idcrl.svc/", RootDomain="sharepoint.com", Policy="MBI"'
  'RESPONSE_HTTP_HEADER_SERVER'='Microsoft-IIS/10.0'
  'RESPONSE_HTTP_HEADER_REQUEST-ID'='78265f9f-40b3-b000-f2bb-2df685280534'
  'RESPONSE_HTTP_HEADER_X-MS-INVOKEAPP'='1; RequireReadOnly'
  'RESPONSE_HTTP_HEADER_X-CONTENT-TYPE-OPTIONS'='nosniff'
  'RESPONSE_HTTP_HEADER_X-FORMS_BASED_AUTH_REQUIRED'='https://<mytenant>.sharepoint.com/_forms/default.aspx?ReturnUrl=/_layouts/15/error.aspx&Source=%2f_vti_bin%2fclient.svc%2fweb%3f%24select%3dTitle'
  'RESPONSE_HTTP_STATUS'='403'
  'RESPONSE_HTTP_HEADER_DATASERVICEVERSION'='3.0'

I also tried it with different HTTP Headers, by using cookies and by skip them. Nothing works from middleware but everything from my PC.

Patrick

Answer 1

I use some kind of middleware called "Lobster data". This is a software product to map data between different kind of systems. It's comparable to Microsoft BizTalk or others.

However this software uses some special prefix for HTTP header which I was not aware of. Thanks to their support team, I was able to overcome this issue.

Commonly, if you set a HTTP header, you simple use the name of the header you want to add like "content-type" or "authorization" and pass a value.

When using Lobster, you need to add "REQUEST_HTTP_HEADER_" as a prefix, so it needs to be "REQUEST_HTTP_HEADER_authorization" instead of just "authorization". Otherwise it will not send the data as a HTTP Header.

This is only true when using Lobster and not in general. I wasn't aware that they use this syntax.

Answer 2

You could try this way to get authentication：

https://www.c-sharpcorner.com/article/access-sharepoint-online-rest-api-via-postman-with-user-context/