Automated way to purge SPO User Profiles

Question

As part of a clean up task, I'm looking for a way to programmatically purge deleted AAD accounts from the User Profile Manager in Sharepoint Online.

I was using the Sharepoint Powershell module (Microsoft.Online.SharePoint.PowerShell) to manually do it, using the Remove-SPOUserProfile commandlet, which worked perfectly if I was using it in an interactive session. But as soon as I tried implementing my script into Azure Automation I found that particular module falls back to Basic Authentication when using a PSCredential object in the Connect-SPOService statement. And Basic Auth is blocked at my Organisation (I can't see them allowing it just for me!)

I found the PnP Module (PnP.PowerShell), which does allow authentication via stored credentials. But it doesn't have an equivalent User Profile Remove cmdlet.

Finally, I tried resorting to pure REST API, and while I can get an existing user profile, I can't get a profile for an account that has been deleted (marked as 'Profiles Missing from Import' in the SPO ProfMngr.aspx page). This is because the SP.UserProfiles.PeopleManager/GetPropertiesFor(accountName=@v) API needs to have an exact match (eg i:0#.f|membership|vardhaman@siteurl.onmicrosoft.com), and when an AAD account is deleted the profile username gets DELETED-<GUID> appended to it.

So my questions are:

1. Am I right or wrong about the sharepoint module and stored creds? (IE, so the module can be used from Azure Automation with a service principal or service account)
2. Am I right or wrong about the PnP module and it is missing the similar Remove-SPOUserProfile?
3. With the REST API, how do you search for profiles, especially profiles "missing from import"?
4. Is there some way to predict what the DELETED-<GUID> will be for a given user? Because I was able to get a user profile if I looked up the full deleted name and supply that to my REST call.

The official documentation on this is light - the old traditional sharepoint APIs aren't being developed any more, in favour of MS Graph, but the Graph Documentation doesn't seem to cover my particular use case.

Any pointers appreciated

Update 1

Thanks @Michael Han_MSFT.

I was using a pre-release/nightly build (0.3.32) but looking at Release documentation so didn't realise remove profile was in there.

I'm still getting problems though:

Connect-PnPOnline `
-url "https://<tenantname>.sharepoint.com" `
-ClientId $ClientId `
-ClientSecret $ClientSecret

# $guest1 = Guest account's email address

$azureEmail = ($guest1 -replace "@", "_") + "#ext#@<tenantname>.onmicrosoft.com"

Remove-PnPUserProfile `
-LoginName $azureEmail


Remove-PnPUserProfile : The remote server returned an error: (401) Unauthorized.
At line:11 char:1
+ Remove-PnPUserProfile `
+ ~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : NotSpecified: (:) [Remove-PnPUserProfile], WebException
    + FullyQualifiedErrorId : System.Net.WebException,PnP.PowerShell.Commands.UserProfiles.RemoveUserProf
   ile

So I tweaked the URL:

Connect-PnPOnline `
-url "https://<tenantname>-admin.sharepoint.com" `
-ClientId $ClientId `
-ClientSecret $ClientSecret

$azureEmail = ($guest1 -replace "@", "_") + "#ext#@azurediagovt.onmicrosoft.com"

Remove-PnPUserProfile `
-LoginName $azureEmail

Remove-PnPUserProfile : 
At line:11 char:1
+ Remove-PnPUserProfile `
+ ~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : WriteError: (:) [Remove-PnPUserProfile], HttpRequestException
    + FullyQualifiedErrorId : EXCEPTION,PnP.PowerShell.Commands.UserProfiles.RemoveUserProfile

So you can see if I go to <tenantname> I get a 401, but if I go to <tenantname>-name the response is simply blank.

I was certain I had given my App the right permissions (Is there some way to review what permissions have been assigned?)

In AppInv.aspx I think had this permissions code (I was following a couple of blogs):

<AppPermissionRequests AllowAppOnlyPolicy="true">
 <AppPermissionRequest Scope="http://sharepoint/content/Tenant" Right="FullControl"/>
 <AppPermissionRequest Scope="http://sharepoint/social/Tenant" Right="FullControl"/>
</AppPermissionRequests>

As a further test, I tried the PnP version of what I was doing in REST (Get-PnpUserProfileProperty) and got

Get-PnPUserProfileProperty : Current user is not a tenant administrator.
At line:1 char:1
+ Get-PnPUserProfileProperty -Connection $pnpctx -Account "scottdu@data ...
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : WriteError: (:) [Get-PnPUserProfileProperty], ServerException
    + FullyQualifiedErrorId : EXCEPTION,PnP.PowerShell.Commands.UserProfiles.GetUserProfileProperty

Which is strange, because REST would give me a results.d response.

At this stage, I could look at making the App Id a Sharepoint Service Admin (I already have approval to allow Azure Automation to have whatever rights it needs to solve this).

(Update 1a: Made no difference, unless there is a delay between assigning the role and the permissions taking affect).

Answer 1

AAD registered app can be used to connect PnpOnline and delete user profile. Please see my below steps: (Main refer article: https://learn.microsoft.com/en-us/sharepoint/dev/solution-guidance/security-apponly-azuread)

Step1:Registering an Azure AD application in the Azure Active Directory tenant that is linked to your Office 365 tenant and grant permission

1. Use admin account to access https://portal.azure.com/#blade/Microsoft_AAD_RegisteredApps. Click “New registration” and create a name of your app.

2. Go to “API permissions” and click on the "Add a permission" button and grant SharePoint API permission.

3. Select needed permissions．

4. Admin need to consent for those permissions, after that in status column will show green.

Step2: Create a self signed certificate and connect with app

1. Go to this link（https://learn.microsoft.com/en-us/sharepoint/dev/solution-guidance/security-apponly-azuread#setting-up-an-azure-ad-app-for-app-only-access）, copy the scripts and save as “Create-SelfSignedCertificate.ps1”.

2. Run below command with PowerShell. You will be asked to give a password to encrypt your private key, and both the .PFX file and .CER file will be exported to the current folder.

.\Create-SelfSignedCertificate.ps1 -CommonName "YourCompanyName" -StartDate 2020-1-09 -EndDate 2022-10-01

3. Go to AAD app,　ｃlick on "Certificates & secrets" in the left menu bar. Click on the "Upload certificate" button, select the .CER file you generated and upload it.

Step3: Connect pnp online and perform delete profile

$ Connect-PnPOnline -ClientId <$application client id as copied over from the AAD app registration above> -CertificatePath '<$path to the PFX file generated by the PowerShell script above>' -CertificatePassword (ConvertTo-SecureString -AsPlainText "<$password assigned to the generated certificate pair above>" -Force) -Url https://<$yourtenant>.sharepoint.com -Tenant "<$tenantname>.onmicrosoft.com" ＄Remove-PnPUserProfile -LoginName $UPN

For REST api way, I found this article noted REST for delete user profile is not implemented. (https://learn.microsoft.com/en-us/sharepoint/dev/general-development/work-with-user-profiles-in-sharepoint)

Answer 2

You could try the command : Remove-PnPUserProfile

https://pnp.github.io/powershell/cmdlets/Remove-PnPUserProfile.html

You should install the prerelease version of PnP.PowerShell: https://www.powershellgallery.com/packages/PnP.PowerShell/0.3.8-nightly

Update:

You could try to use tenant administrator account to connect the sharepoint admin site, then run the command Remove-PnPUserProfile. This works for me: