Can't use broker mode with adfs authority

Question

I want to achive cross app sso so that my outlook android app is auto logged in with my adfs account.

I use exchange 2019, ADFS 2019, and windows server 2022.

Here are the error from debug output in vs 2022

ERROR => MSAL.Xamarin.Android.4.66.2.0.MsalServiceException: ErrorCode: unknown_error Microsoft.Identity.Client.MsalServiceException: Broker response returned error: com.microsoft.identity.common.java.authorities.ActiveDirectoryFederationServicesAuthority cannot be cast to com.microsoft.identity.common.java.authorities.AzureActiveDirectoryAuthority at Microsoft.Identity.Client.Internal.Broker.BrokerInteractiveRequestComponent.ValidateResponseFromBroker(MsalTokenResponse msalTokenResponse) at Microsoft.Identity.Client.Internal.Broker.BrokerInteractiveRequestComponent.FetchTokensAsync(CancellationToken cancellationToken) at Microsoft.Identity.Client.Internal.Requests.InteractiveRequest.FetchTokensFromBrokerAsync(String brokerInstallUrl, CancellationToken cancellationToken) at Microsoft.Identity.Client.Internal.Requests.InteractiveRequest.GetTokenResponseAsync(CancellationToken cancellationToken) at Microsoft.Identity.Client.Internal.Requests.InteractiveRequest.ExecuteAsync(CancellationToken cancellationToken) at Microsoft.Identity.Client.Internal.Requests.RequestBase.<>c__DisplayClass11_1.<b__1>d.MoveNext() End of stack trace from previous location --- at Microsoft.Identity.Client.Utils.StopwatchService.MeasureCodeBlockAsync(Func`1 codeBlock) at Microsoft.Identity.Client.Internal.Requests.RequestBase.RunAsync(CancellationToken cancellationToken) at Microsoft.Identity.Client.ApiConfig.Executors.PublicClientExecutor.ExecuteAsync(AcquireTokenCommonParameters commonParameters, AcquireTokenInteractiveParameters interactiveParameters, CancellationToken cancellationToken) at RecipesApp.MainPage.OnLoginClicked(Object sender, EventArgs e) in C:\MyApp\MainPage.xaml.cs:line 28 StatusCode: 0 ResponseBody:
Headers:

Here are my code

try
            {
                var publicClientApplicationBuilder = PublicClientApplicationBuilder
                    .Create("my_client_id_generated_from_adfs")
#if ANDROID || IOS
                    .WithBroker()
#endif
                    .WithAdfsAuthority("https://adfs.mydomain.com/adfs)
                    .WithRedirectUri($"msauth://com.test.app/my_encoded_base_64")
                    .Build();

                var authResult = await publicClientApplicationBuilder.AcquireTokenInteractive({"email"})
                   .WithParentActivityOrWindow(EntraConfig.ParentWindow)
                   .ExecuteAsync().ConfigureAwait(false);

                Debug.WriteLine($"SUCCESS => {authResult.AccessToken}");
                Debug.WriteLine($"SUCCESS => {authResult}");

            }
            catch (Exception ex)
            {
                Debug.WriteLine($"====\nERROR => {ex}\n====");
            }

Already tried without .WithBroker() but there isn't any account in the microsoft authenticator and outlook app. Is the scenario of cross app sso with on prem only adfs account possible?