GCP loadbalancer doesn't work with IAP(Identity-Aware Proxy) settings

Question

I have my loadbalancer in GCP which has HTTPS frontend with my certificate and SSL policy set on my domain. After enabled IAP(Identity-Aware Proxy) in backend, the OAuth consent screen works well when accessing the frontend URL, but following error displays after I log in using Google account in OAuth consent screen.

There was a problem with your request. Please reference https://cloud.google.com/iap/docs/faq#error_codes. Error code 53

After checking Google's guide in error message, I got following offical solution from https://cloud.google.com/iap/docs/allowed-domains

If you receive Error code 53, ask an IAP administrator to add your host name to the list of allowed domains.

But I have already set both my PC's host name and domain name which used for SSL policy to the list of allowed domains in IAP. I'm not sure what's the root cause of this, based on Google's official information. Anyone can help me on this? Thank you!