Google Ignoring User Selected Canonical URL causing pages to not be served on Google

Question

I am looking for some advice and guidance on an issue I am having with Google Search indexing and Canonical URLs.

Background:

We have a website for Quest Apartment Hotels built on a Sitefinity CMS, featuring numerous content pages. The site includes a Single Page Application (SPA) for property searches. When users search for a property within the SPA, they are directed to a unique property page. These property pages are not easily navigable by Google due to their dynamic nature.

To address this, we've created static content pages with URLs for each property, allowing Google's crawler to access these pages. Each property page has structured data, a user-declared canonical URL pointing to its specific page, a unique meta title, meta description, and extensive property-specific content.

The SPA pre loads content to be served and uses pushState on history changes.

Problem:

Despite the unique content and explicit canonical tags, Google is incorrectly selecting the main search page URL ('/properties') as the canonical URL for individual properties. For instance, the property page at 'https://www.questapartments.com.au/properties/vic/mont-albert/quest-mont-albert' is distinct in content from the '/properties' page, yet Google is not recognizing this uniqueness.

Refer to the below screenshot, where Figure 1 (Marked F1) is a web server that has not existed for a long time: Picture of GSC for a Property Page URL inspection.

You can refer to the below URLs in what I am referencing:

Individual Property Page:

https://www.questapartments.com.au/properties/vic/geelong/quest-geelong-central

High Level Search Page:

https://www.questapartments.com.au/properties

What I've Tried:

Ensuring each property page has a tag pointing to its unique URL.

Verifying that each page has unique meta titles, descriptions, and structured data.

Ensuring the Sitemap points to each property page respectively.

Ensuring a Robots.txt on the site and that each property page is not being blocked.

Question:

Why is Google ignoring my user selected Canonical URL?

What have I potentially missed that may cause this to happen?

Thank you in advance for any help or insights!

Brayden

EDIT - 20/11/2023 - New Screenshot for updated Canonical tag

EDIT - 21/11/2023 - Clarification Points.

1. The Sitemap.gz is something not created or managed by us. This could potentially be a Sitefinity thing... Regardless, we have pointed in GSC to our Sitemap.xml, which is the correct one.

2. The 'prod_web_02.' referral URL is the same site as the 'www.' version. After speaking to my developer, it was meant to be IP restricted, but wasn't... I still don't understand considering its identical why this page is not being indexed.

3. The reason this is placed on Stack Overflow is because the /properties/ pages are all an angular SPA, and I wasn't sure if this was a development oversight in the application, or something in Sitefinity or the frontend potentially.

4. Yes, I have used GSC URL Inspect to test the live pages in which they are showing the exact page I need indexed.

Answer 1

A few things that can be improved:

• http://prod_web_02.questapartments.com.au - you said that server does not exist, but it is still accessible to me, which means google will index it. Either update its robots.txt to disallow indexing OR redirect it to www if feasible.
• https://www.questapartments.com.au/sitemap/sitemap-index.xml - this one points to http://prod_web_01.questapartments.com.au/sitemap/sitemap.gz - in this sitemap all the links are also http and not https. Also, why the url is prod_web_01 and not www? I think you need to configure the LiveUrl of your default Sitefinity site and then regenerate the sitemap. Also, uncheck the Compress option of the sitemap, so you can inspect it easier.

I think once you fix your sitemaps and block access to non-www urls, you should be fine. Note, that it may take some time for google to reindex everything.