Google 404 soft error on index page that is working fine

Question

A friend of mine has been having trouble getting her site indexed by google and asked me to have a look, but that is not something I really know much about and was hoping for some assistance.

Looking at her search console, google crawl shows an error of soft-404 on the index page. I marked this as fixed a few times, because the site looks fine to me but it keeps coming back.

If I fetch the site as google it seems to be working fine, although it is showing the mobile version instead of the desktop.

It keeps giving another reoccurring 404 of a page http://www.smeyan.com/new-page, which doesn't exist anywhere I can see including server files or sitemaps.

Here is what I know about this site:

It used to be a wix site and was moved to a host gator shared server 2-3 months ago.

It's using JavaScript/jQuery .load to get page content outside the index.html template.

It has 2 sitemaps one for the URLs and one for both URLs and images http://www.smeyan.com/sitemap_url.xml http://www.smeyan.com/sitemap.xml

It has been about 2 months since it was submitted for indexing and google has not indexed any of the content when you search for site:www.smeyan.com it shows some old stuff from the wix server. Although search console says it has 172 images indexed.

it has www. as a preference set in search console.

Has anyone experienced this and has an direction for a fix?

Answer 1

Google bot currently use web rendering service (WRS) that is based on old Chrome 41 (M41), so it may fail where browsers succeed.

To learn how google boot works read this.

Add this code to the page to see the real error. You can see the error using Url Inspector live, from google search console. It will show at more info tab.

Note: if the bot gets a 301 code or if the page is too little to have significant content it will return a soft 404 error, and won't preview or show any other error.

Answer 2

I checked your site with Tor Browser which has... DISABLED SCRIPTS. You should provide any content on your site with use of <noscript/> tag. It doesn't have to be beautiful but should be visible for bots. <a href... ></a>, <img/> etc. and... TEXT. Without it the site is NOT OPTIMIZED for search bots. Read about SEO. The sitemap content can be never indexed if the content will be never linked.

Probably your webpage also doesn't meet requirements for screen readers (for blind people).

Note: The image with "SMEYAN" caption is visible on webpage and is indexed.

second image on the webpage (in source): <img class="gallery-full-image" src="./galleries/home_gallery/smeyan_home-1.jpg" /> and indexed

The menu also doesn't work without scripts.

I thought the step is good implemented.

Please use <noscript/> element and implement version for blind people (without scripts, provide alt tag for images) and for noscript browsers. You can test it via disabling script or via NOSCRIPT extension for Firefox.

BTW. You should use HTML, CSS (including animations) and... use the JS ONLY if it is needed. Or... <noscript/> method.

Answer 3

The only download error that I saw while using Chrome's Inspect function pertains to a SCRIPT tag with a Facebook url as the source (src) file.

This is the error as reported by Inspect.

This is the SCRIPT tag that caused the error.

I am not sure that this is the cause of the reoccurring 404 error, but it is an issue that needs attention on this website.

Answer 4

How long time was set for this site in Cache-Control header? If long, you should use "google removals" for obsolete snippets and cache. I simulated Google visit on your webpage. Correct 404 return code. Correct headers. Thus. Report google removals for "not found" pages. You must request visit of Googlebot and keep calm and wait for reaction.

BTW: For permanently removed content use 410 Gone for Google or... report via Removals. https://support.google.com/webmasters/answer/1663419?hl=en