CODEWITHSUNDEEP
.htaccessgoogle-crawlersprerendergoogle-search-console
neptune
neptune

Reputation: 1261

Google crawler not seeing prerendered Angular generated page

Trying to get my Angular app generated pages to be crawled perfectly by Google without using HashBangs #!. So I generate pushstate URLs with:

$locationProvider.html5Mode(true);
$locationProvider.hashPrefix('!');

added to app's config, and

<base href="/hockey-att/">
<meta name="fragment" content="!">

to the html header.

And I have this in the .htaccess:

RewriteEngine on

RewriteBase /1/

RewriteCond %{REQUEST_FILENAME} !-f
RewriteCond %{REQUEST_FILENAME} !-d
RewriteRule ^(.*)$ #/$1 [L]

<IfModule mod_headers.c>
    RequestHeader set X-Prerender-Token "prerendertoken"
</IfModule>

<IfModule mod_rewrite.c>
    <IfModule mod_proxy_http.c>
        RewriteEngine On RewriteCond %{HTTP_USER_AGENT} baiduspider|googlebot|googlebot-mobile|bingbot|facebookexternalhit|twitterbot|rogerbot|linkedinbot|embedly|quora\ link\ preview|showyoubot|outbrain|pinterest|slackbot|vkShare|W3C_Validator [NC,OR]
        RewriteCond %{QUERY_STRING} _escaped_fragment_=(\%2F|/)_(._) 

        # Only proxy the request to Prerender if it's a request for HTML
        RewriteRule ^(?!.*?(\.js|\.css|\.xml|\.less|\.png|\.jpg|\.jpeg|\.gif|\.pdf|\.doc|\.txt|\.ico|\.rss|\.zip|\.mp3|\.rar|\.exe|\.wmv|\.doc|\.avi|\.ppt|\.mpg|\.mpeg|\.tif|\.wav|\.mov|\.psd|\.ai|\.xls|\.mp4|\.m4a|\.swf|\.dat|\.dmg|\.iso|\.flv|\.m4v|\.torrent|\.ttf|\.woff))(.*) http://service.prerender.io/http://%{HTTP_HOST}/$2 [P,L]
    </IfModule>
</IfModule>

I have the app in the subfolder 1 and the RequestHeader set X-Prerender-Token section in .htaccess set to prerender.io token.

The app with the new urls is working perfectly, and using prerender.io to render the pages. The pages are prerendered and are accessible without problem via http://service.prerender.io/{{http://myurl.com/page}}

My issue is, that the google crawler not accessing the prerendered pages. Tried it via Search Console's Fetch as Google functionality with and without ?_escaped_fragment_= added to the fetched url. But it still get the raw page with angular directives in it's source.

I'm think I missing something in the .htaccess, but I don't know what.

Using ui-router for angular states.

Upvotes: 0

Views: 382

Answers (1)

Prerender.io
Prerender.io

Reputation: 1604

It looks like you are redirecting all requests to a # URL:

RewriteCond %{REQUEST_FILENAME} !-f
RewriteCond %{REQUEST_FILENAME} !-d
RewriteRule ^(.*)$ #/$1 [L]

Is that correct? If so, that code is preventing the Prerender config from being run on each request.

If you are using html5 push state, can you remove that section since you don't need a redirect to a # URL.

Upvotes: 0

Related Questions