CODEWITHSUNDEEP
facebookfacebook-sharer
mTuran
mTuran

Reputation: 1834

Facebook doesn't crawl my site's URLs automatically

I have blog post URLs like these:

http://www.topluisilanlari.com/blog/goster/48/abc

The abc part is just a placeholder, it does not make a difference for the page. When i try to share this URL at Facebook, it does not show an image of the page, although there is an og:image property in the code. But when I enter the same URL in the Facebook debugger and click the "Fetch new scrape information" button, my page image shows properly and at the sharing function too.

How can I fix that?

Upvotes: 0

Views: 2522

Answers (3)

Uriahs Victor
Uriahs Victor

Reputation: 1155

If someone is coming across this looking for details on how to make Facebook crawl a URL programmatically then the answer is on this page: https://developers.facebook.com/docs/sharing/opengraph/using-objects#update

Send a POST request with the URL you want to get scraped/crawled and add on the query parameter ?scrape=true or &scrape=true to that URL(accordingly). Also add the Access Token in the request.

Below is an example of the post request in WordPress but you can manipulate it for Guzzle, Curl etc. The endpoint would be https://graph.facebook.com and the body would be an array containing the URL with the ?scrape=true query param and the access token.

$scrape['id'] = $yoururl . '?scrape=true';
$scrape['access_token'] = $access_token;
                                
$response = wp_remote_post(
                'https://graph.facebook.com',
                array(
                    'body' => $scrape,
                    'headers' => array(
                        'Content-Type' => 'application/x-www-form-urlencoded',
                    ),
                    'timeout' => 60,

                )
            );
print_r($response, true); //would echo out the results of the scrape

Upvotes: 0

Mousey
Mousey

Reputation: 1865

Facebook does not crawl URLs regularly: it is not a search engine. It scrapes a page when it is first shared and saves this information until you re-scrape or re-fetch the page (as you have tried).

More information on open graph, with examples, can be found on http://ogp.me although this open graph generator may help you in future.

When I looked at the scrape information today it contained errors, and meta tags did not end with /> (only with >). It looks like re-scraping the page, which I have just done, has fixed this. The only warning now can be fixed by adding:

<meta property="og:url" content="http://www.topluisilanlari.com/blog/goster/48/abc"/>

Your code has several problems, it has validation errors although these aren't causing the sharing problem.

Upvotes: 0

C3roe
C3roe

Reputation: 96339

https://developers.facebook.com/docs/sharing/best-practices#precaching:

When content is shared for the first time, the Facebook crawler will scrape and cache the metadata from the URL shared. The crawler has to see an image at least once before it can be rendered. This means that the first person who shares a piece of content won't see a rendered image

There are two ways to avoid this and have images render on the first Like or Share action:

1. Pre-cache the image with the URL Debugger
Run the URL through the URL debugger to pre-fetch metadata for the page. You should also do this if you update the image for a piece of content.

2. Use og:image:width and og:image:height Open Graph tags
Using these tags will specify the image to the crawler so that it can render it immediately without having to asynchronously.

So do the second one – specify the dimensions of your preview image through those og meta tags.

Upvotes: 2

Related Questions