CODEWITHSUNDEEP
httpmeteorcallresponse
John McNeill
John McNeill

Reputation: 153

Meteor HTTP.call("get") response

I am trying to pull basic information from various websites such as a title and description. I can successfully get a response object and what I get is the entire website HTML. I have been playing around with the options in the HTTP.call method but can't figure out how to only return exactly what I want from the response object. Here are the two elements I am trying to get:

<meta property="og:description" content="Our unique teaching style lets students develop their creative potential while learning solid computing skills.">

and 

<meta property="og:site_name" content="Goldsmiths, University of London">

I could get the title easy enough by searching the result for <title></title>, but there must be a better way using params or data in the call method options.

Meteor.methods({
    getInfo: function (url) {
        HTTP.call('GET', url, {}, function (error, result) {
            if (!error) {
                //console.log(result);
                var titleStart = result.content.toLowerCase().indexOf('<title>'),
                    titleEnd = result.content.toLowerCase().indexOf('</title>'),
                    titleText = result.content.substring(titleStart + '<title>'.length, titleEnd)}

Upvotes: 1

Views: 230

Answers (1)

Brett McLain
Brett McLain

Reputation: 2010

Check out the meteor-scrape package.

https://github.com/Anonyfox/meteor-scrape

Here is an example of how you may use it:

# scrape any website
websiteData = Scrape.website "http://example.com/article"

Results in:

{
  title: 'The Avengers (2012 film)'
  lang: 'en'
  descriptions: [ '2012 superhero film produced by Marvel Studios' ]
  tags: [ 'avengers' ]
  url: 'http://en.wikipedia.org/wiki/The_Avengers_(2012_film)'
  summary: '<p><i><b>Marvel\'s The Avengers</b></i> (classified under the name <i><b>Marvel Avengers Assemble</b></i> in the United Kingdom and Ireland), or simply <i><b>The Avengers</b></i>, is a 2012 American superhero film based on the Marvel Comics superhero team of the same name, produced by Marvel Studios and distributed by Walt Disney Studios Motion Pictures.<sup class="reference plainlinks nourlexpansion" id="ref_1">1</sup> It is the sixth installment in the Marvel Cinematic Universe. The film was written [...]'
  meta:
    caption: 'Theatrical release poster'
    director: '[Joss Whedon](http://en.wikipedia.org/wiki/Joss_Whedon)'
    producer: '[Kevin Feige](http://en.wikipedia.org/wiki/Kevin_Feige)'
    screenplay: 'Joss Whedon'
    based: '[The Avengers](http://en.wikipedia.org/wiki/Avengers_(comics))'
    music: '[Alan Silvestri](http://en.wikipedia.org/wiki/Alan_Silvestri)'
    cinematography: '[Seamus McGarvey](http://en.wikipedia.org/wiki/Seamus_McGarvey)'
    studio: '[Marvel Studios](http://en.wikipedia.org/wiki/Marvel_Studios)'
    runtime: '143 minutes'
    country: 'United States'
    language: 'English'
    budget: '$220 million'
    gross: '$1.518 billion'
}

Upvotes: 1

Related Questions