How To Extract Data From JSON with Rest Template in Java

Question

i have to extract the first 5 articles from https://newsapi.org/v2/top-headlines?sources=bbc-news&apiKey=19acc3a371d145ecb37a093f9985ea21, having a result like this:

{
    "total": 5,
    "articles": [
{
    "source": "Ilmessaggero.it",
    "title": "Title",
    "author": "Author",
    "url": "URL"
  }
 ]
}

I did this, having all the JSON as String as output for the localhost...

@RequestMapping("/news")
    public Article connection() {

        return restTemplate.getForObject
                ("https://newsapi.org/v2/top-headlines?sources=bbc-news&apiKey=19acc3a371d145ecb37a093f9985ea21",  Article.class);

The result in the localhost is:

{"source":null,"title":null,"author":null,"url":null}

But the problem now is, how do i put the data into the list of articles? and how do i save them into mongodb? thanks for the effort

Answer 1

I solved it! SImply, the NewsAPI json of Article has a field called Source, which i was trying to parse as a string, but it was NOT! Infact, it is a field described with another object! I simply had to create a class called Source with id and name, and it works! Thanks everyone for the effort!

Here's the codes of the classes:

public class Article {

    private Source source;

    private String author;

    private String title;

    private String url;

    //getters and setters

News, which has a list of articles:

public class News {

    private int totalResults;

    private List<Article> articles;

    //getters and setters

And source, which is called in Article:

public class Source {

    private String id;

    private String name;

    //getters and setters

Here it is! The parse code is the same of the answer. Just change the return type (Article) as News and the Article.class parameter of getForObject into News.class

Answer 2

A simple (i.e. missing exception handling, etc) way is as follows:

First, you need a class to represent the data you are receiving, with fields that match the API response fields, for example:

public class Article {

    private String source;
    private String title;
    ...  // more fields

   // getters and setters
}

The code to fetch the data from the API then looks like this:

RestTemplate template = ...  // initialized earlier
ResponseEntity<Article[]> response = template.exchange(
    API_URL,  // url to the api
    HttpMethod.GET,  // use the Http verb "GET"
    new HttpEntity<>(headers),  // optional headers, e.g. for basic auth
    Article[].class  // the expected response type is Article[]
);

Article[] articles = response.getBody();
List<Article> list = Arrays.asList(articles);  // if you need to use collections

Note, a ResponseEntity being non-null does not imply that the request was successful. You can use responseEntity.getStatusCode() to determine the status code of the response.

Be careful, however, since by default, RestTemplate throws an exception when a non-200 error code is recieved (HttpClientErrorException and HttpServerErrorException for 4XX and 5XX codes respectively). If you want your own custom error handling, you should call:

template.setErrorHandler(new ResponseErrorHandler() {
    @Override
    public boolean hasError(ClientHttpResponse response) throws IOException {
        // implement here
    }

    @Override
    public void handleError(ClientHttpResponse response) throws IOException {
        // implement here
    }
});

For persistence into MongoDB, you can use JPA, although JPA is not a perfect fit for MongoDB due to its inherently relational nature clashing with Mongo's non-relational structure. Something like Spring Data can more sensibly map this, and is worth looking into: https://spring.io/projects/spring-data-mongodb

EDIT - calling this code

Typically, I will create an class/interface with implementation (called ArticleResource for example) that looks like:

public class ArticleResource {

    private final RestTemplate template = new RestTemplate();

    public List<Article> getAllArticles() {
        ResponseEntity<Article[]> response = template.exchange(API_URL, HttpMethod.GET, new HttpEntity<>(headers), Article[].class);

    // some error checking here

    return response.getBody() == null ? Collections.emptyList() : Arrays.asList(response.getBody());
    }
}

For methods that expect a single value (e.g. findArticleByTitle(String title)) I typically return an Optional<Article> (it is bad practice to return Optional<List<T>>, as an empty list represents "no values" already).

From there in your code you can call:

ArticleResource resource = new ArticeResource();

// if you want to print all the names for example:
resource.getAllArticles().stream().map(Article::getName).forEach(System.out::println);