Adding elements to mocked list

Question

I'm trying to unit test the method responsible for adding to map categorized books.

@Service
public class BookService {

    private final List<BookServiceSource> sources;

    @Autowired
    public BookService(List<BookServiceSource> sources) {
        this.sources = sources;
    }

    public Map<Bookstore, List<Book>> getBooksByCategory(CategoryType category) {
        return sources.stream()
                .collect(Collectors.toMap(BookServiceSource::getName,
                        source -> source.getBooksByCategory(category)));
    }
    }

BookSerivceSource is an interface. This interface is implemented by two classes. I'm gonna provide just one, as the second is really similiar.

EmpikSource (one of implementation)

package bookstore.scraper.book.booksource.empik;

import bookstore.scraper.book.Book;
import bookstore.scraper.book.booksource.BookServiceSource;
import bookstore.scraper.enums.Bookstore;
import bookstore.scraper.enums.CategoryType;
import bookstore.scraper.urlproperties.EmpikUrlProperties;
import bookstore.scraper.utilities.JSoupConnector;
import org.jsoup.nodes.Document;
import org.jsoup.nodes.Element;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.stereotype.Service;

import java.util.*;
import java.util.stream.IntStream;

@Service
public class EmpikSource implements BookServiceSource {

    private static final int FIRST_PART_PRICE = 0;
    private static final int SECOND_PART_PRICE = 1;

    private static final int BESTSELLERS_NUMBER_TO_FETCH = 5;
    private static final int CATEGORIZED_BOOKS_NUMBER_TO_FETCH = 15;
    private static final String DIV_PRODUCT_WRAPPER = "div.productWrapper";
    private static final String DATA_PRODUCT_ID = "data-product-id";

    private final EmpikUrlProperties empikUrlProperties;
    private final JSoupConnector jSoupConnector;
    private Map<CategoryType, String> categoryToEmpikURL;

    @Autowired
    public EmpikSource(EmpikUrlProperties empikUrlProperties, JSoupConnector jSoupConnector) {
        this.empikUrlProperties = empikUrlProperties;
        this.jSoupConnector = jSoupConnector;
        categoryToEmpikURL = createCategoryToEmpikURLMap();
    }

    @Override
    public Bookstore getName() {
        return Bookstore.EMPIK;
    }

    @Override
    public List<Book> getBooksByCategory(CategoryType categoryType) {
        Document document = jSoupConnector.connect(categoryToEmpikURL.get(categoryType));

        List<Book> books = new ArrayList<>();
        List<Element> siteElements = document.select("div.productBox__info");

        IntStream.range(0, CATEGORIZED_BOOKS_NUMBER_TO_FETCH)
                .forEach(iteratedElement -> {

                    String author = executeFetchingAuthorProcess(siteElements, iteratedElement);
                    String price = convertEmpikPriceWithPossibleDiscountToActualPrice(siteElements.get(iteratedElement).select("div.productBox__price").first().text());
                    String title = siteElements.get(iteratedElement).select("span").first().ownText();
                    String productID = siteElements.get(iteratedElement).select("a").first().attr(DATA_PRODUCT_ID);
                    String bookUrl = createBookURL(title, productID);

                    books.add(Book.builder()
                            .author(author)
                            .price(price)
                            .title(title)
                            .productID(productID)
                            .bookURL(bookUrl)
                            .build());
                });

        return books;
    }


    private Map<CategoryType, String> createCategoryToEmpikURLMap() {
        Map<CategoryType, String> map = new EnumMap<>(CategoryType.class);

        map.put(CategoryType.CRIME, empikUrlProperties.getCrime());
        map.put(CategoryType.BESTSELLER, empikUrlProperties.getBestSellers());
        map.put(CategoryType.BIOGRAPHY, empikUrlProperties.getBiographies());
        map.put(CategoryType.FANTASY, empikUrlProperties.getFantasy());
        map.put(CategoryType.GUIDES, empikUrlProperties.getGuides());
        map.put(CategoryType.MOST_PRECISE_BOOK, empikUrlProperties.getMostPreciseBook());
        map.put(CategoryType.ROMANCES, empikUrlProperties.getRomances());

        return map;
    }

    private String convertEmpikPriceWithPossibleDiscountToActualPrice(String price) {
        String[] splittedElements = price.split("\\s+");
        return splittedElements[FIRST_PART_PRICE] + splittedElements[SECOND_PART_PRICE];
    }

    private String createBookURL(String title, String productID) {
        return String.format(empikUrlProperties.getConcreteBook(), title, productID);
    }

    //method is required as on empik site, sometimes occurs null for author and we need to change code for fetching
    private static String executeFetchingAuthorProcess(List<Element> siteElements, int i) {
        String author;
        Element authorElements = siteElements.get(i).select("span > a").first();
        if (authorElements != null)
            author = authorElements.ownText();
        else
            author = siteElements.get(i).select("> span > span").first().text();
        return author;
    }

    private String concatUrlWithTitle(String url, String title) {
        return String.format(url, title);
    }
}

JsoupConnector:

package bookstore.scraper.utilities;

import org.jsoup.Jsoup;
import org.jsoup.nodes.Document;
import org.springframework.stereotype.Component;

import java.io.IOException;

@Component
public class JSoupConnector {

    public Document connect(String url) {
        try {
            return Jsoup.connect(url).get();
        } catch (IOException e) {
            throw new IllegalArgumentException("Cannot connect to" + url);
        }
    }
}

Properties class:

package bookstore.scraper.urlproperties;

import lombok.Getter;
import lombok.Setter;
import org.springframework.boot.context.properties.ConfigurationProperties;
import org.springframework.stereotype.Component;

@Getter
@Setter
@Component
@ConfigurationProperties("external.library.url.empik")
public class EmpikUrlProperties {

    private String mostPreciseBook;
    private String bestSellers;
    private String concreteBook;
    private String romances;
    private String biographies;
    private String crime;
    private String guides;
    private String fantasy;
}

While debugging the test I see that the sources size is 0. How should I add mocked object to the sources list or could you tell me if there is better way to do this?

//EDIT Forgot to paste the test :P

Test

package bookstore.scraper.book;

import bookstore.scraper.book.booksource.BookServiceSource;
import bookstore.scraper.book.booksource.empik.EmpikSource;
import bookstore.scraper.book.booksource.merlin.MerlinSource;
import bookstore.scraper.dataprovider.EmpikBookProvider;
import bookstore.scraper.dataprovider.MerlinBookProvider;
import bookstore.scraper.enums.Bookstore;
import bookstore.scraper.enums.CategoryType;
import bookstore.scraper.urlproperties.EmpikUrlProperties;
import bookstore.scraper.urlproperties.MerlinUrlProperties;
import bookstore.scraper.utilities.JSoupConnector;
import org.jsoup.nodes.Document;
import org.junit.Test;
import org.junit.runner.RunWith;
import org.mockito.InjectMocks;
import org.mockito.Mock;
import org.mockito.junit.MockitoJUnitRunner;

import java.util.List;
import java.util.Map;

import static bookstore.scraper.dataprovider.MergedBestsellersMapProvider.prepareExpectedMergedBestSellerMap;
import static org.junit.Assert.assertEquals;
import static org.mockito.Mockito.mock;
import static org.mockito.Mockito.when;

@RunWith(MockitoJUnitRunner.class)
public class BookServiceTest {

    @Mock
    MerlinSource merlinSource;
    @Mock
    EmpikSource empikSource;
    @Mock
    BookServiceSource bookServiceSource;
    @Mock
    private EmpikUrlProperties empikMock;
    @Mock
    private MerlinUrlProperties merlinMock;
    @Mock
    JSoupConnector jSoupConnector;
    @Mock
    List<BookServiceSource> source;

    @InjectMocks
    BookService bookService;

    @Test
    public void getBooksByCategory() {
        List<Book> merlinBestsellers = MerlinBookProvider.prepare5Bestsellers();
        List<Book> empikBestsellers = EmpikBookProvider.prepare5Bestsellers();
        Document empikDocument = mock(Document.class);
        Document merlinDocument = mock(Document.class);

        source.add(empikSource);
        source.add(merlinSource);

        when(bookServiceSource.getName()).thenReturn(Bookstore.EMPIK);
        when(jSoupConnector.connect("https://www.empik.com/bestsellery/ksiazki")).thenReturn(empikDocument);
        when(empikMock.getBestSellers()).thenReturn("https://www.empik.com/bestsellery/ksiazki");
        when(empikSource.getBooksByCategory(CategoryType.CRIME)).thenReturn(empikBestsellers);

        when(bookServiceSource.getName()).thenReturn(Bookstore.MERLIN);
        when(jSoupConnector.connect("https://merlin.pl/bestseller/?option_80=10349074")).thenReturn(merlinDocument);
        when(merlinMock.getBestSellers()).thenReturn("https://merlin.pl/bestseller/?option_80=10349074");
        when(merlinSource.getBooksByCategory(CategoryType.CRIME)).thenReturn(merlinBestsellers);

        Map<Bookstore, List<Book>> actualMap = bookService.getBooksByCategory(CategoryType.CRIME);
        Map<Bookstore, List<Book>> expectedMap = prepareExpectedMergedBestSellerMap();

        assertEquals(expectedMap, actualMap);
    }


}

Answer 1

As mentioned before do not try to mock the List object.
Also generally avoid to create mocks for objects that you can simply create on your own and try to restrict yourself to mock only dependencies.

A simplified version of your test could look like this:

As your test covers quite a bit more than the Unit BookService I decided to minimize it for this example.

You might want to do all the other stuff in a test for the specific implementation instead.

@Test
public void getBooksByCategory() {

    List<Book> empikBestsellers = EmpikBookProvider.prepare5Bestsellers();
    List<Book> merlinBestsellers = MerlinBookProvider.prepare5Bestsellers();

    BookServiceSource bookServiceSource1 = Mockito.mock(BookServiceSource.class);
    Mockito.when(bookServiceSource1.getName()).thenReturn(Bookstore.EMPIK);
    Mockito.when(bookServiceSource1.getBooksByCategory(CategoryType.CRIME)).thenReturn(empikBestsellers);   

    BookServiceSource bookServiceSource2 = Mockito.mock(BookServiceSource.class);
    Mockito.when(bookServiceSource2.getName()).thenReturn(Bookstore.MERLIN);
    Mockito.when(bookServiceSource2.getBooksByCategory(CategoryType.CRIME)).thenReturn(merlinBestsellers);      

    List<BookServiceSource> sources = new ArrayList<>();
    sources.add(bookServiceSource1);
    sources.add(bookServiceSource2);

    BookService service = new BookService(sources);
    Map<Bookstore, List<Book>> actualMap = service.getBooksByCategory(CategoryType.CRIME);

    // compare result
}

Answer 2

Try @Spy. It allows you to inject actual instance of a list that you have initialized by yourself and which also can be mocked partially.

@Spy
private List<BookServiceSource> sources = new ArrayList<>();

It seems that you have used different name for the List, prefer to use the smae name that field to mock is injected is; sources.

Good explanation here.

5. Mock vs. Spy in Mockito :

When Mockito creates a mock – it does so from the Class of a Type, not from an actual instance. The mock simply creates a bare-bones shell instance of the Class, entirely instrumented to track interactions with it.

On the other hand, the spy will wrap an existing instance. It will still behave in the same way as the normal instance – the only difference is that it will also be instrumented to track all the interactions with it.

Answer 3

I don't believe you should be mocking the list of BookServiceSource since your adds will do nothing since it is not a real list.

This answer here should provide the information you are looking for: Mockito - Injecting a List of mocks

Edit for more clarity:

@InjectMocks should not be used if you can help it, it has a tendency to fail silently.

The other point I was attempting to make is that you are using a mocked list, and because of that when it is told to add elements it will not.

There are two solutions to the problem that you can use. Firstly you could create a when thenreturn for the stream of BookServiceSources, not the recommended solution.

Secondly what would be better is to create a testSetup method making use of the @Before annotation to create the BookService.

@Before
public void testSetup(){
 List<BookServiceSource> list = new LinkedList<>();
 list.add(merlinSource);
 list.add(empikSource);
 bookService = new BookService(list);
}