Testing URL and URLConnection

Question

I don't know how should I test this without really taking connection to real url to server.

I have read few articles about using Mockito in this kind of situation and tried to search around, but can not find good tutorial or advices how should I make jUnit-test for URL and URLConnection in my project.

Here is the code that I have problems when trying to test it:

public JSONObject getJSONObj()
        throws MalformedURLException, IOException, ParseException {
    String jsonString;
    try (InputStream is = getURLConnection("RealUrlStringGoesHere").getInputStream();) {
        jsonString = IOUtils.toString(is);
    }
    return (JSONObject) JSONValue.parseWithException(jsonString);
}

public URLConnection getURLConnection(String urlStr) throws MalformedURLException, IOException {
    URL url = new URL(urlStr);
    URLConnection conn = url.openConnection();
    return conn;
}

Here is also used imports I use for these, if someone wants to know:

import java.net.URLConnection;
import org.apache.commons.io.IOUtils;
import org.json.simple.JSONObject;
import org.json.simple.JSONValue;
import org.json.simple.parser.ParseException;

---

EDITED

Thanks for you answers, but it seems that I'm totally lost with this. Maybe I'm trying to think too complicated, but unit testing is pretty new stuff for me, but really want to learn it more.

Yes, I try to test getJSONObj-method, but those URL & URLConnection is making it difficult for me to understand how to test my method by "faking" it to believe it really takes connection.

Can't realize what you really mean, so here is the current code when I tried to do as you said Jeff Bowman. (Still using that big String, because I tried to get it first done with the current style and then get better performance with Reader's after this is working.)

Discusser.java

import java.io.IOException;
import java.io.InputStream;
import java.net.URL;
import java.net.URLConnection;
import org.apache.commons.io.IOUtils;
import org.json.simple.JSONObject;
import org.json.simple.JSONValue;
import org.json.simple.parser.ParseException;

public class Discusser implements URLOpener {

    public JSONObject getJSONObj() throws IOException, ParseException {
        String jsonString;
        try (InputStream is = openURL("RealUrlStringGoesHere");) {
            jsonString = IOUtils.toString(is);
        }
        return (JSONObject) JSONValue.parseWithException(jsonString);
    }

    @Override
    public InputStream openURL(String urlStr) throws IOException {
        URL url = new URL(urlStr);
        URLConnection urlConnection = url.openConnection();
        return urlConnection.getInputStream();
    }
}

URLOpener.java

import java.io.IOException;
import java.io.InputStream;

public interface URLOpener {
    InputStream openURL(String urlStr) throws IOException;
}

This test is almost useless to show, because I think it's totally wrong how I try to use the mock. (It's returning null when discusser.getJSONObj())

DiscusserTest.java

import static org.junit.Assert.assertEquals;
import java.io.ByteArrayInputStream;
import org.json.simple.JSONObject;
import org.junit.Test;
import org.mockito.Mockito;

public class DiscusserTest {

    @Test
    public void testGetJSONObj() throws Exception {
        JSONObject expectedJSONObject = createExpected();
        ByteArrayInputStream inputForMock = new ByteArrayInputStream(generateJSONString().getBytes("UTF-8"));
        // Should I mock like this or...
        Discusser discusser = Mockito.mock(Discusser.class);
        Mockito.when(discusser.openURL("asd")).thenReturn(inputForMock);
        //
        assertEquals(expectedJSONObject, discusser.getJSONObj());
    }

    private String generateJSONString() {
        StringBuilder sb = new StringBuilder();
        sb.append("{");
        sb.append("\"id\":\"123\",");
        sb.append("\"name\":\"test\"");
        sb.append("}");
        return sb.toString();
    }

    @SuppressWarnings("unchecked")
    private JSONObject createExpected() {
        JSONObject obj = new JSONObject();
        obj.put("id", 123);
        obj.put("name", "test");
        return obj;
    }
}

Could you or someone else give guidance / example how getJSONObj()-method in Discusser should be tested?

Answer 1

I managed to get it working and here are the results. If you have improving ideas or other suggestions, I'm very pleased to have them.

I added setter for URLOpener in Discusser, so then I can put that mocked one there quite easily.

Discusser.java

import java.io.IOException;
import java.io.InputStream;
import java.io.InputStreamReader;

import org.json.simple.JSONObject;
import org.json.simple.JSONValue;
import org.json.simple.parser.ParseException;

public class Discusser implements URLOpener {

    private URLOpener urlOpener;

    public JSONObject getJSONObj() throws IOException, ParseException {
        JSONObject jsonObj;
        try (InputStream is = openURL("RealUrlStringGoesHere");) {
            jsonObj = (JSONObject) JSONValue.parse(new InputStreamReader(is));
        }
        return jsonObj;
    }

    @Override
    public InputStream openURL(String urlStr) throws IOException {
        return urlOpener.openURL(urlStr);
    }

    public void setURLOpener(URLOpener urlOpener) {
        this.urlOpener = urlOpener;
    }

}

DiscusserTest.java

import static org.junit.Assert.assertEquals;
import static org.mockito.Mockito.mock;
import static org.mockito.Mockito.when;
import java.io.ByteArrayInputStream;
import java.io.IOException;
import org.json.simple.JSONObject;
import org.junit.Test;

public class DiscusserTest {

    @Test
    public void testGetJSONObj() throws Exception {
        Discusser discusser = new Discusser();
        discusser.setURLOpener(createMockURLOpener());
        assertEquals(createExpected(), discusser.getJSONObj());
    }

    private URLOpener createMockURLOpener() throws IOException {
        URLOpener mockUrlOpener = mock(URLOpener.class);
        ByteArrayInputStream input = new ByteArrayInputStream(generateJSONString().getBytes("UTF-8"));
        when(mockUrlOpener.openURL("RealUrlStringGoesHere")).thenReturn(input);
        return mockUrlOpener;
    }

    private String generateJSONString() {
        StringBuilder sb = new StringBuilder();
        sb.append("{");
        sb.append("\"id\":\"123\",");
        sb.append("\"name\":\"test\"");
        sb.append("}");
        return sb.toString();
    }

    @SuppressWarnings("unchecked")
    private JSONObject createExpected() {
        JSONObject obj = new JSONObject();
        obj.put("id", "123");
        obj.put("name", "test");
        return obj;
    }
}

URLOpener.java

import java.io.IOException;
import java.io.InputStream;

public interface URLOpener {
    InputStream openURL(String urlStr) throws IOException;
}

Answer 2

What are you trying to test?

If you're trying to test that interactions happen correctly with the real server, then no amount of mocking will help you. You'd want to write an integration test.

If you're trying to test that interactions happen through Java's URLConnection, then a mock server might work the way Stefan Birkner describes. I don't think that's necessarily a useful thing, though, as the Java URLConnection framework is exceedingly well-tested third-party code.

It looks like the component that is most testable here is getJSONObj(), where the part that is not so testable is the function that turns a URL into an InputStream. Make that your interface, instead:

interface URLOpener {
  InputStream openURL(String url);
}

At that point, you can use a very simple real implementation in your production code, or pass in a dead-simple mock that returns a ByteArrayInputStream.

---

Side note: You may find you have better performance if you use JSONValue.parse(Reader) instead of trying to construct one big String containing the entire JSON file. This wouldn't interfere with mocking, as you could just use StringReader instead.

/* in prod, within your actual URLOpener */
return new InputStreamReader(urlConnection.getInputStream());

/* in test, for your mock URLOpener */
when(mockUrlOpener.openURL(expectedURL)).thenReturn(new StringReader(testJSON));

JSONValue value = JSONValue.parse(new BufferedReader(readerFromUrl));

Answer 3

You could start a server within your test and test against this server. You can use MockServer for this.