phpunit - fake api response or real response connecting to service?

Question

I have created a few methods in a service class to connect to external service/provider via Guzzle using API POST Request.

I like to use phpunit for testing - should I use fake the HTTP Json response without connecting to a service or should it connect to a service to get real response from service?

Answer 1

You don't want to keep testing a 3rd party remote API whenever you run your own app tests, even if you group/isolate them and don't run them often.

Test your code up to the point of calling the real API, then trigger a mock response. This tells you that all your code is working, and if the API would have been connectable and would have returned a response then everything is fine.

Unit tests are not for testing remote APIs, that is the job of some other area of the site to ensure is up, like server monitoring with it and your app issuing error logs on production when something with the remote API fails.

The chance is so low of running your tests at the time that the remote API has some issue, you will run your tests and the remote API works fine, and you deploy, then next day the remote API fails and no-one is running the tests.

You have to assume the remote API works 99% of the time, and when it doesn't you get an error log that the request timed out or returned a bad response etc. And if the remote only works a low amount of time like 50/60%, it's time to find another service provider.

Answer 2

You should mock external API calls. Your tests are intended to test YOUR code not the external API.

Note: Ultimately if you DO use the external API with your tests you will likely be connecting to a test version of the API (different to the live version of the API that your production environment would connect to) so you're not really ensuring the consistency of the API response anyway.

Answer 3

A common tenet in testing is "Don't mock what you don't own". Mocking those API-calls makes your tests less reliable and will give you a false sense of security as you will likely get false positives.

For example when the API unexpectedly introduces a breaking change your tests will be green and once you deploy to production you will finally notice that something is wrong. This is probably what you want your tests to catch.

When you test against the real API you will get a sense of reliability. Do your tests often fail due to service outages or timeouts? If so you can introduce measure like a retry-mechanism, circuit breakers or decoupling the API calls from the rest of your application.

When you mock the API all you can tell is that your code behaves according to a (possibly outdated) specification of the service. That is ok, but not nearly as useful as relying on the actual service.

What you could do is use groups to run those tests separately. This will make it easy to include/exclude these possibly slow and sometimes flaky tests from your remaining tests. This also helps around rate limiting, e.g. by only running those tests on critical branches.