Mocking Requests in Python

Unit tests are easier to write when given a set of inputs you know you're always going to get an expected response. It becomes a bit difficult when the same set of inputs can lead to varying responses.

When you work on programs where sometimes you have to make a request to an external API or making requests to other services in your microservices, you can't be sure you're going to get 200 status code on each request.

You'll mostly put in measures to handle when your requests fail but how do you write unit tests to ensure your system works as expected.

Writing Program

Let's build a program that a pings a url and returns True if the status code is 200 and False when it is not.

1. Create a file called client.py
   


2. Write the following code
   

# client.py



# Make sure you have requests installed.

# If not, you can install with `pip3 install requests` 

# or `pip install requests`.

import requests



def ping(url):

    res = requests.get(url)

    retun res.status_code == 200



print(ping("https://google.com"))

1. Run python3 client.py in your terminal in the directory where client.py resides. The below should be your response;
   

❯ python3 client.py

True

Writing Tests

Our program works as expected. It's time to write tests to ensure that we return True when our request is 200 and False when the status code is not 200.

1. Create a file called test_client.py in the same directory as client.py
   


2. Write the following code;
   

# test_client.py

import unittest

from client import ping



class TestClient(unittest.TestCase):

    def setUp(self):

        self.url = "https://google.com"



    def test_ping_returns_200(self):

        result = ping(self.url)

        self.assertTrue(result)



if __name__ == "__main__":

    unittest.main()

1. Run python3 test_client.py. You should get a response like below;
   

❯ python3 test_client.py

.

---------------------------------------------------------

Ran 1 test in 1.113s



OK

All our tests are passing but now let's test for the case when the status code is not 200. Let's add another test to our test

# test_client.py

import unittest

from client import ping



class TestClient(unittest.TestCase):

    def setUp(self):

        self.url = "https://google.com"



    def test_ping_returns_200(self):

        result = ping(self.url)

        self.assertTrue(result)



    # New

     def test_ping_returns_500(self):

        result = ping(self.url)

        self.assertFalse(result)





if __name__ == "__main__":

    unittest.main()

When we rerun python3 test_client.py, our new test fails.

❯ python3 test_client.py

.F

==========================================================

FAIL: test_ping_returns_500 (__main__.TestClient.test_ping_returns_500)

----------------------------------------------------------

Traceback (most recent call last):

  File "/Users/mock/test_client.py", line 46, in test_ping_returns_500

    self.assertFalse(result)

AssertionError: True is not false



-----------------------------------------------------------

Ran 2 tests in 2.076s



FAILED (failures=1)

This is because for now our request returns 200, so we are unable to test for when our ping fails.

To be able to do this we are going to mock our requests and responses. This gives us the ability to determine what response we should get.

This way we can manipulate our requests to return 500 response status code instead of the actual status code.

Mocking requests and responses

First we're going to import the patch decorator and MagicMock object from unittest.mock.

The patch decorator allows us to edit our requests, it takes a target which will be the requests we want to change, which in this case is the one in client.py.

The MagicMock object allows us to create the response we want so we can pass it to requests.

We have add an additional parameter which can be named anyway we want. I'll name it mock_requests. This is the object we can attach our response to.

Let's add these changes to our test_client.py;

# test_client.py

import unittest

from unittest.mock import patch, MagicMock

from client import ping



class TestClient(unittest.TestCase):

    def setUp(self):

        self.url = "https://google.com"



    def test_ping_returns_200(self):

        result = ping(self.url)

        self.assertTrue(result)





    @patch("client.requests")   # New

    def test_ping_returns_500(self, mock_requests):    # New



        # New

        mock_response = MagicMock()

        mock_response.status_code = 500

        mock_requests.get.return_value = mock_response



        result = ping(self.url)

        self.assertFalse(result)





if __name__ == "__main__":

    unittest.main()

Let's dive into what we've done so far.

1. By adding the decorator @patch("client.requests"), we're saying we want to modify the requests function that we imported in client.py.
   > It can be any function, we could have targetted ping itself. You can try that on your own after this
   The patch decorator modifies our test method and passes in a second argument to our test_ping_returns_500 method. Thus, we add mock_requests as a parameter to our method. This mock_requests is a mock variant of requests
   


2. We instantiate our MagicMock object and assign to mock_response. Our MagicMock object allows us to take the form of any class we want and in this case a Response object.


3. We add a status_code attribute to our mock_response and assign it the value of 500.


4. We proceed to assign our mock_response to mock_requests.get.return_value.
   We have now modified our requests in that, we're saying, when the requests in client.py calls it's get function, we want the value returned to be that of our mock_response.
   Thus when requests.get() is called, the value returned is going to be the mock_response.

If we run our test again, we see everything passes now.

❯ python3 test_client.py

..

----------------------------------------------------------------------

Ran 2 tests in 1.609s



OK

We can proceed to do the same for test_ping_returns_200 method but make the status_code 200 so we can add a fake url instead of making requests to https://google.com.

# test_client.py

import unittest

from unittest.mock import patch, MagicMock

from client import ping



class TestClient(unittest.TestCase):

    def setUp(self):

        self.url = "https://api.service.com"    # New



    @patch("client.requests")   # New

    def test_ping_returns_200(self, mock_requests):    # New



        # New

        mock_response = MagicMock()

        mock_response.status_code = 200

        mock_requests.get.return_value = mock_response





        result = ping(self.url)

        self.assertTrue(result)





    @patch("client.requests")

    def test_ping_returns_500(self, mock_requests):



        mock_response = MagicMock()

        mock_response.status_code = 500

        mock_requests.get.return_value = mock_response



        result = ping(self.url)

        self.assertFalse(result)





if __name__ == "__main__":

    unittest.main()

We can run our tests again to ensure everything works well.

❯ python3 test_client.py

..

----------------------------------------------------------------------

Ran 2 tests in 1.609s



OK

Mocking response body

We can also add response body to our mock_response. Assuming, the 3rd party API we're pinging returns a json response with an attribute {"status": "ok"}.

1. Let's modify our function in client.py to return the json response too when the status_code is 200 and None when it is not.
   

# client.py



import requests



def ping(url):

    res = requests.get(url)

    # New

    if res.status_code == 200:

        return (True, res.json())

    return (False, None)

1. We modify our test_client.py to this;
   

# test_client.py

import unittest

from unittest.mock import patch, MagicMock

from client import ping



class TestClient(unittest.TestCase):

    def setUp(self):

        self.url = "https://api.service.com"



    @patch("client.requests")

    def test_ping_returns_200(self, mock_requests):



        mock_response = MagicMock()

        mock_response.status_code = 200

        mock_response.json.return_value = {"status": "ok"}  # New

        mock_requests.get.return_value = mock_response





        status, body = ping(self.url)  # New

        self.assertTrue(status)

        self.assertEqual(body["status"], "ok")





    @patch("client.requests")

    def test_ping_returns_500(self, mock_requests):



        mock_response = MagicMock()

        mock_response.status_code = 500

        mock_requests.get.return_value = mock_response



        status, body = ping(self.url)  # New

        self.assertFalse(status)





if __name__ == "__main__":

    unittest.main()

Adding mock_response.json.return_value, we're saying when we call the json function on our mock_response we should get {"status": "ok"}.

Thus, {"status": "ok"} is the value we get when we call res.json() in client.py.

Conclusion

Mocking requests make testing easier especially when you have to deal with 3rd party APIs and even other services in your microservices.

Let's say, you have an API that first makes a request to your auth service to confirm if a user has certain permissions before allowing them access to certain resources.

You can mock the response you have to test that only users with the needed permissions are allowed to access the resource.

Code for this tutorial can be found here https://github.com/quamejnr/python-mock-requests