Mocking, Python

I have used Python mock and patch to do some testing for the OPTIONS call and also the server type check in Swift. unittest.mock is a library for testing in Python. It can be used to replace parts of the code, return data and then make assertions about them.

class unittest.mock.Mock(side_effect=None, return_value=DEFAULT, **kwargs)

side_effect: A function to be called whenever the Mock is called. It is useful to raise exceptions.
return_value: The value returned when the mock is called.

Mock provides a patching module that could be used to patch a specific function, class, class level attributes in the scope of test. For instance,

@mock.patch(‘eventlet.green.urllib2.urlopen’)
def test_foo(self, mock_urlopen):
def getheader(name):
d = {‘Server’: ‘server-type’}
return d.get(name)
mock_urlopen.return_value.info.return_value.getheader = getheader

In the above function that I used a mock patch to mock urlopen and it’s return value. To test the OPTIONS call in swift-recon, this simulation will faciliate me by returning header value of “Server” as “server-type”. Further, I could assert the return of the actual value returned by the scout_server_type method in recon (which calls OPTIONS), like this:

self.assertEqual(content, ‘server-type’)

Content here is the return value of OPTIONS call in recon.py.

To test the validate servers method (server_type_check) call, I could have created a fake class for scout_server_type or mock the method itself. I have done the latter:

def mock_scout_server_type(app, host):
url = ‘http://%s:%s/’ % (host[0], host[1])
response = responses[host[1]]
status = 200
return url, response, status

patches = [
mock.patch(‘swift.cli.recon.Scout.scout_server_type’,
mock_scout_server_type),
mock.patch(‘sys.stdout’, new=stdout),
]

This mock, will allow me to replace the scout_server_type call and instead call “mock_scout_server_type” and returns those method’s values for the scope of the test.

with nested(*patches):
self.recon.server_type_check(hosts)

With the above, when server_type_check is called, the internal call to scout_server_type will be replaced by mock_scout_server_type.

Swift-recon

In continuation with the last post – the recent development is I have build some tests to test the base class. Also added a Server header for OPTIONS. Server is a standard header to inform the client about the type of software used by origin server.[1]

The goal is to add a feature to swift-recon cli to validate servers in the ring. Swift-recon cli is a command line utility to obtain various metrics and telemetry from the servers. To validate servers, I need to get the Server type by calling OPTIONS. This support is added in swift-recon to call OPTIONS (only GET is supported until now). The new command to validate ring will be

–validate-servers

The server_type_check method which I newly created will be called when the command is given. This fetches all the hosts and loops through them by calling scout_server_type (in Scout) which in turn calls OPTIONS on the host:

for url, response in self.pool.imap(recon.scout_server_type, hosts)

imap is a itertool [2]¬†which creates iterator for looping. In the above line, “recon.scout_server_type” is called for each of the “hosts”.

The method responds to the arguments – object/container/account and verify if the servers in the ring indeed match to the argument given. For instance,

swift-recon object –validate-servers

For the above command, it will verify if all of the servers are of object type (by passing the host name to OPTIONS and getting the Server header). If it is not, it displays what they are – container/account.

This needs tests to verify. This post will be continued in the next by detailing the tests and/or any obstacles.

References:
[1] http://tools.ietf.org/html/rfc7231#section-7.4.2
[2] https://docs.python.org/2/library/itertools.html