# Testing tips & examples

## General philosophy
https://pythontesting.net/strategy/given-when-then-2/

## Unit testing API client code

### Mocking and testing "wrapper" code for an API client

Let's say we're using a Python package that provides client functionality for the [**Workflow Execution Service**]() API. This library includes the `WESClient` class that we can instantiate and use to interact with a WES server. However, if we have code that *wraps* and does something with the `WESClient` object, we want to be able to test just that added functionality (not the behavior of the `WESClient` class itself).

In this [example](https://github.com/Sage-Bionetworks/workflow-interop/blob/develop/wfinterop/wes/client.py#L62-L75), the wrapping method is actually part of an "adapter" class for the WES API; the `WESAdapter` is designed to pass and translate inputs/outputs to and from the `WESClient`, but through a more standard interface. So, we have a method called `GetServiceInfo()` that wraps `get_service_info()` from our client library. How do we test that `GetServiceInfo()` as behaving correctly?

class WESAdapter(WESInterface):
    """
    Adapter class for the WES client functionality from the
    workflow-service library.
    :param wes_client:
    """
    _wes_client = None

    def __init__(self, wes_client):
        self._wes_client = wes_client

    def GetServiceInfo(self):
        return self._wes_client.get_service_info()

Here is the [test code](https://github.com/Sage-Bionetworks/workflow-interop/blob/develop/tests/test_wes.py#L62-L72) for the method. We instanstiate a `WESAdapter` object (`wes_adapter`), make an example request, and verify that the underlying `get_service_info()` method is called with the expected arguments.

def test_GetServiceInfo(self, mock_client_lib):
    mock_response = {}
    mock_client_lib.get_service_info.return_value = mock_response

    wes_adapter = WESAdapter(wes_client=mock_client_lib)
    test_args = {arg: '' for arg in    
                inspect.getargspec(WESClient.get_service_info)[0][1:]}
    test_response = wes_adapter.GetServiceInfo()

    mock_client_lib.get_service_info.assert_called_once_with(**test_args)
    assert test_response == mock_response

Some things to note:

+ We don't care about the output of `get_service_info()` — that's not what we're testing here. It's not obvious here, but the `mock_client_lib` variable is a `mock.Mock` object. This object includes the `return_value()` setter, which allows us to pre-define the output for any given method. In this case, we'll just set the response to be an empty dict.
+ Rather than manually defining test values for all the arguments to `get_service_info()`, we're a list comprehension to automatically assign a blank string `''` for any arguments captured by `inspect.getargspec()` — in hindsight, this might be problematic if any of the arguments expected non-string types...
+ But where does `mock_client_lib` actually come from? We'll get to that in a minute.

First, a brief digression on naming. This is totally a matter of personal preference, but I tend to use the `mock_` prefix to denote items that are part of my *state* (or "environment") for the test. I use `test_` to indicate the inputs and outputs of the test itself. Using the Given-When-Then structure from the article linked above, the test would look something like this:

def test_GetServiceInfo(self, mock_client_lib):
    # GIVEN a mock client library and a dummy response
    mock_response = {}
    mock_client_lib.get_service_info.return_value = mock_response

    # WHEN a WESAdapter object is initialized to wrap the client library
    # (I maybe should have named this 'test_wes_adapter'...)
    wes_adapter = WESAdapter(wes_client=mock_client_lib)
    
    # AND the GetServiceInfo method of the object is called with a
    # known set of arguments
    test_args = {arg: '' for arg in    
                inspect.getargspec(WESClient.get_service_info)[0][1:]}
    test_response = wes_adapter.GetServiceInfo()

    # THEN the client library should have been called with the expected
    # arguments
    mock_client_lib.get_service_info.assert_called_once_with(**test_args)
    
    # AND the response should match the expected dummy response
    assert test_response == mock_response

It's not a perfect naming convention by any means, so no need to be overly strict about it — but it can help to keep track of things.

Back to `mock_client_lib`...

For convenience and reuse across our tests, we can define a "fixture"[example](https://github.com/Sage-Bionetworks/workflow-interop/blob/develop/tests/conftest.py#L134-L139)

@pytest.fixture()
def mock_client_lib(request):
    mock_wes_client = mock.Mock(name='mock WESClient')
    with mock.patch('wes_client.util.WESClient', 
                    autospec=True, spec_set=True):
        yield mock_wes_client

Mocking API responses:
[example](https://github.com/Sage-Bionetworks/workflow-interop/blob/develop/tests/conftest.py#L150-L161)

@pytest.fixture(params=[None, 'workflow-service'])
def mock_wes_client(request):
    if request.param is None:
        mock_api_client = mock.Mock(name='mock SwaggerClient')
        with mock.patch.object(SwaggerClient, 'from_spec', 
            return_value=mock_api_client):
            yield mock_api_client
    else:
        mock_api_client = mock.Mock(name='mock WESAdapter')
        with mock.patch('wfinterop.wes.client.WESAdapter', 
                        autospec=True):
            yield mock_api_client

[example](https://github.com/Sage-Bionetworks/workflow-interop/blob/develop/tests/test_wes.py#L184-L194)

def test_get_service_info_direct(self, mock_wes_client):
    mock_service_info = {'workflow_type_versions': ['CWL', 'WDL']}

    mock_wes_client.GetServiceInfo.return_value = mock_service_info

    wes_instance = WES(wes_id='mock_wes', 
                       api_client=mock_wes_client)
    test_service_info = wes_instance.get_service_info()

    assert isinstance(test_service_info, dict) 
    assert test_service_info == mock_service_info

Testing business logic & monkeypatching

def test_run_submission(mock_submission, 
                        mock_run_log,
                        mock_wes, 
                        monkeypatch):
    monkeypatch.setattr('wfinterop.orchestrator.get_submission_bundle', 
                        lambda x,y: mock_submission['mock_sub'])
    monkeypatch.setattr('wfinterop.orchestrator.update_submission', 
                        lambda w,x,y,z: None)

    monkeypatch.setattr('wfinterop.orchestrator.run_job', 
                        lambda **kwargs: mock_run_log)

    test_run_log = run_submission(queue_id='mock_queue',
                                  submission_id='mock_sub')

    assert test_run_log == mock_run_log