Does my test automation strategy sound ludicrous?

Question

I am developing an automation testing framework for testing a web service. The web service is SOAP and implemented in Java (via Apache Axis2), however, our tests are implemented in Python and uses the suds library to issue requests to the server. The tests are high level tests that tests scenarios such as backing up a user's data.

Now this web service is complicated in that certain methods require passing lots of different types of objects and often require calling other methods to acquire certain objects. For example, we have a call for backing up a user whose pre-reqs looks like this :

1. call method getUser() to retrieve object User
2. call method getDataset() to retrieve object Dataset
3. call method getXService() to retrieve object XService
4. call method doBackup(User, Dataset, XService) to begin the backup

This is just a simple example of one of our calls that have many other pre-requisite calls before the primary call can be made. Since these scenarios need to be executed often in the tests, I want to abstract the set of calls so that to perform the above backup, I just need to call one method that just calls the other methods for me.

My question is, should I do this in an object-oriented fashion and pretty much create a 1-to-1 mapping of Python classes that map to the Java version of the objects? So my testing framework would just include classes so I could do:

# User, XService, and Dataset are classes that correspond with 
# types implemented in the SOAP web service
from lib import XService, Dataset 

class User():
  def __init__(self, **kwargs):
    self.id = kwargs.get('id', None)

  def create(self):
    soap_client.call('createClient', self.id)

  def backup(self):
    dataset = Dataset.get(1234)
    service = XService.getInstance() 
    soap_client.call('doBackup', self, dataset, service)

So all I have to do is call backup() rather than issue 3 different methods everytime.

The downside of this is that I have to create an object for every Type on the server. Moreover, the objects on the testing side may get stale since they don't sync with data on the server.

My other idea was not to go the Object Oriented route and just create a module with just methods like backupUser() or restoreBackupToUser() and just feed them actual data objects returned from the server. This approach would solve the stale data issue but also would create massive modules that would be hard to maintain.

Can anyone who has encountered this problem give some tips or critique on my approaches. Perhaps I'm thinking too much and there is a better way to go about testing the web service methods.

Answer 1

I had to create an automated test suite for testing a web service a while back. I wrote the whole thing object orientated as there were a lot of methods that each test would use in common. Also this saved a lot of time when it came to negative testing etc. If you have the time writing you're test this way will be earier to maintain and once you have wrote all the initial methods to be called you'll save a lot of time as you're tests will just be several function calls with very little to no logic. Here is an example of something we done:

import suds, unittest, pexpect, re, os,time, sys, random
from random import randrange
from setauth import Authentication

####################################################################
#
#                 Add/Get Profile Function Tests
#
####################################################################

class TestAddGetProfile(unittest.TestCase):

    def setUp(self):
        project = testvars[0]
        media_address = testvars[5]
        mgmt_address = testvars[4]
        self.profile = testvars[7]

        # connects to WSDL file and stores location in variable 'client'
        self.client = testvars[15]
        self.client.options.location = media_address
        self.mgmtclient = testvars[14]
        self.mgmtclient.options.location = mgmt_address

        self.getProfileToken = self.client.factory.create('ns4:ReferenceToken')
        self.getProfileToken.value = self.profile

        self.createProfileToken = self.client.factory.create('ns4:ReferenceToken')
        self.createProfileToken.value = self.profile

        self.createProfileName = self.client.factory.create('ns4:Name')
        self.createProfileName.value = self.profile

        self.deleteProfileToken = self.client.factory.create('ns4:ReferenceToken')
        self.deleteProfileToken.value = self.profile

        self.auth = Authentication()
        self.auth.setAuth('Administrator', self.client, self.mgmtclient)
        self.auth.makeUsers(self.mgmtclient)


    def sudsCreateProfile(self):

        create_result = self.client.service.CreateProfile(self.createProfileName, self.createProfileToken)

    def tearDown(self):
        self.auth.deleteAllUsers(self.mgmtclient)
        self.client = None

class AdminCreateProfile(TestAddGetProfile):

    def runTest(self):
        self.auth.setAuth('Administrator', self.client, self.mgmtclient)
        try:
            self.sudsGetProfile()
            self.sudsDeleteProfile()
        except:
            pass
        try:
            self.sudsCreateProfile()
            self.assertTrue(True)
        except:
            self.assertFalse(True)

We have several other script's here for authentication and a test wrapper to run all the tests. But you should get the general idea here.

Hope this help's you're decision.