CODEWITHSUNDEEP
pythonautomated-testspytest
GreenGrassTunnel
GreenGrassTunnel

Reputation: 88

Pytest: How to run variable number of tests defined in json file

I'm using the pytest framework to test an executable. For this executable, I defined multiple test cases in a json file:

{
    "Tests": [
    {
        "name": "test1",
        "description" : "writes hello world to file",
        "exe" : "%path_to_exe%",
        "arguments": "--verbose",
        "expression" : "test1.txt",
        "referencedir": "%path_to_referencedir%",
        "logdir" : "%path_to_logdir%"
    },
    {
        "name": "test2",
        "description" : "returns length of hello world string",
        "exe" : "path_to_exe",
        "arguments": "--verbose",
        "expression" : "test2.txt",
        "referencedir": "%path_to_referencedir%",
        "logdir" : "%path_to_logdir%"
    }
    ]
}

For each of these test cases, the exe should start and execute the expression that is passed by the 'expression' attribute. Its output is written to the logdir (defined by the 'logdir' attribute), which should then be compared with the referencedir. Pytest should then indicate for each of the test cases whether the outputfile in the logdir is identical to to the file in the referencedir.

I'm struggling with making pytest go over each test case one by one.

I'm able to loop over each test, but assertions don't indicate which test exactly is failing.

def test_xxx():
    with open('tests.cfg') as f:
        data = json.loads(f.read())
        for test in data['Tests']:
            assert test['name'] == "test1"

Furthermore, I tried to parametrize the input, but I cannot get it to work either:

def load_test_cases():
    # Opening JSON file
    f = open('tests.cfg')

    # returns JSON object as 
    # a dictionary
    data = json.load(f)

    f.close()

    return data


@pytest.mark.parametrize("test", load_test_cases())
def test_xxx(test):
    assert test['name'] == "test1"

Which returns 'test_json.py::test_xxx[Tests]: string indices must be integers', indicating that it's not really looping over test objects.

Upvotes: 1

Views: 541

Answers (2)

simpleApp
simpleApp

Reputation: 3158

I would suggest parametrizing is a better option here. parametrize expects an iterator object, refer to the example and comments added for code.

import json
import pytest

# method returns an iterator
def get_the_test(): 
    with open('tests.cfg') as f:
        data = json.loads(f.read())
    return iter(data['Tests'])

# use the iterator object to feed into parameters
@pytest.mark.parametrize("test", get_the_test())
def test_xxx(test):
    assert test['name'] == "test1"

Upvotes: 2

GreenGrassTunnel
GreenGrassTunnel

Reputation: 88

Storing the individual test cases in a dictionary did the trick:

import json
import pytest

def load_test_cases():
    # Opening JSON file
    f = open('tests.cfg')
    
    testlist = []
    
    # returns JSON object as 
    # a dictionary
    data = json.load(f)
    for test in data['Tests']:
        testlist.append(test)

    f.close()

    return testlist


@pytest.mark.parametrize("test", load_test_cases())
def test_xxx(test):
    assert test['name'] == "test1"

Upvotes: 0

Related Questions