CODEWITHSUNDEEP
pythontestingdocker
cs_stackX
cs_stackX

Reputation: 1527

Code from different git branches in one Python test

I have some machine learning code (but for the purposes of my question, it could be any algorithm) written in Python under git version control. My colleague has just concluded a significant refactor of the code base on a feature branch. Now I want to write some acceptance tests to make sure that the results of querying the machine learning process in production are within a reasonable statistical range compared to the same query on the refactored branch of the code base.

This is a somewhat complex app running in docker containers. The tests need to be run as part of a much larger series of tests.

I am not confused about how to write the part of the test that determines whether the results are within a reasonable statistical range. I am confused about how to bring the results from the master branch code together with the results from the WIP branch for comparison in one test in an automated way

So far, my best (only) idea is to launch the production docker container, run the machine learning query, write the query result to a csv file and then to use the docker cp to copy this file to the local machine where it can be tested against the equivalent csv on the feature branch. However, this seems inelegant.

Is there a better/smarter/best practice way to do this? Ideally that keeps things in memory.

Upvotes: 1

Views: 411

Answers (1)

littlepea
littlepea

Reputation: 1034

I would consider using the http://approvaltests.com/ framework, you just need to write the test code that will produce some output after executing the Python code being tested, the output can be anything (text, JSON/CSV file, etc).

You can run this test on the master branch first so it records the output as the approved baseline and then you can switch to your WIP branch and run the same test there if output differs the approval test will fail.

Check out this podcast episode for more info.

Upvotes: 1

Related Questions