CODEWITHSUNDEEP
pythongoogle-cloud-platform
Pierre56
Pierre56

Reputation: 567

BigQuery unit testing using Python

I am trying to test a BigQuery class with a Mock object to represent the table. Instances of my BigQueryRequest class must provide the BigQuery table uri. Would it be possible for me to create a mock BigQuery table directly from Python? How would it possible?

class BigQueryRequest:
    """BigQueryRequest
    Contains a BigQuery request with its parameter.
    Receive a table uri ($project_id.$dataset.$table) to run query
    Args:
        uri (str): BigQuery table uri
    Properties:
        BigQueryRequest.project: return the project running BigQuery
        BigQueryRequest.dataset: return the dataset
        BigQueryRequest.table: return the table to query
        BigQueryRequest.destination_project: same as project but for destination project
        BigQueryRequest.destination_dataset: same as project but for destination dataset
        BigQueryRequest.destination_table: same as project but for destination table
    Methods:
        from_uri(): (@classmethod) parse a BigQuery uri to its project, dataset, table
        destination(): return a uri of the BigQuery request destination table
        query(): run the given BigQuery query
    Private methods:
        __set_destination(): generate a destination uri following the nomenclature or reuse entry uri
    """

    def __init__(self, uri="", step="", params={}):
        self.project, self.dataset, self.table = self.from_uri(uri)
        self.step = step
        self.params = self.set_params(params)
        self.overwrite = False
        (
            self.destination_project,
            self.destination_dataset,
            self.destination_table,
        ) = self.__set_destination()

Upvotes: 4

Views: 7654

Answers (2)

Bounkong KHAMPHOUSONE
Bounkong KHAMPHOUSONE

Reputation: 183

if you plan to test your SQL and assert your result based on input, I would suggest bq-test-kit. This framework allows you to interact with BigQuery in Python and make tests reliables.

You have 3 ways to inject data into it :

  • Create datasets and tables with an ability to isolate their name and therefore have your own namespace
  • Rely on temp tables, where data is inserted with data literals
  • data literal merged into your query

Hope that this helps.

Upvotes: 1

Dustin Ingram
Dustin Ingram

Reputation: 21520

You'd have to do it yourself, Google Cloud does not provide an official mocking library for GCP products or service.

You could also try https://github.com/Khan/tinyquery as an alternative.

Upvotes: 4

Related Questions