CODEWITHSUNDEEP
scalaunit-testingtestingamazon-s3junit
B. Smith
B. Smith

Reputation: 1193

Can you unit test s3 writes?

Unit testing a method that verifies a particular s3 bucket exists is simple because the unit test would not change the state of s3. However, unit testing a method that writes to an s3 bucket is more problematic because it actually changes the state of s3 every time the unit test is run. For example, a method that uses rdd.saveAsTextFile(path).

Can a unit test be written for a method that writes to s3 that still allows you to verify that the file got written?

Note: I thought about using a mock object, but wasn't sure how this might be implemented.

Edit: I should probably note that I am using Scala

Upvotes: 1

Views: 4381

Answers (3)

kag0
kag0

Reputation: 6054

You might want to check out mockito.

What you'd do is create a mock of your s3 client (using mock), stub out the put method (with when and .thenReturn), and then verify that the s3 put was done with the right data to the right bucket and key (using verify).

val s3 = org.scalatest.mock.MockitoSugar.mock[AmazonS3]
when(s3.putObject(any[String], any[String], any[String]).thenReturn(new PutObjectResult)

-- execute the code under test here --

verify(s3).putObject(theExpectedBucket, theExpectedKey, theExpectedData)

Upvotes: 4

Carl Manaster
Carl Manaster

Reputation: 40336

It's not strictly a unit test, but I've had good luck with something like the following

describe ('s3', () => {
  const contents = uuid.v4()
  const key = uuid.v4()

  it('should write to s3', () => {
    const result = write(key, contents)
    assertSuccess(result)
  }

  it('should read from s3', () => {
    const result = read(key)
    assertSuccess(read, contents)
  }

  it('should clear from s3', () => {
    const result = clear(key)
    assertSuccess(result)
  }

  it('should no longer have the value', () => {
    const result = read(key)
    assertFailure(result) // or maybe assertSuccess(result, empty), depending on your requirements
  }

})

In my experience this has satisfied the loose requirements of unit testing (fast, reliable) without 100% satisfying the hard requirements of test independence and decoupling.

Upvotes: 0

Shane Burroughs
Shane Burroughs

Reputation: 111

Typically when writing unit tests, it is advised as you noted that you "mock" or "stub" out any calls to external APIs. This allows you to test ONLY a given unit of work, in this case your code that writes to s3.

In your case, you would mock something like the aws sdk object used to communicate with s3 and specify what the expected return would be for calls.

http://www.baeldung.com/mockito-annotations

Upvotes: 0

Related Questions