CODEWITHSUNDEEP
rubyrspec
MuffinTheMan
MuffinTheMan

Reputation: 1591

Identical rspec tests succeed and fail respectively

I am writing a simple program to convert assembly language instructions to their respective 24-bit binary instructions for a processor we're building. I'm writing the program in Ruby (2.0.0) and testing it with Rspec (2.14.6). What's weird is that I can run two identical tests--one of which fails and the other of which succeeds. Here is an example:

Failing test:

it "Returns 24-bit binary instruction for valid line of assembly code" do

  #...25 other instructions are tested before these...
  expect(Parser.build_instruction("Li r2, 0000111001010001")).to eq("001000001110010100010010")
  expect(Parser.build_instruction("Li r2, 0000111001010001")).to eq("001000001110010100010010")

end

Passing test:

it "Returns 24-bit binary instruction for valid line of assembly code" do

  #...25 other instructions are tested before these...
  expect(Parser.build_instruction("Li r2, 0000111001010001")).to eq("001000001110010100010010")
  #expect(Parser.build_instruction("Li r2, 0000111001010001")).to eq("001000001110010100010010")

end

I began to wonder if maybe you could have too many tests in one it...do...end block (or if it was a problem to run the same expect twice), but I tried duplicating some of the tests above these last two, and the tests continued to pass. I also tried pulling some of the tests out into their own it...do...end block, which then caused a different expectation (that was passing before) to fail. I'm a bit baffled. Ideas?

P.S. I don't actually want to run the same expect twice, but it was doing strange things with the below two tests, so I changed the second test to match the first, and it still failed. Also, if I comment out the first test (of these two), the second one passes. In other words, it only fails if I run both tests--not one of the other.

it "Returns 24-bit binary instruction for valid line of assembly code" do

  #...25 other instructions are tested before these...
  expect(Parser.build_instruction("Li r2, 0000111001010001")).to eq("001000001110010100010010")
  expect(Parser.build_instruction("Li r2, 111001010001")).to eq("001000001110010100010010")

end

Output:

1) Parser Returns 24-bit binary instruction for valid line of assembly code
 Failure/Error: expect(Parser.build_instruction("Li r2, 111001010001")).to eq("001000001110010100010010")

   expected: "001000001110010100010010"
        got: "0010000011100101000100001110010100010010"

   (compared using ==)
 # ./spec/parser_spec.rb:65:in `block (2 levels) in <top (required)>'

Edit 1

@micahbf and @felix, I have a method that takes an input file with an instruction on each line for it to translate to its binary code. I just ran it on an input file with Li r2, 111001010001 repeated ~50 times (which I should have done before :/), and only the first output was correct--subsequent lines appended 1110010100010010 to the end like...

001000001110010100010010
0010000011100101000100001110010100010010
00100000111001010001000011100101000100001110010100010010
etc..

So...it's not rspec ;) I'll have to dig into my code further to figure out where the sticking is.

Edit 2

For those who wish to know (I would), in my code, I found a spot where I must have been returning the reference to an object instead of the value. This caused the original object to be modified (and my head to hurt and tests to fail!):

def self.get_reg_code reg
    @logger.debug { "Getting register code for [ #{reg.inspect} ]" }
    reg.upcase!
    if register? reg
      return CODES["#{reg}".to_sym] # <-- Returning object reference!
    else
      return reg
    end
end

I changed it to this:

def self.get_reg_code reg
    @logger.debug { "Getting register code for [ #{reg.inspect} ]" }
    reg.upcase!
    if register? reg
      reg_code = CODES["#{reg}".to_sym].clone # <-- Return a copy--not the referenced object!
      return reg_code
    else
      return reg
    end
end

Upvotes: 0

Views: 442

Answers (2)

Michael Durrant
Michael Durrant

Reputation: 96484

It is important to have a known state before tests. For this reason setup and teardown for each test should be done. There is also a general recommendation of "only 1 assert per test".

Based on this I would pursue moving the test into its own it...end block. You mentioned trying this but has an issue but didn't show the issue. I would pursue this route.

Upvotes: 0

Felix
Felix

Reputation: 4716

Sorry, it looks as if rspec is right and Parser is wrong. :)

Especially given with what it fails ("0010000011100101000100001110010100010010") it looks as if Parser has some state in which it stores the instructions.

To test this hypothesis, run the code a third time (but the second time without expect). You collected an argument for my state-hypothesis if the fail then reads "001000001110010100010000111001010001001000100001110010100010010".

Aother way to "test" this is to create a new parser instance for each call of build_instructions. Not to actually circumvent this problem, but to investigate it.

You also probably need something like

describe "#build_instruction"
  it "resets state" do
    expect(Parser.build_instruction("Li r2, 0000111001010001")).to eq("001000001110010100010010")
    expect(Parser.build_instruction("Li r2, 111001010001")).to     eq("001000001110010100010010")
  end
end

where you explicitly test the behaviour.

Upvotes: 1

Related Questions