Why do test:units and test:functionals insist on running in development environment?

Question

I'm trying to test some environment-specific settings (middleware to be specific, but it doesn't seem to matter), but I'm having trouble getting the tests to run in the correct environment. The symptoms:

• If I run ruby test/unit/my_test.rb, all is fine because the first thing it does is require test/test_helper.rb, which sets the environment to "test" and then loads the Rails environment.
• If I run rake test, the first batch (functionals) run fine, but the second batch (units) fail. The failure is that ENV['RAILS_ENV] somehow gets unset between batches, then config/environment.rb sees that none is set and uses the default of "development". The environment is correct at the beginning of config/environment.rb and at the beginning of the configuration block in that file, but wrong by the end. I can tell by using a puts or by deleting config/development.rb causing it to not find the file.
• If I run rake test:units, I get the same problem as the second batch of rake test (that is, all fail)
• If I run rake test:functionals, I get the same as for rake test:units
• If I run rake test RAILS_ENV=test or rake test:units RAILS_ENV=test or rake test:functionals RAILS_ENV=test, all is fine!

One thing I tried that doesn't work at all is the following:

# in Rakefile:
task :set_test_env { ENV['RAILS_ENV'] = 'test' }
Rake::Task[:test].prerequisites.unshift :set_test_env
# and similarly for other test tasks

I even tried creating a separate one of those :set_test_env tasks for each test task so that I was sure it would get called before each one instead of just once. Still no dice. It does get called, and the environment is correct at the beginning of config/environment.rb but something goes wrong inside.

I have no calls to set ENV['RAILS_ENV'] in my code anywhere.

Answer 1

I pretty much always want to force my tests to run themselves and their prerequisites in the "test" environment, especially when ENV['RAILS_ENV'] is set to any of the common defaults (to avoid catastrophic accidents), but I also want to be able to run tests on, say, an environment named "v_2_0_maint_test" or something like that by calling rake test:units RAILS_ENV=v_2_0_maint_test on the command line.

So I have a test_tasks.rake file that prepends a prerequisite onto each of the test tasks that I'm interested in. Since this prerequisite is prepended, any other prerequisites (e.g. db:test:prepare, db:fixtures:load) run in the same environment. This claims the virtue of affecting only the tests you want to affect, and their prerequisites.

namespace :test do |n|
  [ n[:units], n[:functionals], n[:integration] ].each do |t|
    t.prerequisites.unshift(:set_test_env_dammit)
  end

  task :set_test_env_dammit do |t|
    if [ nil, "", "development", "staging", "production" ].index ENV['RAILS_ENV']
      RAILS_ENV = "test"
    end
  end
end

Answer 2

To give you a full answer I would have to take a look at the code, but I'll try to give you some clues that might be helpful.

First of all rake test and all the other variants (test:units, test:functionals etc.) work as follows

1. Rake process is invoked and task test is executed in the current environment (which is development by default), that's why development.rb is always read.
2. The Rake task invokes the test loader in a separate child process (you can verify this with ps or pstree), this is where the test_helper.rb is sourced and environment is set to test.

When you run ruby test/unit/my_test.rb the first step is skipped, so it looks like the problem lies there. Maybe you do something in development.rb that has side effects for the subprocess?

Answer 3

I do not actually believe that you have an issue here at all. I believe that at some time you spotted that your rake task was actually hitting the development environment and you started to try and work out why that was the case. You then added a line within the development config file to raise an exception and this is why your rake tests are failing.

If you remove the line that is raising an exception then you may find that the tests all run succesfuly.

If you try the following you may find that it explains the issue.

1. Set the environment to development (Just for purposes of the trace)

   export RAILS_ENV=development

2. Remove all lines that artificially raise exceptions in your environment files.

3. Add the following line to the end of each file in config/environments

   puts "**** In #{ENV['RAILS_ENV']} environment config ****"
   

4. Add the following line to the test/test_helper.rb just below the line that sets the environment to test.

   puts "**** Loading test helper **** Environment = #{ENV['RAILS_ENV']}"
   

5. Run rake using the trace option to watch the tasks execute with a trace of the currently active environment.

   rake -t test
   

6. Examine the output to determine where your environment is not set correctly.

7. Run the tests again but this time set the environment directly before running the tests using

   export RAILS_ENV=test

If you really do have an issue then perhaps you should post up the output of the rake -t with the tracing code in.

This is the output of my tests in a brand new project with some minimal testing in place. You will notice that before the actual tests are run the environment is always "test"

** Invoke test (first_time)
** Execute test
** Invoke test:units (first_time)
** Invoke db:test:prepare (first_time)
** Invoke db:abort_if_pending_migrations (first_time)
** Invoke environment (first_time)
** Execute environment
**** In development environment config ****
** Execute db:abort_if_pending_migrations
** Execute db:test:prepare
** Invoke db:test:load (first_time)
** Invoke db:test:purge (first_time)
** Invoke environment
** Execute db:test:purge
** Execute db:test:load
** Invoke db:schema:load (first_time)
** Invoke environment
** Execute db:schema:load
** Execute test:units
**** Loading test helper **** Environment = test
**** In test environment config ****
Loaded suite /Library/Ruby/Gems/1.8/gems/rake-0.8.3/lib/rake/rake_test_loader
Started
.
Finished in 0.071771 seconds.

1 tests, 1 assertions, 0 failures, 0 errors
** Invoke test:functionals (first_time)
** Invoke db:test:prepare
** Execute test:functionals
**** Loading test helper **** Environment = test
**** In test environment config ****
Loaded suite /Library/Ruby/Gems/1.8/gems/rake-0.8.3/lib/rake/rake_test_loader
Started
.....
Finished in 0.133776 seconds.

5 tests, 6 assertions, 0 failures, 0 errors
** Invoke test:integration (first_time)
** Invoke db:test:prepare
** Execute test:integration

Answer 4

Search through your entire project for RAILS_ENV and its variants. See if you set it somewhere in your application or in your tests.

Also, what platform are you running on? Can you run tests on another machine and see if the results are the same?

If this is a relatively recent development and you're using a RCS like Git or SVN, you should look through the recent commits, and if you're specifically using Git, you should look into git bisect. If you're not using an RCS, you should be.

If this is truly a fresh app problem, then it's probably a problem with your environment.

What plugins and gems do you have installed/configured? Can we see the backtrace?

Answer 5

At the top of the test_helper.rb file I have the code

ENV["RAILS_ENV"] = "test"

If you do not have that line then the system problem would run in the default environment (i.e. development).