Reason: undef while running common tests in erl console [Erlang]

Question

I have running application and in the interactive console I try to run common test suites located in test/common directory:

ct:run("test/common").

But I get a bunch of errors:

Reason: undef

and all tests fail. I tried to run them from linux shell

ct_run -dir test/common

or like this:

ct_run -boot start_sasl -spec test/common/app_ct.spec -erl_args -config env/dev.config

with the same result.

But when I run them using rebar (the second version, not rebar3)

rebar ct

Everything works, tests pass. But it takes too much time to compile the app and to start it.

In the rebar.config I have:

{ct_dir,"test/common"}.
{ct_log_dir,"test/logs"}.
{ct_extra_params,"-boot start_sasl -pa deps/*/ebin -pa ebin -spec test/common/app_ct.spec -erl_args -config env/dev.config"}.

and in the test/common/app_ct.spec I have:

{verbosity, 100}.
{init, {eval, [{application, ensure_all_started, [app]}, {timer, sleep, [30000]}]}}.
{alias, common, "./test/common/"}.
{suites, "", [app_srv_SUITE, app_client_SUITE]}.

What can I do to run tests from erl console using ct:run("test/common")?

My goal is to be able to recompile single test files individually and run tests from working application console without stopping and recompiling all application.

I recompile singular test module without problems like this:

c("test/common/new_mod_SUITE.erl", [{i, "include"}, {i, "deps"}, {outdir, "test/common"}]). 

But I still can't run tests after that.

Answer 1

This probably won't help, but here is what happens for me using rebar3:

~/erlang_programs/myrebar/myapp$ ls
LICENSE     _build      rebar.lock  test
README.md   rebar.config    src

~/erlang_programs/myrebar/myapp$ cd src
~/erlang_programs/myrebar/myapp/src$ ls
a.erl       myapp_app.erl   rebar.lock
myapp.app.src   myapp_sup.erl

~/erlang_programs/myrebar/myapp/src$ cat a.erl
-module(a).
-compile(export_all).
%%-include("eunit.hrl").


hello() -> io:format("hello").

~/erlang_programs/myrebar/myapp/src$ cd ../test
~/erlang_programs/myrebar/myapp/test$ ls
a_SUITE.erl

~/erlang_programs/myrebar/myapp/test$ cat a_SUITE.erl 
-module(a_SUITE).
-compile(export_all).

all() -> [go].

go(_Config) ->
   1 = 1. 

~/erlang_programs/myrebar/myapp/test$ cd ..
~/erlang_programs/myrebar/myapp$ rebar3 compile
===> Verifying dependencies...
===> Compiling myapp
~/erlang_programs/myrebar/myapp$ rebar3 shell
===> Verifying dependencies...
===> Compiling myapp
Erlang/OTP 20 [erts-9.3] [source] [64-bit] [smp:4:4] [ds:4:4:10] [async-threads:1] [hipe] [kernel-poll:false]

Eshell V9.3  (abort with ^G)
1> ===> The rebar3 shell is a development tool; to deploy applications in production, consider using releases (http://www.rebar3.org/docs/releases)
===> Booted myapp
 
1> ct:run("test").

Common Test: Running make in test directories...
Recompile: a_SUITE
a_SUITE.erl:2: Warning: export_all flag enabled - all functions will be exported

CWD set to: "/Users/7stud/erlang_programs/myrebar/myapp/ct_run.nonode@nohost.2021-02-14_14.26.08"

TEST INFO: 1 test(s), 1 case(s) in 1 suite(s)

Testing myrebar.myapp: Starting test, 1 test cases
Testing myrebar.myapp: TEST COMPLETE, 1 ok, 0 failed of 1 test cases

Updating /Users/7stud/erlang_programs/myrebar/myapp/index.html ... done
Updating /Users/7stud/erlang_programs/myrebar/myapp/all_runs.html ... done
{1,0,{0,0}}
2> 

Then after exiting the shell:

~/erlang_programs/myrebar/myapp$ 

~/erlang_programs/myrebar/myapp$ rebar3 ct
===> Verifying dependencies...
===> Compiling myapp
test/a_SUITE.erl:2: Warning: export_all flag enabled - all functions will be exported

===> Running Common Test suites...
%%% a_SUITE: .
All 1 tests passed.

~/erlang_programs/myrebar/myapp$ 

I didn't touch rebar.config, it is the default produced by rebar3:

{erl_opts, [debug_info]}.
{deps, [{eleveldb, "2.2.20"}]
}.

{shell, [
  % {config, "config/sys.config"},
    {apps, [myapp]}
]}.

I would try printing your current working directory when you are in the shell:

2> pwd().
/Users/7stud/erlang_programs/myrebar/myapp
ok

Then try using either a full path or a relative path from that directory, e.g.:

"./test/common"
"Users/../../test/common"

Then, I would move all your *_SUITE.erl files except one into a directory outside your app, and just work with one test file. Then, I would get rid of all that config file stuff and try again.

Answer 2

First of all, need to be sure that all tests is compiled and they are inside of folder what is put as argument into ct:run/1. If need to run only one test from specific folder, can be used ct:run/2. If need to run specific test case from specific folder, can be used specific folder ct:run/3. Examples:

1> ct:run("test/common").
2> ct:run("test/common", "some_SUITE").

However I would recommend using rebar3, in rebar3 was added option like --dir and tests from specific folder can be run like:

$ ./rebar3 ct --dir="test/common"