From Erlang Community

Contents 1 Author 2 Article 3 Writing a test suite 3.1 Generate the skeleton 3.2 Running all the tests in the new suite 4 Download

Author

Rudolph van Graan

Article

For this you can use the erlang test server. It's home page is http://www.erlang.org/project/test_server/index.html. It is quite an involved process getting it to work, as you can find on this page.

In order to speed up the testing process and to fit in with a philosophy of test as you go, we created File:testhelper.erl. It makes it easy to build test suites that are compatible with the Test Server, but still allows you to run your tests in Emacs.

Writing a test suite

The basic steps are exactly the same as for Test Server. For details, look at Writing Test Suites

Here is the quick recipe:

1. Generate the skeleton in Emacs
2. Create a number of tests
3. Run the test suite using testhelper

Generate the skeleton

In Emacs select Erlang|Skeletons|Erlang test suite TS Frontend:

This will fill in your current module with a basic skeleton. If we remove all the comments, the test should look like this:

-module(simple_test_SUITE).

-compile(export_all).
-include("test_server/include/test_server.hrl").

init_per_suite(Config) ->
  Config.

end_per_suite(_Config) ->
  ok.

init_per_testcase(_TestCase, Config) ->
  Config.

end_per_testcase(_TestCase, _Config) ->
  ok.

all(doc) -> 
  ["This test links to all tests in this suite"];

all(suite) -> 
  [test_case].

test_case(doc) -> 
  ["Describe the main purpose of test case"];

test_case(suite) -> 
  [];

test_case(Config) when is_list(Config) -> 
  ok.

Next, compile the suite:

(gdb_rdbms@wyemac)1> c("simple_test_SUITE",[debug_info]).
{ok,simple_test_SUITE}

Running all the tests in the new suite

In order to run our new test suite, we simply call testhelper like this:

(gdb_rdbms@wyemac)2> testhelper:run(simple_test_SUITE,all).
****************************************************
Running simple_test_SUITE:init_per_suite (runtime version) Case all 
DataDir = "/Users/rvg/svn/modules/common/test/simple_test_SUITE_data/" 
Config = [{data_dir,"./simple_test_SUITE_data/"},
          {priv_dir,"./simple_test_SUITE_priv/"}]


====================Start of Test test_case======================
====================End of Test test_case========================
Running simple_test_SUITE:end_per_suite (runtime version)
****************************************************
1 tests passed
0 tests failed
ok

At this stage you'll notice that testhelper indicated 1 tests passed, 0 failed. The test that passed was our test_case(...) created above. Of course it doesn't do anything yet.

Let's make it fail. Change test_case(...) as follows:

test_case(doc) -> 
  ["Describe the main purpose of test case"];

test_case(suite) -> 
  [];

test_case(Config) when is_list(Config) -> 
  My1stAtom = the_new_atom,
  My2ndAtom = the_other_atom,
  My2ndAtom = My1stAtom,
  ok.

The test simply creates two atoms, one name My1stAtom, the other My2ndAtom. In the third line, we make an assumption that these two atoms are the same by trying to match the two variables. Lets test this. Run testhelper again:

(gdb_rdbms@wyemac)4> testhelper:run(simple_test_SUITE,all).
****************************************************
Running simple_test_SUITE:init_per_suite (runtime version) Case all 
====================Start of Test test_case======================
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
Testhelper catch error {badmatch,the_new_atom}
[{simple_test_SUITE,test_case,1},
 {timer,tc,3},
 {testhelper,run_test2,4},
 {testhelper,'-run_test/4-fun-0-',6},
 {proc_lib,init_p,3}]
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
====================End of Test test_case========================
Running simple_test_SUITE:end_per_suite (runtime version)
****************************************************
simple_test_SUITE:test_case                                                                         failed (84ms) (No Line) []
0 tests passed
1 tests failed
ok

This time, one test failed. You can see in the debug output that a bad match error occurred. The reason for this is that we tried to match two Atoms and they are just not created equal.

Unfortunately, we have to figure out where the error occurred (The third line of the third function clause), but in larger test suites, it becomes difficult to trace errors. Let us fix this. Change the code and recompile:

test_case(Config) when is_list(Config) -> 
  ?line My1stAtom = the_new_atom,
  ?line My2ndAtom = the_other_atom,
  ?line My2ndAtom = My1stAtom,
  ok.

The ?line macro is a mechanism that helps you find the error. Run testhelper again:

(gdb_rdbms@wyemac)6> testhelper:run(simple_test_SUITE,all).
****************************************************
====================Start of Test test_case======================
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
Testhelper catch error {badmatch,the_new_atom}
[{simple_test_SUITE,test_case,1},
 {testhelper,execute_test,3},
 {timer,tc,3},
 {testhelper,run_test2,4},
 {testhelper,'-run_test/4-fun-0-',6},
 {proc_lib,init_p,3}]
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! 
====================End of Test test_case========================
Running simple_test_SUITE:end_per_suite (runtime version)
****************************************************
simple_test_SUITE:test_case                                                                         failed (489us) Line 33 []
0 tests passed
1 tests failed
ok 

This time, testhelper tells us that test_case failed again, but on line 33 of the code. Much better.

Download

testhelper.zip