So you want to easily test the code your writing? The following recipes cover how to write automated tests. We will use an established and well-designed regression testing framework called 'RT'. In the last recipe we will discuss other options.
You can learn more about Dr. Richard C. Waters' RT on CLiki. We will get up and running quickly, by doing a standard asdf-install which will pull down Kevin Rosenburg's packaged version. Note: sbcl users can simply use sb-rt which is prebundled with their system.
;; Install rt from cliki (asdf-install:install :rt)
This command downloads and installs the RT package. This step only needs to be done once. If you just did the install then you can skip the next step.
(asdf:operate 'asdf:load-op :rt)
This line loads the RT package into your Lisp image.
Normally we would be writing tests against our own code, but for clarity let's write some tests against operations from the String recipes sections.
; using *my-string* from the substrings recipe "Groucho Marx" (rt:deftest test-subseq-one-index (subseq *my-string* 8) "Marx") TEST-SUBSEQ-ONE-INDEX
That substring recipe told us that we could expect "Marx" if we ask for substring to 8.
We use the deftest macro, naming our test test-subseq-one-index. In the body of the test
we specify the code we want to test, followed by the expected results. The code
(subseq *my-string* 8)is evaluated
and the number of values return must match the number of expected results. In this example
their is only one value returned and it matches the one expected value "Marx".
For each actual, expected pair, they must be EQUAL. Note that the REPL prints
the test name and under the covers this test is added to the current test suite.
We have defined a test, let's run it.
CL-USER> (rt:do-test 'test-subseq-one-index) TEST-SUBSEQ-ONE-INDEX
We have RT test our assertion for how SUBSEQ should behave.
The test passes, so no errors are reported. do-test simply returns the name of the test. In the next example
we will see how RT behaves with failing tests.
Tests can be run one at a time with DO-TEST as you incrementally build
up your system, or they can be run all at once with DO-TESTS. For our second test, let's define
and run a failing test.
CL-USER> (rt:deftest test-subseq-two-indicies (subseq *my-string* 0 7) "Xroucho") TEST-SUBSEQ-TWO-INDICIES CL-USER> (rt:do-test 'test-subseq-two-indicies) Test TEST-SUBSEQ-TWO-INDICIES failed Form: (SUBSEQ *MY-STRING* 0 7) Expected value: "Xroucho" Actual value: "Groucho". NIL
We purposely provide an expected value which is wrong, so that the test will fail. Notice
RT describes a failed test to standard output. Also the DO-TEST function
returns NIL instead of the test as before.
The great value of test code is running it automatically and frequently to find
issues early in development. RT provides the DO-TESTS function to
regress all of the tests.
CL-USER> (rt:do-tests) Doing 2 pending tests of 2 tests total. TEST-SUBSEQ-ONE-INDEX Test TEST-SUBSEQ-TWO-INDICIES failed Form: (SUBSEQ *MY-STRING* 0 7) Expected value: "Xroucho" Actual value: "Groucho". 1 out of 2 total tests failed: TEST-SUBSEQ-TWO-INDICIES. NIL
Let's redefine test-subseq-two-indicies with a passing expected value.
Additionally let's define a third test.
CL-USER> (rt:deftest test-subseq-two-indicies (subseq *my-string* 0 7) "Groucho") WARNING: Redefining test TEST-SUBSEQ-TWO-INDICIES TEST-SUBSEQ-TWO-INDICIES (rt:deftest test-concatenate-three-strings (concatenate 'string "Karl" " " "Marx") "Karl Marx") TEST-CONCATENATE-THREE-STRINGS CL-USER> (rt:do-tests) Doing 3 pending tests of 3 tests total. TEST-SUBSEQ-ONE-INDEX TEST-SUBSEQ-TWO-INDICIES TEST-CONCATENATE-THREE-STRINGS No tests failed. T
Note that DO-TESTS picks up our third test test-concatenate-three-strings
because it runs the entire test suite and prints a report to standard out that no tests have failed.
The value T is returned to signal a successful test run.
If we pretend for a moment that SUBSEQ and CONCATENATE
are Lisp operations which we have written, you can see how easy it is to build
up a collection of tests to exercise and regress code as we write and maintain it.
If you like using RT so far, you will want to read up on the documentation which comes with it, or read the original paper from 1991 which covers RT and COVER (a code coverage framework). At a minimum you will want to add
(rem-all-tests)to the beginning of your test code for the current project. You can use
CONTINUE-TESTING to run all of the tests which have
either failed, or are newly defined.
Test Driven Development, xUnit testing frameworks, Automated Acceptance Tests, etc are quite the rage in some areas of contemporary programming. If RT doesn't fit your needs, or you would like to see what else is out there, please see the following: