This task may be suitable for GSoC: testing framework

A collection of thoughts with respect to unit testing.

We definitely want to add unit test suites to our code base.

We should select a tool that we like to use, and that is supported (not abandoned).

• SC Test

• DejaGnu / Expect

  • used by the GCC testsuite, GDB testsuite, binutils testsuite, etc.

• The glibc testsuite has a home-grown system (Makefile-based), likewise does the Open POSIX Test Suite.

• Kyua (and its predecessor ATF).

  • Primarily used by NetBSD as its testing framework; FreeBSD is in the process of adopting it.

  • Provides bindings to write tests in C, C++ and POSIX shell. Lua is planned.

  • Builds and runs on many different Unix-based operating systems.

• check

  • used by some GNU packages, for example GNU PDF (Jose E. Marchesi)

• CodeSourcery's QMTest

  • useb by?

  • documentation:

    • http://www.codesourcery.com/public/qmtest/whitepaper.pdf

    • http://www.python.org/workshops/2002-02/papers/01/index.htm

    • http://gcc.gnu.org/ml/gcc/2002-05/msg01978.html

    • http://www.codesourcery.com/public/qmtest/qmtest-snapshot/share/doc/qmtest/html/tutorial/index.html

    • http://www.codesourcery.com/public/qmtest/qmtest-snapshot/share/doc/qmtest/html/manual/index.html

• Git has an elaborate unit testsuite, which is also used in Notmuch.

• [ANNOUNCE] ktest.pl: Easy and flexible testing script for Linux Kernel Developers by Steven Rostedt, 2010-10-28. v2, 2010-11-08.

• http://autotest.kernel.org/wiki/WhitePaper

Related

• nightly builds

  • nightly builds deb packages

• http://www.phoronix-test-suite.com/ -- ``comprehensive testing and benchmarking platform''. This one might be useful for performance testing, too?

• http://ltp.sourceforge.net/

• LaBrea, or similar tools can be used for modelling certain aspects of system behavior (long response times, for example).

Discussion

See the GSoC project idea's discussion subpage.