Functional Tests¶

The next level up from unit tests are functional tests. Functional tests ensure that a set of code units work together for a desired outcome. They examine a specific feature/function of the software, typically by testing APIs and the user interface, to ensure requirements are met.

Functional tests can be written in any language, but usually, the language used matches the languages the application is written in. Functional tests should use some type of test framework to help with test automation and reporting.

There is no hard and fast rule as to how many functional test cases to write. Usually functional test cases are derived from user requirements, stories, or external documentation. The challenge in testing FOSSology is to understand how the system is supposed to work and then design tests to prove/disprove that it works that way.

At a minimum, functional tests should test:

• all valid inputs in the ranges supported.
• invalid inputs should be supplied, and the code should error or die gracefully.
• all APIs (if they exist)
• all cli options. For example, if a program has a cli interface like

foo -h [-d dir] [-c comment] [-x] -f <file>

Then every option should be tested for valid values, and then every option should be tested for invalid values.
For example:

• foo <no parameters> (expect help message)
• foo -h (expect help message)
• foo -d <invalid dir> (should produce an error)
• foo -f <no file> (should produce an error)
• foo -c <no comment> (should produce an error)

etc...

Keep your test cases small, and have them test only one thing (if possible). There should always be more than one test for any given module.