Advancing open source analysis and development
|
FOSSology Advancing open source analysis and development |
This shows you the differences between the selected revision and the current version of the page.
| how_to_write_php_tests 2010/07/22 15:30 | how_to_write_php_tests 2010/07/22 15:49 current | ||
|---|---|---|---|
| Line 96: | Line 96: | ||
| For a discussion of how to interpret test output and failure please see | For a discussion of how to interpret test output and failure please see | ||
| [[How to configure and run UI Tests]]. | [[How to configure and run UI Tests]]. | ||
| - | |||
| - | ===== Creating Test Plans and Test Cases ===== | ||
| - | |||
| - | Before you can write a test, it's a good idea to have a test case in mind that needs testing. Everyone on this team already does this when they test informally, they just might not think of it that way. For example, I'm going to run X, and I expect Y from the data that it will process. That's a test case. Of course test cases can get much more complex than that. | ||
| - | |||
| - | Test cases are usually written down, either informally or formally. Writing them down in a simple text test plan is usually good enough. Wikis are great places to store test plans. There are several advantages to having your test cases written down. The first is the it provides a record of what you plan to test. Hopefully all the documented test cases will get implemented, but if not there is a record of them so that when time allows they can be implemented. The second advantage is that when a defect occurs after your code passes all the tests, the test plan provides an easy way to see examine why didn't the tests catch the defect and provides a place to document new test cases added to fix defects. The third advantage is that a test plan allows one to look at all the test cases and pose the question what else needs to be tested? This is hard for most humans to do if there are more than about 3-7 items to consider and we have to hold them in our head. I'm sure you can think of other great reasons why it makes sense to have a written test plan. (A test plan in a wiki is a written test plan). | ||
| - | |||
| - | ==== Where do test cases come from? ==== | ||
| - | |||
| - | The sources that enable test case creation come from user requirements, External reference documents (man pages, api specs, etc.), internal specifications and defect reports. So for example if the man page says the program always creates a log file then one can test if that is true. You can read code to help generate test cases, but that is not very efficient. | ||
| - | |||
| - | This sounds very //old school// but it works. Write the code, write the man page, write the tests, run tests/add test till all tests pass and user requirements are met. Release! Even Xtreme and Agile methods use this, sometimes with a different order. | ||
| - | |||
| - | All methods assume that the user requirements are understood and will be understood before final code. Rapid prototyping for example allows one to learn requirements as one goes but still starts with an initial set of requirements. The point here is that if the requirements are captured in some way it allows the programmer or tester to use the requirements to generate test cases. | ||
| - | |||
| - | If Selenium is used to web test, it is very useful to only do one test case for playback. This helps keep the tests from being too brittle and one test failure has a smaller chance of cause a failure down stream. | ||
| - | |||
| - | ===== Test Creation Best Practices ===== | ||
| - | |||
| - | Definition: a //brittle// test is a test that breaks easily due to environment changes or dependence on prior tests or test results. | ||
| - | |||
| - | * each test stands alone (if possible). This means it does not depend on a prior test to set something up or leave a condition around for the test to use. Each test should try to set up what it needs to test and then remove it when done. | ||
| - | * setup/tear down is fine, try to minimize. A test should always try to clean up after itself. Or create a cleanup script that cleans up the testing cruft as the last step. | ||
| - | * keep tests small. Don't try to test too much at once. This keeps you sane. Also helps with debugging when a failure occurs. | ||
| - | |||
| - | ===== How Many Test Cases? ===== | ||
| - | |||
| - | There is no hard and fast rule as to how many test cases one should write. Here are some basic guidelines: | ||
| - | |||
| - | * For any type of testing, if a defect is found by ad hoc testing (i.e. while not running your tests), write a test case for that situation. | ||
| - | * For Functional tests | ||
| - | * have the user requirements been met? | ||
| - | * are all options in the external specs tested? Why not? (Maybe some are too hard or too expensive to test). | ||
| - | * have destructive tests been written (try to do something the application says you cannot do or test a limit, e.g. no more than 10 files for input, so give it 11, then give it 0, what happens?) | ||
| - | * has the agreed upon Code Coverage metric been met? | ||
| - | * has the defect per lines of code metric been met? For example, there will be no more than 1 defect per 1000 lines of code. | ||
| - | * For Unit tests | ||
| - | * have all function parameters been tested and verified? | ||
| - | * have all function return values been tested and verified? | ||
| - | * any special cases identified and tested (file system full, db is gone, etc...) | ||
| ==== FOSSology Test Areas ==== | ==== FOSSology Test Areas ==== | ||
