• 2.0-specific Test Suite
  • Component Tests (Candidates for Automation if Not Already Covered)
    • Module Tests
    • Common UI Components
    • nomos license and copyright verification tests
    • CLI Tests
  • Basic Funtionality via the UI
    • Basic functionality
    • Web Site sanity checks
    • Basic FOSSology functionality based on the top menu items
  • RESTful UI test cases (in support of PTS requirement)
  • Install Tests
    • Source Installs
    • Package Installs
  • Cluster Testing
    • Stress Tests
  • Performance Testing
  • Concurrent User Testing
  • Migration Testing
• Post-Beta Testing to get to RC1
• Testing for RC2
• Testing for RC3

2.0-specific Test Suite¶

FOSSology test suite tracking for 2.0

Note for details on specific test cases, see Manual_Test_Cases

Component Tests (Candidates for Automation if Not Already Covered)¶

Module Tests¶

Module	Owner	Status/Notes
pkgagent	AUTO	unit tests: package agent unit tests
Scheduler	bob	#849 #873 #789 #561 #861
fossconfig	AUTO	unit tests: fossconfig unit tests
write 3 automated unit tests to test functions in libfossagent (fo_GetAgentKey, fo_WriteARS, fo_CreateARSTable)	bob
libfossdb	AUTO	unit tests: libfossdb unit tests
write automated unit tests to test functions in libfossrepo	bob
libfossscheduler	AUTO	unit tests: libfossscheduler unit tests
sqlCopy	AUTO	unit tests: slqCopy unit tests
adj2nest	bob
buckets	larry	done
nomosdiff	larry	done
bucketsdiff	larry	done #1581
tagging	larry	done #1389
copyright	mark	All cli parameters tested, waiting for Alex to verify latest accuracy numbers. phpunit tests written to capture cli test cases. copyright Functional Tests
delagent	AUTO	unit tests: delagent unit tests
mimetype	larry	done
nomos	mark	All cli parameters tested, phpunit tests written to capture test cases Nomos Functional Tests
ununpack	mark	#915 Issue is not critical, and can be solved by documentation. Larry helped port the 1.4.1 functional tests over to the 2.0 branch and Mark incorporated them into the nightly functional tests. All Tests pass. Unpack Functional Tests
wget_agent	paul	#879 #867

Common UI Components¶

Testing for..	Owner	Status/Notes
common-active.php	AUTO	http://fonightly.usa.hp.com/fossologyCodeDocs/d7/dc5/test__common__active_8php.html
common-agents.php	DUP	Already tested elsewhere (upload tests)
common-auth.php	AUTO	http://fonightly.usa.hp.com/fossologyCodeDocs/d8/d1d/test__common__auth_8php.html
common-cache.php	AUTO	http://fonightly.usa.hp.com/fossologyCodeDocs/d4/dee/test__common__cache_8php.html
common-cli.php	AUTO	http://fonightly.usa.hp.com/fossologyCodeDocs/dd/d9d/test__common__cli_8php.html
common-buckets.php	DUP	Already tested elsewhere (nomos diff, buckets)
common-compare.php	DUP	Already tested elsewhere (nomos diff)
common-db.php	DUP	Already tested elsewhere (various)
common-dir.php	AUTO	http://fonightly.usa.hp.com/fossologyCodeDocs/d1/d53/test__common__dir_8php.html
common-folders.php	DUP	Already tested elsewhere (various)
common-job.php	DUP	Already tested elsewhere (upload)
common-license-file.php	AUTO	http://fonightly.usa.hp.com/fossologyCodeDocs/d8/d06/test__common__license__file_8php.html
common-menu.php	AUTO	http://fonightly.usa.hp.com/fossologyCodeDocs/da/d65/test__common__menu_8php.html
common-parm.php	AUTO	http://fonightly.usa.hp.com/fossologyCodeDocs/df/d6e/test__common__parm_8php.html
commonphp	DUP	Already tested elsewhere (various)
common-pkg.php	AUTO	http://fonightly.usa.hp.com/fossologyCodeDocs/d5/d50/test__common__pkg_8php.html
common-plugin.php	DUP	Already tested elsewhere (various)
common-repo.php	DUP	Already tested elsewhere (various)
common-scheduler.php	DUP	Already tested elsewhere (various)
common-sysconfig.php	AUTO	http://fonightly.usa.hp.com/fossologyCodeDocs/d7/df3/test__common__sysconfig_8php.html
common-tags.php	DUP	Already tested elsewhere (tagging)
common-ui.php	DUP	Already tested elsewhere (various)
bootstrap.php	DUP	Already tested elsewhere (various)
libschema.php	DUP	Already tested elsewhere (various)

nomos license and copyright verification tests¶

Functionality to test	Owner	Status/Notes
nomos cli oneshots	mark	5 new functional tests for nomos: Nomos Functional Tests
verifyFoss23D1F1L.php	mark	This test and the tests below need to have an upload loaded first. There is no way to do that today without driving the ui. Or by rewriting the tests.
verifyFossDirsOnly.php	mark
verifyFossI16L335U29.php	mark

CLI Tests¶

Command Line utility to test	Owner	Status/Notes
cp2foss	larry
fossjobs	larry
fo_cli	paul
delagent	vincent
pkgagent	vincent
schema-export	vincent	#1155

Basic Funtionality via the UI¶

Basic functionality¶

Functionality to test	Owner	Status/Notes	Post-Beta test status (Larry)
Homepage	Larry	done	done cluster
Upload	Larry	done #861 #1365	done #1191 (not 2.0)
Jobs	Larry	done	done
Search	Larry	done	done
Browse	Larry	done	done
Organize	Larry	Done	done #1185 (not 2.0)
User Permissions	Larry	done User Permission function is not finished in my opinion, including ui and cli, firstly I need one user doc about this part, sencondly I do not think it is necessary to file all bugs about this part, I will be too much, execpt Admin almost each role has defects for now	done
Admin	Larry	done	done #1833

Please add tests as needed.

Web Site sanity checks¶

Functionality to test	Owner	Status/Notes
AboutMenuTest.php	DUP	Already tested in UI test cases
login.php	DUP	Already tested in UI test cases
OrgFoldersMenuTest-Create.php	DUP	Already tested in UI test cases
OrgFoldersMenuTest-Delete.php	DUP	Already tested in UI test cases
OrgFoldersMenuTest-Edit.php	DUP	Already tested in UI test cases
OrgFoldersMenuTest-Move.php	DUP	Already tested in UI test cases
OrgUploadsMenuTest-Delete.php	DUP	Already tested in UI test cases
OrgUploadsMenuTest-Move.php	DUP	Already tested in UI test cases
SearchMenuTest.php	DUP	Already tested in UI test cases
UploadFileMenuTest.php	DUP	Already tested in UI test cases
UploadInstructMenuTest.php	DUP	Already tested in UI test cases
UploadOne-ShotMenuTest.php	DUP	Already tested in UI test cases
UploadServerMenuTest.php	DUP	Already tested in UI test cases
UploadUrlMenuTest.php	DUP	Already tested in UI test cases

Basic FOSSology functionality based on the top menu items¶

Functionality to test	Owner	Status/Notes
CreateFolderTest.php	DUP	Already tested in UI test cases
DeleteFolderTest.php	DUP	Already tested in UI test cases
editFolderDescriptionOnlyTest.php	DUP	Already tested in UI test cases
editFolderNameOnlyTest.php	DUP	Already tested in UI test cases
editFolderTest.php	DUP	Already tested in UI test cases
moveFolderTest.php	DUP	Already tested in UI test cases
DupFolderTest.php	DUP	Already tested in UI test cases
moveUploadTest.php	DUP	Already tested in UI test cases
can create duplicate upload	DUP	Already tested in UI test cases
upload properties	DUP	Already tested in UI test cases
upload removal	DUP	Already tested in UI test cases
UploadFileTest.php	DUP	Already tested in UI test cases
UploadUrlTest.php	DUP	Already tested in UI test cases
uploadSrvFileTest.php	DUP	Already tested in UI test cases
UploadSrvArchiveTest.php	DUP	Already tested in UI test cases
uploadSrvDirTest.php	DUP	Already tested in UI test cases
addUserTest.php	DUP	Already tested in UI test cases
dupUserTest.php	DUP	Already tested in UI test cases
emailUserTest.php	larry	done #849 **
noEmailUserTest.php	larry	done
uploadWEmail.php	larry	done
uploadWoutEmail.php	larry	done
Browse tests	DUP	Already tested in UI test cases
Schedule Agents tests	DUP	Already tested in UI test cases
Admin tests	DUP	Already tested in UI test cases

RESTful UI test cases (in support of PTS requirement)¶

Test	Owner	Status/Notes
One-Shot nomos license test for PTS support	bob

Install Tests¶

Install tests take two forms, source installs and package installs. Source installs are just that, the sources are checked out, utils/fo-installdeps is run, fossology is made, installed and fo-postinstall run. Tests are then run on the installed system.

Source Installs¶

Functionality to test	Owner	Status/Notes
Source Install	vincent	DONE
fo-installdeps	vincent	DONE #915

For details on testing a source install in a cluster, see Cluster Testing below.

Package Installs¶

The package installs are automated by a php script. The script expects to be run on a Virtual Machine that was set up for package install testing. See Package Install Testing for details.

Functionality to test	Owner	Status/Notes
test package installs	mark	Jenkins job is set up, will turn it on when packages are made/ready
Install Packages on Test Systems	mark	will install packages on 64 and 32 bit machines and stress test
Upgrade testing 1.4.1 -> 2.0	mark

Functionality to test	Owner	Status/Notes
verify agent.agent_rev gets bumped when we install a new agent	Larry	need an agent with a new version

Cluster Testing¶

Cluster Testing

Stress Tests¶

Performance and Load Testing

Performance Testing¶

Performance and Load Testing

Concurrent User Testing¶

A place holder

Migration Testing¶

Migration Testing

Post-Beta Testing to get to RC1¶

• All packages install successfully (Vincent & Mark) http://fossbuild.usa.hp.com/fossology/2.0.0/testing/
• test & close all 2.0 bugs (issue Authors): http://www.fossology.org/projects/fossology/issues?query_id=9
  Mary: #1653 #1515 #1467 #1641 #1665 #21 #1683
  Alex:
  Bob: #1035 #1077 #561 #1053 #999 #837
  Vincent: #1047 #855
  Larry #1605 #1299 #1389 #1401 #291 #807 #1581 #1455 #921 #1389
  Mark: #1545 #1101 #1539 #1323 #915 #969 #1257 #1263 #1125 #1131 #1587 #1233 #1239 #1329 #1575 #897 #1113 #1119 #1137 #1143 #1281 #1359 #1431 #1413 #957 #1275 #1689

• perform regression tests
  http://www.fossology.org/projects/fossology/wiki/20-specific_Test_Suite#Basic-functionality (Larry done on ubuntu 1110/64 and debian 6.0/64, rhel 6.0/64, rhel 6.0/32 till April 17, one cluster(rhel 6.0/64 and rhel 6.0/32) )
  http://www.fossology.org/projects/fossology/wiki/20-specific_Test_Suite#Functionality (Vincent)
  http://www.fossology.org/projects/fossology/wiki/20-specific_Test_Suite#Cluster-Ops (Bob)
  http://www.fossology.org/projects/fossology/wiki/20-specific_Test_Suite#Stress-Tests (Mary)

Testing for RC2¶

• test & close all "High" priority 2.0 bugs: http://www.fossology.org/projects/fossology/issues?query_id=9
  Bob: please test #777 #1857, #1917, #1875, #1887, #1899, #1911, #1923, #1965
  Mary: test #1965 (on fo.usa)

• perform scheduler regression tests
  http://www.fossology.org/projects/fossology/wiki/Scheduler_Test_Cases (whoever can get to it first: Mary, Vincent or Larry vincent and larry have done it on single system, cluster test is needed)

Testing for RC3¶

• perform regression tests to exercise code changes made post-RC2
1. Test substitution of "/" with "`" in ufile_name using this file as a test case: http://newbobg.ostt/~bobg/testfiles/testfiles.tar, fixed in branches/fossology2.0/fossology/src/ununpack/agent/utils.c, Completed by Larry
   http://fossology.svn.sourceforge.net/viewvc/fossology/branches/fossology2.0/fossology/src/ununpack/agent/utils.c?view=markup&pathrev=5877 &
   http://fossology.svn.sourceforge.net/viewvc/fossology/branches/fossology2.0/fossology/src/ununpack/agent/utils.c?view=markup&pathrev=5878 &
   http://fossology.svn.sourceforge.net/viewvc/fossology/branches/fossology2.0/fossology/src/ununpack/agent/utils.c?view=markup&pathrev=5879
2. Test memory leak fix in _RepGetHost using valgrind (Eg. valgrind --tool=memcheck --leak-check=yes ./nomos myfile.c) fixed in branches/fossology2.0/fossology/src/lib/c/libfossrepo.c, fossconfig.c & libfossscheduler.c Completed by Larry
   http://fossology.svn.sourceforge.net/viewvc/fossology/branches/fossology2.0/fossology/src/lib/c/libfossrepo.c?view=markup&pathrev=5876 &
   http://fossology.svn.sourceforge.net/viewvc/fossology/branches/fossology2.0/fossology/src/lib/c/fossconfig.c?view=markup&pathrev=5875 &
   http://fossology.svn.sourceforge.net/viewvc/fossology/branches/fossology2.0/fossology/src/lib/c/libfossscheduler.c?view=markup&pathrev=5875
3. Agent regression tests
   adj2nest (Mary) DONE testfiles.tar, RedHat.tar & all.zip passed
   buckets (Mary) DONE all.zip & RedHat.tar passed
   copyright (Mary) DONE all.zip & LTS_dev_acpi.tar.gz passed
   delagent (Mary) DONE testfiles.tar passed
   mimetype (Mary) DONE testfiles.tar, RedHat.tar & all.zip passed
   nomos (Mary) DONE testfiles.tar, RedHat.tar & all.zip passed
   pkgagent (Mary) DONE python-devel-2.6.0-8.6.x86_64.rpm
   ununpack (Mary) DONE testfiles.tar, RedHat.tar passed
   wget_agent (Mary) DONE testfiles.tar via upload_url passed; RedHat.tar via upload_url passed; all.zip via upload_file passed
   ubuntu & rhel distros ran to completion on fo.usa (running RC1 w/all RC3 changes patched in by Bob)
4. Check to see garbage in email notification has been corrected (http://www.fossology.org/issues/1875) Deferred to 2.0.1 http://www.fossology.org/issues/1989
5. Verify the scheduler will accept and use "-c sysconfdir" passed on the command line (http://www.fossology.org/issues/2007)
   larry's test notes: For now, the scheduler can use the new sysconf to schedule agents, but the agents cannot finish successfully, please refer to http://www.fossology.org/issues/2025, so for now, I cannot say if issue #2007 is fixed or not.

• Test the package upgrade path from RC2 to RC3 (for fossology.usa upgrade) (Mary) DONE