Perform automated package install testing
|Assignee:||Dong Ma||% Done:||
Updated by Paul Holland about 1 year ago
- Category set to Automation
- Assignee set to Dan Stangel
- Estimate set to 5
Updated by Dan Stangel 11 months ago
- Status changed from New to In Progress
The nightly package builds are now running (as of 7-25-2012) and generating packages every day. However we are not running the package install testing (the title of this backlog item), so I will not mark it done yet.
Updated by Dan Stangel 11 months ago
- IterNum changed from 5 to 6
Bump to iteration 6.
Work with Vincent to include the newly created VMs.
Updated by Paul Holland 10 months ago
- IterNum changed from 6 to 7
Updating to iteration 7 to be addressed.
Updated by Paul Holland 9 months ago
- Estimate changed from 13 to 6
Updating the estimate since Dan reported earlier that 7/13 of the points were completed.
Updated by Dan Stangel 9 months ago
- Target version deleted (
This is not a 2.1.0 release target. Manual package testing is being done for 2.1.
Removing target version as I don't think this has been set yet.
Updated by Bob Gobeille 8 months ago
- Target version set to 2.2.0
Dan, do you want to transfer this over to Vincent to free up some of your time?
Updated by Mary Laser 8 months ago
- Assignee changed from Dan Stangel to Dong Ma
Reassigning to Vincent as Dan will not be working on this release. Dan & Vincent should schedule time to knowledge share about this task.
Updated by Dan Stangel 8 months ago
Adding some notes about what needs to be done to complete this backlog item:
The FOSSology continuous integration system (Jenkins) should automatically install the nightly build packages every day, and validate that they installed without errors.
We already have a nightly automated job in Jenkins to generate new Linux distro packages from top of tree. This job can be seen here (Sorry, internal link only): http://fotest.fc.hp.com:8080/job/Fossology_Package_Build/ (The only thing this job does is execute the script
When this script completes, it copies all of the new packages into a web-accessible location at (Sorry, internal link only):
(Note: We should probably change the name of these to something other than "2.0.1" to avoid any confusion, such as "2.2.0~testing")
What needs to happen is we need another automated Jenkins job that is run after the packages have been built, to execute a package install for each of the Linux distro platforms.
We already have VMware virtual machines set up for package install testing. These are available (sorry, only project internal at the moment) at:
These VMs should be used to test the packages. ALTERNATIVELY, if it is easier/faster to do, please feel free to create KVM VM's or any other mechanism that you find easy and simple to set up and administer. The VM platform really does not matter, as long as the VMs are available and can be used for testing.
The idea of the nightly package install test is something like this:
1. Create a VM, install with the Linux distro of interest (e.g. Ubuntu 12.10 x86_64)
2. Once the base OS has been installed and configured to work in the environment, but BEFORE any other actions are taken on the VM, create a snapshot of the Initial OS Install. This snapshot will be used at the start of every test.
3. Before beginning the FOSSology package install test, revert the VM to this Initial OS Install snapshot.
4. Use the native distro package manager tool (i.e. 'yum' for Fedora/CentOS, 'apt' for Debian/Ubuntu) to point to the nightly package build repository. Install the package using the native package manager (i.e., 'sudo apt-get install fossology', or 'sudo yum install fossology').
5. Just scan the console output for any sign of problems, such as "ERROR" or "FAILURE". the native Jenkins console output parser should be sufficient for this task, at least initially. Later on we can go back to add more sophisticated error checking, and we can add additional post-installation tests. But for now, we just need to check that the package appears to install without any errors.
6. That's it. Do this for every flavor of distro that we generate packages for.
We already had a package install test automation job set up for the 2.0.0 release a long time ago. The tool is still around, and it may be useful to modify and update the existing scripts rather than start fresh. (Or it may not be useful...). The existing package install automation job can be seen here (sorry, internal link only):
Note: This is a Jenkins "Matrix job" which should run the test on several Jenkins slave agents simultaneously. This job executes the script
(and the associated files in trunk/fossology/src/testing/install)
If we plan to use this script, we will need to make extensive changes to it to account for new package locations, new VM hostnames, etc., etc.. but it may be a useful starting point.
- Status changed from In Progress to Resolved
Finished automated packaging install testing.
The package install testing runs everyday after package build finished.
Testing run list:
Testing job here:
Issue: the fedora system repository not stable, yum install sometime success sometime fail.
- Status changed from Resolved to Feedback
how to confirm this issues is fixed or not, I did not received any mail from firstname.lastname@example.org
Check from here: http://fotest.fc.hp.com:8080/job/Fossology_Package_Install/
For now I didn't open the email notification.
- Status changed from Feedback to Resolved
Open email notification, the fedora distro url also not stable, sometime success sometime fail,there is one success job : http://fotest.fc.hp.com:8080/job/Fossology_Package_Install/43/ marked as resolved.
- Status changed from Resolved to Closed
sometimes, package test work successfully. sometimes test fails in some nodes, the reason is because the proxy is unstable. so from my point of view, wen can close this issue.
if package testing fail in the future, we should look into/fix it ASAP.
if you do not agree with me, please re-open this issue.