« Previous -
Version 32/95
(diff) -
Next » -
Current version
Dan Stangel, 05/29/2012 03:50 pm
update one of dan's tasks
This is the FOSSology backlog, or list of all desired but unimplemented features that need to be prioritized and developed over time. It is dynamic and will be updated frequently.
| Ongoing Team Activities | |
|---|---|
| Task | Notes |
| Keep build running successfully | |
| Community support - Users (emails, help installations, etc) - Community presentations (Bruno, Martin, etc) - Fossology.org (process, ownership for interfacing with OSU/OSL) - Community development (incorporating contributors, bringing people in, opportunities on website, etc. |
|
| Performance improvements (repository, delagent, …) | |
| Current Iteration (May 22-June 4, 2012) | |||
|---|---|---|---|
| Task | Owner | Estimate | Notes |
| Fix ununpack failure on fossology.usa | Bob | 1 | |
| Fix unit test failures | Vincent, Larry | 1.5 | larry: 1 day, fixed unit test failures; 0.5 day, fixed bug #2043(wait for close) vincent: verified bug#2043 closed |
| Tag RC3 and create packages | Vincent/Dan | 1 | Done. Re-tag RC3 and new packages created. |
| Test RC1+ to RC3 upgrade path | Mary | 2 | |
| Test RC3 -- see the RC3 list (including regression testing the agents) | Mary/Vincent/Larry | 13 | Larry: found some issues, re-test on RC3, No problems on a single rhel system |
| Perform automated package install testing | Dan | 5 | |
| Update fossology.usa with new RC and update disks - Done means: fossology.usa is upgraded and running with RC3. |
Mary | 1 | |
| Move 2.0 branch to trunk - There is no per file merging. The 2.0 branch completely supersedes the trunk |
Dan | 1 | - First create a tag of the trunk prior to doing anything else, i.e. 1.4.something.post.whatever - then move the branches/fossology2.0 contents over to trunk - finally create a 2.0 branch (?) to track any future changes to 2.0 |
| Complete analysis of available tools for FOSS Discovery (OLEX, Sonatype, Antelink, Daniel German’s tool) - Deliverable will be a side-by-side comparision of o JOA as it stands today o OLEX as it stands today o Sonatype as it stands today |
Bob | 9 | - Daniel German using tool (JOA) on HP data; fixing math errors as they are found - Looking beyond Java to other languages but hit JOA memory limit. Need to refactor JOA. - bat tool from Armijn also looks good but has the same limitation as DG's JOA - corpus currently limited to debian. - Antelink public API announced 5/21/2012. Have not had a chance to look at it yet. - NextB Dejacode investigation - no OSS discovery, just license admin. - OpenLogic OSS discovery crashed on sample data. - We still have an OLEX server but I've not submitted code to it yet due to learning curve. - Still following up with Sonatype |
| Finish transition with Mark - Build/run unit tests on every commit (concise, useful feedback) - Functional testing tbd - Daily package testing |
Dan | 13 | |
| Check in a set of usable test agents for the Scheduler - test agents will be checked in - scheduler will be working against test agents in sandbox (until we can check scheduler changes in) |
Alex | 13 | |
| Add more automated tests (package/install with Dan) | Vincent | 5 | Need to work with Dan to understand how automated package install testing works. |
| Add more automated tests | Larry | 5 | 1. To add/modify tests for cp2foss and fossjobs, will check in later(did not check in, difficult to test cp2foss and fossjobs); 2. added tests for wget_agent(checked in) |
| Create plan for lab equipment move - create a high-level plan for when/how/what - does not include detailed plans yet |
Dan | 5 | |
| Move Alex to FO pod and switch with PaulW | PaulH | 2 | Done: PaulW moved May 23, Alex into pod on May 24 |
| Prioritized Possible Future Work | |||
|---|---|---|---|
| Task | Owner | Estimate | Notes |
| High | |||
| Fix 2.0.1 Defects - cull and prioritize the list - draw a cutline - fix the ones above the cutline |
Alex, Vincent, Larry | 100 | |
| Initial Design for FOSS Discovery | Bob | 40 | |
| Truly make it a push-button end-to-end, continuous integration environment (at least nightly check out, build, unit test, functional test, package, install on supported distros, functional test) | Dan | 40 | |
| Report on current state of testing (pass/fail, # successful, # run, etc) – make emails more informative, refine reports to make it easier to digest | Dan | 13 | |
| Clean up test plans and be clear what needs testing (test authoring to be done separately) - Supported distros - Automated vs. manual tests - UI manual tests - etc. |
Mary | 13 | |
| Training on existing functionality (tagging, etc) | Bob | 3 | |
| Refine/add manual system test cases as necessary - determine gaps - prioritize - add defects |
Team | 100 | |
| Increase the number and coverage of automated unit and functional tests - address units that don't have any tests - reach 80% of coverage for all units |
Team | 100 | |
| Improve documentation (is there a budget from Phil?) - improve navigation - refine content - migrate content that is still valid |
Paul | 40 | |
| Modify code to support the db server on a separate system. This has always been a design goal but has not been tested. (same as "what does it take to put the web server on a separate system?" | 2 | Easy – test | |
| Remove pfile.pfile_liccount from schema and code (common/common-license.php, plugins/agent-license.php, plugins/agent-license-reanalyze.php, plugins/ui-license.php. This was an experimental feature that mistakenly had code checked in around it. | 1 | Easy | |
| Implement FOSS Discovery and create output | TBD | ?? | |
| UI reconciliation/editing for PTS integration | TBD | ?? | |
| Need method to correct license scan (#1869) | 13 | ||
| Set up integration staging machine - Right now, there isn't a testing machine that mimics what production (fo.usa) looks like; there needs to be for troubleshooting defects and testing |
13 | ||
| Metric development for fo.usa (creation, capturing, reporting) - # of users - # of analyses - ... |
5 | ||
| Password reset capability - from ScottL - passwords are stored encrypted in the database - send the user a temporary, modified password and allow them to change |
5 | ||
| Aligning license names with SPDX license names - includes missing names into nomos logic |
13 | ||
| Create a license test data set from analysis we've done already - one exists for nomos - one exists for unpack - this would create a common dataset all tests/agents can use |
20 | ||
| Add load testing - push the boundaries of FOSSology and ensure things finish/work |
|||
| Add performance testing - performance regression tests for UI and database - add a timer to check how fast code runs |
|||
| Upload from version control system (CVS, SVN, PVCS) - From George Pace, Jeroen, SpottedOtter | |||
| When viewing a file add ability to pop-up the full text of a license. | |||
| Update license reference list so nomos refers to the license ref with the license text. | |||
| Add to nomos licenses used by ninka and spdx. This includes exceptions. | |||
| Add capability for reanalysis without breaking data persistence ie. do new analysis without removing previous analysis results. This can be used, to compare new and old analysis results, and to insure that report url's are persistent. 1.2 implemented data collection for this for nomos and buckets. The UI needs to catch up and allow one to select the data set they want to see. The code is already in ui-buckets.php and ui-nomos-license.php (search for FUTURE). But we need to decide if this is the interface we want. | |||
| Public FOSSology instance at UNO | |||
| Move fo.usa siteminder service from ITG to PRO | |||
| Move from the LIT lab | |||
| Set up backups | |||
| Write a FOSSology article for the FOSS Legal magazine | |||
| Medium | |||
| Transition Alex's work | Alex expects to remain through summer | ||
| How can one tell who, when and from where an upload came? Add to ui-browse | Easy; add to info | ||
| Document FOSSology recommended hardware configs | |||
| Consider changing repo layout to work with "large" perhaps 16MB files kind of like hadoop. Using small files like now (avg size ~32k) makes repo deletes take forever and makes copying the repo take forever. For our repo, we could pack as many files as will fit wholly fit into ~16MB. Larger files would be saved as individual files. The whole point is to consolidate the tiny files to use fewer inodes and thus speed up delagent and repo copies. | |||
| FO permission scheme needs full implementation | |||
| We need a client only package so that people can submit jobs (fossjobs) from a machine not running fossology. | |||
| Need agent to remove orphan files from the repository. This can happen on a delagent failure. | |||
| Add license from kernel object modules (license from modsym) to license_file | |||
| Create a user interface to create bucket pools, bucket definitions, scripts and anything else needed, along with a prompt & screen to rerun analysis with your newly defined bucket pool. Current method is too ugly. | |||
| Remove bucket code that determines what bucket a container is in. After implementation we decided that this feature was way too confusing for the user because of the hierarchical nature of bucket evaluation. This will speed up bucket scans. Remove bucket_container table. | |||
| FOSSology.org cleanup (how team uses website, rules/process, etc) – using RedMine most effectively | |||
| Archive and remove old wiki content | |||
| Finish documentation organization | |||
| Identify documentation gaps, inconsistencies | |||
| Commenting uses/contributions/process descriptions | |||
| Update screenshots | |||
| Update instructions | |||
| Update monitoring scripts/administration | |||
| Add Servers | |||
| Fix 2.1.0 defects | |||
| Low | |||
| Detect code snippets | |||
| Binary analysis | |||
| Binary analysis for open source discovery | |||
| Code clone detection | |||
| Archived reports - simple text file dump, PDF reports, eventually full web archive of all analysis reports. Or else just don't throw away any data. Nomos and buckets already can do this but there is still no guarantee that someone doesn't subvert this process (see ars tables). | |||
| Write tar agent - re-tar arbitrary parts of repo - don't have an owner/customer to support | |||
| Easy way to install buttons/links on micro menu to run system utilities/scripts on a file. For example, .ko files could have an nm link, and a modinfo link. The script may need a way to determine if it should be added to menu or not. For example, nm applies to object files, but modinfo only applies to .ko. The purpose is to process fossology files in real time with system scripts/apps. | |||
| Allow license admin to work in read only mode at low (read) permission level. | |||
Completed Iterations¶
| Completed Tasks (May 8-21, 2012) | |||
|---|---|---|---|
| Task | Owner | Estimate | Notes |
| Ubuntu Developer Summit: - Manage the HP table, make introductions, make connections with Ubuntu community & Canonical |
Mary | 5 | Done - good conference |
| Add more disk space -- it is a pressing concern. - There is enough disk left to last about 2-3 weeks, so it needs to be addressed ASAP. - Done means each of the agents has a 1TB (or larger) disk, preserving all of its existing data. - This is a straightforward task of creating new virtual disks on the EVA and transferring the old contents to the new disks. Dan should be able to do this with all systems remaining on-line. |
Dan | 1 | - Dan has added 2 TB disks to all agents and re-synched - Will finish once fo.usa is updated so the system is only down once |
| Transition Mark's tasks to Dan to ensure Dan understands process - This is difficult to measure; the outcome is "fuzzy". For the first iteration, Dan would like to get a solid understanding of the existing test harnesses, test cases, test coverage, build systems, continuous integration systems, configuration management, package configuration, packaging automation, and existing documentation for each of these. - Dan would also like to make sure that he has all the necessary login permissions and memberships needed to participate in the project. - Done means have a drawing (literally) of the existing pieces of the continuous integration systems and how they fit together and can explain each piece (this can then be used in the next iteration to identify gaps and misunderstandings); be added or re-added to all mailing lists and other project resources needed (svn, wiki, mailing lists, system logins, etc). |
Dan | 5 | - Dan having daily discussions with Mark; understanding better the Jenkins/build/test infrastructure - Dan and Paul synched re: long-term vision and are fully aligned: automation is key to team - Dan has several drawings helping to organize thoughts |
| Fix 2.0 defects - #1887 - #1893 - #1899 - #1857 - #1875 - #1965 - #1905 - #1911 - #1917 |
Alex, Vincent | Alex: 3, Vincent: 2, Larry: 1 | - All defects are resolved; owners to close them and Bob/Larry to verify them |
| Validate defect fixes | Bob, Larry | 2 | Done -- all iteration defects verified and closed |
| Create packages and tag RC - Done means: Have an updated set of packages and set of notes (possibly not yet complete) on how to create packages. |
Vincent with Dan | 2 | Done RC2 packages created. Vincent will create the packages but Dan will help so he understands how packages are built using ProjectBuilder. |
| Test validation - Develop final test plan, execute test plan - Done means: all tests run and pass on RC2 |
Mary: 2, Vincent, Larry, Alex: 3 | 5 | Done http://www.fossology.org/projects/fossology/wiki/20-specific_Test_Suite#Testing-for-RC2 |
| Perform Release Checklist - Done means: all tasks on this page are UPDATED & completed |
Mary | 1 | Updated as we go |
| Update instructions for installation (single systems, clusters, fossology.usa.hp.com) - Done means: Instructions are accurate, complete and results in a properly tuned/configured working system |
Mary | 2 | Input from Bob & others will be important as they were instrumental in getting fo.usa properly tuned and running. 5/21 update: Started updates to INSTALL doc (postgres & kernel sections). Started capturing FO.usa changes (bucketpools & apache config) . |
| Increase the number of automated unit and functional tests - Package Agent (Vincent) - Upload from server, url (Larry) |
Team | Vincent: 3, Larry: 3 | larry: done, found a bug #2025, after #2025 is fixed, revise the tests are needed. Vincent: Done, added more unit test case for pkgagent. |
| Total Points | 35/53 | ||