How to Interpret the Job Queue¶

A common misconception about FOSSology is that an uploaded file (project, tarball, zip file, ISO, etc) is immediately analyzed and available for viewing in real-time. In fact this is not the case.

Within FOSSology, Agents are responsible for processing and analyzing uploaded files. The time it takes for an agent to run can vary from a few milliseconds to many hours, depending on the type of agent and the size of the file being processed. The completion time also depends on what agents are already running within FOSSology, and the order in which new uploads were queued for processing.

Because of the wide range of processing times for agents, and the asynchronous nature of uploads and analysis, the operation of the FOSSology system is controlled by a Scheduler. The scheduler is responsible for queuing and running jobs.

Accessing the Job Queue¶

To access the job queue, click on the "Jobs" menu and select the "My Recent Jobs" sub-menu.

This presents a list of all active jobs. A job can be in one of a set of color-coded states:

Job State		Description
Queued		The job has been queued, but not yet scheduled/run
Started		The job is currently running
Completed		The job has completed successfully
Blocked		The job has been blocked by a job upon which it depends
Failed		The job did not complete successfully

In this example, a series of jobs have been queued because a user has created an upload of Cherokee version 1.2.101.

Job 125 is the wget agent, responsible for retrieving the project's source code via the world wide web, and has finished.

Job 126, the unpack agent, is responsible for unpacking the source code. Clearly, it cannot run until the wget agent has completed and downloaded the source code into the FOSSology repository. This relationship is indicated in the left-hand most column, which lists the job number (126) followed by a slash, followed by the job number it depends on (125). In cases where a job has no dependency, only the job number itself is listed.

The other jobs (127-131) are other agents that are queued up but have yet to start. They will begin as soon as the unpack agent completes. They can be paused or canceled by selecting the links to the right of the table.

Clicking on the Jobs -> My Recent Jobs tab will refresh the job status page.

Now a few moments have passed and the rest of the jobs originally seen in the Summary have run and completed.

Viewing detailed job information¶

By clicking on the job number link in the Job/Dependency column, a more detailed view of the jobs can be seen:

In this case, the adj2nest agent has run on Cherokee 1.2.101. The detailed job listing shows additional information about scheduled jobs:

• jq_pk: The id of the job correlated to the job queue.
• Job|Agent Name: The name of the job and agents that are/were scheduled.
• Queued: The time (in Hours:Minutes:Seconds) the job was received and queued by the scheduler. The scheduler may pick up a job (change its status from "queued" to "scheduled") before an agent slot is available to run it. In this case the job will not actually begin running, even though it is scheduled.
• Started: The time (in Hours:Minutes:Seconds) the job was started by the scheduler.
• Ended: The time (in Hours:Minutes:Seconds) the job was completed.
• Elapsed: How long (in Hours:Minutes:Seconds) the job took to run from the time the scheduler picked up the job to when it completed.
• Status: The status of the job whether scheduled, in progress, or finished.
• Number of items processed: For most agents, they will report the number of files that have been processed as part of the job. This includes the ununpack and license agents.
• Submitter: name of the user who submitted the source code for analysis.
• Upload: The id of the upload correlated to the original upload task.
• Log: Any additional information of value.

Manually scheduling an agent to run on an Upload¶

Occasionally it may be desirable to upload a project, but not immediately schedule any analysis. At other times, it may be necessary to re-run an analysis on an upload. You can easily use the scheduler to manually schedule an agent to run on an upload at any time.

To manually schedule an agent to run, select the "Jobs" menu, and the "Schedule Agents" item.

This will provide a page for selecting the upload and agent to run.

First select the folder containing the upload that you wish to schedule an agent for. The drop down menu provides a list of all the folders in the the repository. Once you select a folder, FOSSology will present a list of all uploads in that folder.

In this case we've selected the apache-camel-2.9.1.tar.gz upload in the folder called "Software Repository".

Once an upload is selected, FOSSology provides a list of all the agents that may be scheduled for that upload. With most web browsers, you may select multiple agents from the list by holding down the Control key and clicking on the item(s). Note that if an agent is already queued or scheduled for an upload, it will not be available to schedule manually.

Selecting Jobs --> My Recent Jobs shows that the agents have been scheduled for the additional analysis.

You can not re-run agents that have already been run.

Viewing system-wide agent status¶

It is sometimes useful to view the status of all agents available to the system. This can be accomplished by selecting the "Admin" menu, and then choosing the "Scheduler" sub-menu. Then click on the "Status" item.

This will present a view of all instances of all agents either waiting on a job, or processing a job.

• The first column, "Update Time", shows the last heartbeat received from an agent.
• The second column, "Status", shows the status of the agent, which can be "RUNNING" (Agent is currently processing a job), "READY" (agent is preparing to run a job that has been scheduled), or "FREE" (agent is idle and waiting to be assigned a job by the scheduler).
• The third column shows the name of the agent. Note that the scheduler itself will be listed here as an agent.
• The fourth column shows the hostname of the system where the agent is running. This can be useful when FOSSology is installed in a multi-host environment. For more information on multi-system setup, refer to Multi-system Setup and the FOSSology README file distributed with the software.
• The fifth column shows the parameters supplied by the scheduler to scheduled or running agents. (Note, no parameters will be listed for "FREE" agents).

Viewing Job Queue History¶

You can view the complete history of the jobs that have been queued, scheduled and run. Simply click on the "History" link in the mini-menu above any Job Status page, such as described above in "Accessing the Job Queue".

This shows all of the jobs already run, organized by upload, and then job name. You can use this history to review jobs that have completed or failed. You within the job history view, you can switch between "Summary" and "Detailed" view using the links in the mini-menu at the top.