~wgrant/ubuntu-cve-tracker/mainold

------------------ 
UBUNTU CVE TRACKER
------------------
With the newly revamped ubuntu-cve-tracker, it is much more like Debian's
kernel-sec list where you have these directories:

active/		(CVEs that need attention)
ignored/	(CVEs to be ignored, but want to track with special status/notes)
retired/	(CVEs that have been addressed)

ignored/ also has the file 'not-for-us.txt'.

check-cves pulls down the CVE list from MITRE, cross-references it with
ignored/, ignored/not-for-us.txt and retired/, then adds the new CVEs to
active/.  This is normally run on Wednesday, as this is when MITRE updates
its database.

Be sure to apt-get the following:
apt-get install python-configobj


CHECK-CVES
----------

Run like this:
./scripts/check-cves
./scripts/check-cves http://cve.mitre.org/data/downloads/allitems.xml
./scripts/check-cves <file>

Eg:
wget -N http://cve.mitre.org/data/downloads/allitems.xml
./scripts/check-cves ./allitems.xml

When running check-cves:
'a'dd 		to add to active/ (see Triaging, below)
'i'gnore 	to add to ignored/not-for-us.txt (see Triaging, below)
's'kip 		don't do anything (will come up next time)

* Note: it may take a few seconds before prompts, as allitems.xml is a large
file.


TRIAGING
--------

*** IMPORTANT ***
ubuntu-cve is PUBLIC.  All comments should be professional, and there should be
no embargoed items in ubuntu-cve (until they are made public that is).


Adding
------
1. adjust the Candidate field

2. Add the Description from Mitre

3. Set the Priority, if you can at this time.  Should be one of 'negligible',
'low', 'medium', 'high' or 'critical'

4. for each release, adjust PKG to be the source package for the software.
Note that for the kernel, upstream_PKG becomes 'upstream_linux-2.6' and the
releases should be 'dapper_linux-source-2.6.15', 'edgy_linux-source-2.6.17',
'feisty_linux-source-2.6.20', 'gutsy_linux-source-2.6.22', ...)

5. for each release, assign a status of (after the release_<source-package>
line): 'N/A', 'DNE', 'ignored', 'not-affected', '-unlisted-', 'needs triage',
'needed', 'deferred', 'pending', 'released'

6. for each piece of software that is affected by this CVE, add extra
'$release_<source-package>' fields.  An 'upstream_<source-package>' line
should also be included for each package (excepting special situations like
linux-source-X.Y.Z).

7. fill in any extra fields as needed (eg notes, references to patches, 
Assigned-to, discoverer, etc). If there are patches available, then use:

Patches_PKG:
 debdiff: URL
 vendor: URL
 upstream: URL
 patch: URL

Eg, for source package 'foo' that has a debdiff in launchpad, use:

Patches_foo:
 debdiff: https://bugs.launchpad.net/ubuntu/+source/foo/+bug/XXXXXX

8. Check Debian's secure-testing data/embedded-code-copies to see if any
other software is to be included in this CVE


Ignoring
--------
Be sure list why and what piece of software it is.  Eg:
 Unpackaged software (Does Not Exist): 'DNE - PhpNuke'
 Totally alien stuff (Not For Us):     'NFU - Cisco IOS'


Ubuntu Priorities
-----------------
These are very similar to the Debian priorities, but with some differences.
Priorities can be roughly mapped as:

  negligible	Something that is technically a security problem, but is
		only theoretical in nature, requires a very special
		situation, has almost no install base, or does no real
		damage.  These tend not to get backport from upstreams,
		and will likely not be included in security updates unless
		there is an easy fix and some other issue causes an update.

  low		Something that is a security problem, but is hard to
		exploit due to environment, requires a user-assisted
		attack, a small install base, or does very little damage.
		These tend to be included in security updates only when
		higher priority issues require an update, or if many
		low priority issues have built up.

  medium	Something is a real security problem, and is exploitable
		for many people.  Includes network daemon denial of service 
		attacks, and gaining user privileges. Updates should be made
		soon for this priority of issue.

  high		A real problem, exploitable for many people in a default
		installation.  Includes serious remote denial of services,
		local root privilege escalations, or data loss.

  critical	A world-burning problem, exploitable for nearly all people
		in a default installation.  Includes remote root privilege
		escalations, or massive data loss.


Package Status
--------------
For a given CVE, the package and release with status is encoded as:

 <release>_<source-package>: <status> (<version/notes>)

  DNE		The package (for the given release) does not exist in the
		archive.

  needs-triage	The vulnerability of this package (for the given release)
		is not known.  It needs to be evaulated.  (No version/notes)

  not-affected	This package (for the given release), while related to the
		CVE in some way, is not affected by the issue.  Notes
		should contain further information, if needed.  For example,
		if a given source package is vulnerable to a CVE, but the
		compiled binary is not (for example, linked to use a
		system copy instead of an internal-to-source copy of a
		library, and the CVE is about the internal copy).  For
		such a situation, the note should include the research
		about why the binary is not affected by the CVE.

  needed	This package (for the given release) is vulnerable to the
		CVE and needs fixing.  (Notes are valid.)

  ignored	This package (for the given release), while related to the
		CVE in some way, is being ignored for some reason.  The
		"notes" should detail why.  This is generally used when
		a given CVE's priority is "negligible", and a firm
		determination has been made to not fix a given release.

  pending	This package (for the given release) is vulnerable, and
		an update is pending, usually waiting for upload or
		publication.  The "version" should be the version containing
		the fix.

  deferred	The package (for the given release) is vulnerable, a fix
		is understood, but has been deferred for some reason.
		The "notes" should explain further.

  released	The package (for the given release) was vulnerable, but
		an update has been uploaded and published.  The "version"
		should be the version where the fix first appeared.


Retiring a CVE
--------------
When a CVE fix is released, must manually update the CVE file in active/
with the appropriate information.  Then 'check-syntax' (see below) 
and 'bzr mv' it to retired/.


UBUNTU-CVE Commands
-------------------
Useful commands are (all are run from within the top-level directory):

Verify syntax of CVE-* files
./scripts/check-syntax

To commit, use this command:
./scripts/check-syntax && bzr ci

Full listing:
./scripts/ubuntu-table

Full listing, flagging packages in main:
./scripts/ubuntu-table --supported

Show all active CVEs for supported (main) packages:
./scripts/ubuntu-table --supported 2>/dev/null | grep SUPPORTED

Number of active CVEs that are in main:
./scripts/ubuntu-table --supported 2>/dev/null | grep SUPPORTED | wc -l

Show all active CVEs for partner packages:
./scripts/ubuntu-table --supported 2>/dev/null | grep PARTNER

Show CVEs for a particular source package:
./scripts/pkg_status pkgname1 pkgname2 ...
./scripts/pkg_status -f pkgname1 pkgname 2 ... (full listing)

Show CVEs with undefined priorities (ie untriaged):
./scripts/ubuntu-table --untriaged

Create/edit a new CVE (eg if don't want to wait for check-cves):
./scripts/active_edit -p package -c CVE-XXXX-XXXX

See the status of specific CVEs:
./scripts/cve_status CVE-2006-4519 CVE-2007-2949 CVE-2007-3741

See the status of specific CVEs without viewing the full entry:
./scripts/cve_status -s CVE-2006-4519 CVE-2007-2949 CVE-2007-3741

See all bugs with priority of medium or higher:
./scripts/ubuntu-table --supported | egrep -v '[[:space:]]+(untriaged|negligible|low)$'

See a listing of packages with number of CVEs attached to them, weighted by
CVE priority:
./scripts/cve_packages
./scripts/cve_packages -t	(just totals)

See ordering of SUPPORTED packages that need updates:
./scripts/cve_packages | egrep '(SUP|COM)' | sort -n
./scripts/cve_packages -S | egrep '(SUP|COM)' | sort -n		(skip devel)

See CVEs with patches:
./scripts/cve_patches

Show items needing to be retired:
./scripts/ubuntu-table > /dev/null
./scripts/cve_need_retire
./scripts/cve_need_retire -f (full listing)
./scripts/cve_need_retire -p (list path to CVE)

Retiring items:
bzr mv $(./scripts/cve_need_retire -p) ./retired/


Mark CVEs as "released" from known USNs:
wget -N http://people.ubuntu.com/~ubuntu-security/usn/database.pickle

or use rsync if you have access:
rsync -v --progress -e ssh rookery:~ubuntu-security/public_html/usn/database.pickle ./database.pickle

./scripts/sync-from-usns.py database.pickle -u

Check for releases in devel that are higher than upstream fixed versions:
./scripts/sync-from-versions.py -u

Refresh descriptions from Mitre:
./scripts/check-cves --refresh


Reports:
./scripts/report-todo
./scripts/report-todo -S		(don't include devel)
./scripts/report-todo-numbers
./scripts/report-todo-numbers -S 	(don't include devel)
./scripts/cve_packages -S | egrep '(SUP|COM)' | sort -n

./scripts/html-report
./scripts/html-report -S		(don't include devel)

Embargoed Items
---------------
Embargoed items are supported in the following scripts in all the scripts
except sync-from-usns.py.  If no CVE has been assigned yet, an embargoed item
should be prefixed with 'EMB-', followed by any combination of alphanumerics
and dashes.  Eg:
EMB-xorg-2007-0001
EMB-foo

To include embargoed items simply create a symlink from 'embargoed' to
the directory holding embargoed items. ubuntu-cve-tracker will not
use 'embargoed' unless it is a symlink.


Non-CVE Vulnerabilties
----------------------
If find a non-CVE assigned vulnerability, then:

1. report the bug to Debian
2. in ubuntu-cve, use 00boilterplate and create CVE-NEED-0001
3. if desired, email vendor-sec@lst.de and Cc cve@mitre.org asking for a CVE
4. file a bug in LP (assign to security-team if main, motu-swat if universe

Doing '1' may be enough if its in universe and not high priority, as
it will eventually find its way back to Ubuntu.


Stable Release Actions
----------------------
When a stable release is published, the active CVEs need to be adjusted to
reflect the new stable release.  e.g. when gutsy was published:
  cd active; perl -pi -e 's/^(devel_(.*))/gutsy_$2\n$1/g' CVE-*
The script tools will need to be adjusted as well.  There is usually some
lag time between the new devel archive opening and the stable release
getting published.  This means that "devel" will disappear from ubuntu-table
briefly.  Don't forget to update active/00boilerplate too.


Pre-commit Syntax Checking
--------------------------
To perform pre-commit syntax checking, this little hack should allow for it:

mkdir -p ~/.bazaar/plugins/hooks
cat >~/.bazaar/plugins/hooks/__init__.py <<EOM
#!/usr/bin/python
from bzrlib.branch import Branch

def run_tests(local, master, old_revno, old_revid, new_revno, new_revid, seven, eight):
    #print local, master, old_revno, old_revid, new_revno, new_revid, seven, eight
    if 'ubuntu-cve' in master.base:
        import subprocess
        print ''
        rc = subprocess.call(['./scripts/check-syntax','--verbose'])
        if rc != 0:
            import sys
            sys.exit(1)

Branch.hooks.install_hook('pre_commit', run_tests)
Branch.hooks.name_hook(run_tests, 'run-tests')
EOM