~ubuntu-security/ubuntu-cve-tracker/master

------------------
UBUNTU CVE TRACKER
------------------
With the newly revamped ubuntu-cve-tracker, it is much more like Debian's
kernel-sec list where you have these directories:

active/		(CVEs that need attention)
ignored/	(CVEs to be ignored, but want to track with special status/notes)
retired/	(CVEs that have been addressed)

ignored/ also has the file 'not-for-us.txt'.

check-cves pulls down the CVE list from MITRE, cross-references it with
ignored/, ignored/not-for-us.txt and retired/, then adds the new CVEs to
active/.

check-cves can also be used to import Debian DSAs and CVEs that are missing in
Ubuntu (requires secure_testing_path to be properly set up):
./scripts/check-cves --import-missing-debian

Sometimes MITRE and NVD are out of date. The locate_cves.py command can read
an mbox file and output a list of CVEs that are referenced in the mbox emails
that are not included in Ubuntu yet. This output can then be given to
check-cves:
./scripts/check-cves --untriaged <path to locate_cves.py output>

Eg, for Redhat:
  $ w3m -dump_source https://www.redhat.com/archives/rhsa-announce/<year>-<Month>.txt.gz > ~/<year>-<Month>.txt.gz
  $ gzip -d ~/<year>-<Month>.txt.gz
  $ ./scripts/locate_cves.py ~/<year>-<Month>.txt > ~/redhat.mbox
  $ ./scripts/check-cves --untriaged ~/redhat.mbox

Eg, for oss-security:
  <save last month to mbox file, ~/oss-sec_raw.mbox>
  $ ./scripts/locate_cves.py ~/oss-sec_raw.mbox > ~/oss-sec.mbox
  $ ./scripts/check-cves --untriaged ~/oss-sec.mbox

Be sure to apt-get the following:
apt-get install python-configobj python-yaml python-genshi realpath python-progressbar python-pychart subversion

You might also be interested in $UST/utilities/maildir2mbox.py (though
locate_cves.py has maildir support).

CONFIGURATION
-------------
The following values need to be configured in ~/.ubuntu-cve-tracker.conf:

- plb_authentication: path to Firefox LP cookies, used to authenticate
  to LP in various tools that need it (sis-changes).
- secure_testing_path: path to Debian "secure-testing" SVN tree, so that
  CVEs known to Debian can be shown while triaging CVEs (check-cves).
  (get read-only access with 'svn co svn://anonscm.debian.org/svn/secure-testing')
- packages_mirror: path to archive-layout mirror of supported archs'
  Release and Packages files (used by sis-generate-usn, with the mirror
  created by a tool like packages-mirror).
- usn_tool: path to usn-tool bzr tree, used to manipulate USN databases
  and for templates (sis-generate-usn).
- usn_storage: path to individual USN pickle database output when
  generating a single USN database before merging into the master USN
  database (sis-generate-usn).
- usn_db_copy: path to copy of master USN database, used when creating
  a template for a USN where "N" != 1 (sis-generate-usn).
- partner_mirror: same as packages_mirror, but for the partner repo.
- debian_mirror: same as packages_mirror, but for Debian testing repo.
- mitre_loc: where to download the MITRE CVE database
- nvd_loc: where to download the NVD database
- kernel_team_tools_path: path to Ubuntu Kernel Team Tools git tree:
  git://kernel.ubuntu.com/ubuntu/kteam-tools.git

Checkout the following branch:
$ bzr branch lp:ubuntu-qa-tools

Then add a sylink from UQT/common/lpl_common.py to UCT/scripts/lpl_common.py:
$ ln -s <path to>/ubuntu-qa-tools/common/lpl_common.py ./scripts/lpl_common.py


CHECK-CVES
----------

Run like this:
./scripts/check-cves
./scripts/check-cves http://cve.mitre.org/data/downloads/allitems.xml
./scripts/check-cves <file>

Eg:
wget -N http://cve.mitre.org/data/downloads/allitems.xml
./scripts/check-cves ./allitems.xml
for i in $(seq 2004 $(date +%Y)) recent; do wget --quiet -N http://nvd.nist.gov/download/nvdcve-$i.xml; done
./scripts/check-cves ./nvdcve-2*.xml

When running check-cves:
'a'dd 		to add to active/ (see Triaging, below)
'i'gnore 	to add to ignored/not-for-us.txt (see Triaging, below)
's'kip 		don't do anything (will come up next time)

* Note: it may take a few seconds before prompts, as allitems.xml is a large
file.


TRIAGING
--------

*** IMPORTANT ***
ubuntu-cve is PUBLIC. All comments should be professional, and there should be
no embargoed items in ubuntu-cve (until they are made public that is).


Adding
------
1. adjust the Candidate field

2. Add the Description from Mitre

3. Set the Priority, if you can at this time. Should be one of 'negligible',
'low', 'medium', 'high' or 'critical'. You can also override the Priority
field on a per package basis, by using 'Priority_<source package>: ...'

4. for each release, adjust PKG to be the source package for the software.
Note that for the kernel, upstream_PKG becomes 'upstream_linux-2.6' and the
releases should be 'dapper_linux-source-2.6.15', 'edgy_linux-source-2.6.17',
'feisty_linux-source-2.6.20', 'gutsy_linux-source-2.6.22', ...)

5. for each release, assign a status of (after the release_<source-package>
line): 'DNE', 'ignored', 'not-affected', 'needs-triage', 'needed', 'active',
'deferred', 'pending', 'released'

6. for each piece of software that is affected by this CVE, add extra
'$release_<source-package>' fields. An 'upstream_<source-package>' line
should also be included for each package (excepting special situations like
linux-source-X.Y.Z).

7. fill in any extra fields as needed (eg notes, references to patches,
Assigned-to, discoverer, etc). If there are patches available, then use:

Patches_PKG:
 debdiff: URL
 vendor: URL
 upstream: URL
 patch: URL

Eg, for source package 'foo' that has a debdiff in launchpad, use:

Patches_foo:
 debdiff: https://bugs.launchpad.net/ubuntu/+source/foo/+bug/XXXXXX

If a CVE only affects a binary in universe (from a source in main),
this can be marked using a Tag field. E.g:

Tags_foo: universe-binary

A package may not be supported by the Ubuntu Security team, but might be by
another Canonical team (eg linux-armadaxp is in main, but not officially
supported by the security team). This can be marked like so:
Tags_foo: not-ue

If a package has been protected by some sort of pro-active security measures,
mark from the list of "apparmor", "stack-protector", "fortify-source",
"symlink-restriction", "hardlink-restriction", "heap-protector", "pie" etc:

Tags_bar: stack-protector
Tags_baz_trusty: stack-protector

Use the above tags if the protection will be mentioned in the USN and/or the
protection mechanism is shown to protect against the vulnerability. Also note
that the tag is per release only, so you must say each release that is
protected (ie, in the above example, baz on trusty is protected, but not
lucid or precise). Valid tags are in 'valid_tags' in cve_lib.py.

8. Check Debian's secure-testing data/embedded-code-copies to see if any
other software is to be included in this CVE


Ignoring
--------
Be sure list why and what piece of software it is. Eg:
 Unpackaged software (Does Not Exist): 'DNE - PhpNuke'
 Totally alien stuff (Not For Us):     'NFU - Cisco IOS'

CVEs that affect Ubuntu releases can also be ignored. This is most often done
for packages that exist in a previous release, but that release has reached End
of Life. Eg:

upstream_foo: released (1.2.3)
lucid_foo: ignored (reached end-of-life)
precise_foo: released (1.2.2-1ubuntu1.2)
trusty_foo: not-affected (1.2.3-1)
utopic_foo: not-affected (1.3.0-1)
devel_foo: DNE


Ubuntu Priorities
-----------------
These are very similar to the Debian priorities, but with some differences.
Priorities can be roughly mapped as:

  negligible	Something that is technically a security problem, but is
		only theoretical in nature, requires a very special
		situation, has almost no install base, or does no real
		damage.  These tend not to get backport from upstreams,
		and will likely not be included in security updates unless
		there is an easy fix and some other issue causes an update.

  low		Something that is a security problem, but is hard to
		exploit due to environment, requires a user-assisted
		attack, a small install base, or does very little damage.
		These tend to be included in security updates only when
		higher priority issues require an update, or if many
		low priority issues have built up.

  medium	Something is a real security problem, and is exploitable
		for many people.  Includes network daemon denial of service 
		attacks, cross-site scripting, and gaining user privileges.
		Updates should be made soon for this priority of issue.

  high		A real problem, exploitable for many people in a default
		installation.  Includes serious remote denial of services,
		local root privilege escalations, or data loss.

  critical	A world-burning problem, exploitable for nearly all people
		in a default installation of Ubuntu.  Includes remote root
                privilege escalations, or massive data loss.


Package Status
--------------
For a given CVE, the package and release with status is encoded as:

 <release>_<source-package>: <status> (<version/notes>)

  DNE		The package (for the given release) does not exist in the
		archive.

  needs-triage	The vulnerability of this package (for the given release)
		is not known.  It needs to be evaluated.  (No version/notes)

  not-affected	This package (for the given release), while related to the
		CVE in some way, is not affected by the issue.  Notes
		should contain further information, if needed.  For example,
		if a given source package is vulnerable to a CVE, but the
		compiled binary is not (for example, linked to use a
		system copy instead of an internal-to-source copy of a
		library, and the CVE is about the internal copy).  For
		such a situation, the note should include the research
		about why the binary is not affected by the CVE. Another
		example is when an earlier or later version of the package was
		affected, but the current version is not.  If a package
		was fixed during a devel release by someone outside of the
		security team, it is appropriate to put the version that
		fixed the issue in the note.  If a note is too long,
		it can be moved to the top-level "Notes" section.  Examples
		of common notes used with not-affected:
			code not present
			linked against (poppler|system libraries|etc...)
			(MacOS X|Windows|Redhat) only
			not included in package
			register_globals not supported
			network listening disabled by default
			by design

  needed	This package (for the given release) is vulnerable to the
		CVE and needs fixing.  (Notes are valid.)

  active	The package (for the given release) is vulnerable to the
		CVE, needs fixing, and is actively being worked on by
		the person belonging to the IRC nick in the "Assigned-to"
		field.	(Notes are valid.)

  ignored	This package (for the given release), while related to the
		CVE in some way, is being ignored for some reason.  The
		"notes" should detail why.  This is generally used when
		a given CVE's priority is "negligible", and a firm
		determination has been made to not fix a given release.

  pending	This package (for the given release) is vulnerable, and
		an update is pending, usually waiting for upload or
		publication.  The "version" should be the version containing
		the fix.

  deferred	The package (for the given release) is vulnerable, the problem
		is understood, but has been deferred for some reason. The
		"notes" need to explain further. If a date was specified eg
		"deferred (2012-01-01)" the date indicates the date the CVE
		was put in the deferred state.

  released	The package (for the given release) was vulnerable, but
		an update has been uploaded and published.  The "version"
		should be the version where the fix first appeared. This
		status should also be used for when the 'devel' release is
		fixed due to Ubuntu contributed changes (not-affected should be
		used when a non-Ubuntu contributed sync from Debian fixes the
		issue).


Community Supported Packages
----------------------------
To most accurately reflect reality in community supported packages in the
archive, the following guidelines should be used when triaging community
supported packages:

1. CVEs in community supported packages for EOL released should follow the EOL
for the release (ie desktop/universe is 18 months for non-LTS, 3 years for LTS,
5 years for server). If based on the above the release is EOL, then the status
of the CVE should be 'ignored (reached end-of-life). See 'End of Life' section
for more information. Eg:
upstream_foo: released (1.2.3)
lucid_foo: ignored (reached end-of-life)
...

2. Look carefully at MITRE versions. If it has a 'fixed in' or 'less than',
adjust 'upstream' field accordingly to use this. Mark later versions as
'not-affected' and earlier as 'needed. be careful and leave as 'needs-triage'
if status is not clear for multiple branches (ie 1.2 vs 1.4). The 'upstream'
field may contain multiple versions, eg 'upstream_foo: released (1.0.1, 1.2).

3. If Debian has a fix in its development releases (unstable or testing),
prefer upstream version to Debian version in 'upstream' field, except where it
is fixed in Debian in an earlier version than upstream or no info on upstream
version is available.


Retiring a CVE
--------------
When a CVE fix is released, must manually update the CVE file in active/
with the appropriate information.  Then 'check-syntax' (see below) 
and 'bzr mv' it to retired/.


UBUNTU-CVE Commands
-------------------
Useful commands are (all are run from within the top-level directory):

Verify syntax of CVE-* files
./scripts/check-syntax

To commit, use this command:
./scripts/check-syntax && bzr ci

Full listing:
./scripts/ubuntu-table

Full listing, flagging packages in main:
./scripts/ubuntu-table --supported

Show all active CVEs for supported (main) packages:
./scripts/ubuntu-table --supported 2>/dev/null | grep SUPPORTED

Number of active CVEs that are in main:
./scripts/ubuntu-table --supported 2>/dev/null | grep SUPPORTED | wc -l

Show all active CVEs for partner packages:
./scripts/ubuntu-table --supported 2>/dev/null | grep PARTNER

Show all out-of-sync CVEs for the devel release:
./scripts/ubuntu-table --supported | grep 'out of sync' | awk '{if ($8 == "needed*" || $8 == "needs-triage*") { print $0 }}'

Show open CVEs for a particular source package:
./scripts/pkg_status pkgname1 pkgname2 ...
./scripts/pkg_status -f pkgname1 pkgname 2 ... (full listing)

Show CVE history for a particular source package:
./scripts/pkg_history pkgname1 pkgname2 ...
./scripts/pkg_history -f pkgname1 pkgname 2 ... (full listing)

Show CVEs with undefined priorities (ie untriaged):
./scripts/ubuntu-table --untriaged

Create/edit a new CVE (eg if don't want to wait for check-cves):
./scripts/active_edit -p package -c CVE-YYYY-XXXX

Create a new CVE that does not have a CVE identifier yet:
./scripts/active_edit -p package -c CVE-YYYY-NNN1
./scripts/active_edit -e -p package -c CVE-YYYY-NNN1

See the status of specific CVEs:
./scripts/cve_status CVE-2006-4519 CVE-2007-2949 CVE-2007-3741

See the status of specific CVEs without viewing the full entry:
./scripts/cve_status -s CVE-2006-4519 CVE-2007-2949 CVE-2007-3741

See all bugs with priority of medium or higher:
./scripts/ubuntu-table --supported | egrep -v '[[:space:]]+(untriaged|negligible|low)$'

See a listing of packages with number of CVEs attached to them, weighted by
CVE priority:
./scripts/cve_packages
./scripts/cve_packages -m       (just supported (main, restricted and partner))
./scripts/cve_packages -u       (just unsupported (universe and multiverse))
./scripts/cve_packages -t	(just totals)
./scripts/cve_packages -a	(with assignees)

See ordering of SUPPORTED packages that need updates:
./scripts/cve_packages -m | grep '^[0-9]' | sort -n
./scripts/cve_packages -m -S | grep '^[0-9]' | sort -n	(skip devel)
./scripts/report-date.py -S -m (skip devel)

See all CVE assignments:
./scripts/cve_packages --by-assignee

See CVE assignments by individual:
./scripts/cve_packages --by-assignee --assignee=...
./scripts/cve_packages -a --assignee=...

Pull ubuntu-security bugs from Launchpad marked as 'In Progress':
./scripts/pull-in-progress.py
./scripts/pull-in-progress.py -u (updates active CVEs with patch information)

See CVEs with patches:
./scripts/cve_patches
./scripts/cve_patches -s (only supported)

Show items needing to be retired:
./scripts/ubuntu-table > /dev/null
./scripts/cve_need_retire
./scripts/cve_need_retire -f (full listing)
./scripts/cve_need_retire -p (list path to CVE)

Retiring items:
bzr mv $(./scripts/cve_need_retire -p) ./retired/

Ignore open CVEs for a certain release:
./scripts/pkg_status webkit | cut -f1 -d' ' | xargs -I{} ./scripts/mass-cve-edit -p webkit -r trusty -s ignored -v 'reached end-of-life' {}

Mark CVEs as "released" from known USNs:
wget -N http://usn.ubuntu.com/usn-db/database.pickle

or use rsync if you have access:
rsync -v --progress -e ssh people:~ubuntu-security/public_html/usn/database.pickle ./database.pickle

or pull through bzip2:
curl -s http://usn.ubuntu.com/usn-db/database.pickle.bz2 | bzcat > ./database.pickle

./scripts/sync-from-usns.py database.pickle -u

Check syntax without regard to new (USN'd) packages:
CVE_ALLOW_NEWER_PKGS=1 ./scripts/check-syntax

Check for releases in devel that are higher than upstream fixed versions:
./scripts/sync-from-versions.py -u

Check for releases in 'precise' that are higher than upstream fixed versions
for chromium-browser:
./scripts/sync-from-versions.py -u -r precise -p chromium-browser

Refresh descriptions from Mitre:
./scripts/check-cves --refresh

Refresh descriptions and publication dates from NVD:
wget -N http://nvd.nist.gov/download/nvdcve-200{5,6,7,8}.xml
./scripts/check-cves --refresh nvdcve-200*.xml

Update state for a bunch of CVEs for a particular package and release:
./scripts/mass-cve-edit -p <package> -r <release> -v <version> -s <state> CVE...
Eg:
./scripts/mass-cve-edit -p seamonkey -r karmic -s released -v 1.1.17+nobinonly-0ubuntu1 CVE-2009-1841 CVE-2009-1838 CVE-2009-1836 CVE-2009-1835 CVE-2009-1392 CVE-2009-1832 CVE-2009-1833 CVE-2009-1311 CVE-2009-1307


Reports:
./scripts/report-todo
./scripts/report-todo -S		(don't include devel)
./scripts/report-todo-numbers
./scripts/report-todo-numbers -S 	(don't include devel)

./scripts/html-report
./scripts/html-report -S		(don't include devel)

./scripts/monthly-report

All CVEs fixed in USNs for 10.04, with priorities:
./scripts/report-updates.py --with-eol | grep lucid

Kernel Report Example:
./scripts/ubuntu-table -p linux -P linux -S | cut -c1-14,36-95


Embargoed Items
---------------
Embargoed items are supported in the following scripts in all the scripts
except sync-from-usns.py.  If no CVE has been assigned yet, an embargoed item
should be prefixed with 'EMB-', followed by any combination of alphanumerics
and dashes.  Eg:
EMB-xorg-2007-0001
EMB-foo

To include embargoed items simply create a symlink from 'embargoed' to
the directory holding embargoed items. ubuntu-cve-tracker will not
use 'embargoed' unless it is a symlink.


Non-CVE Vulnerabilties
----------------------
If find a non-CVE assigned vulnerability, then:

1. report the bug to Debian
2. in ubuntu-cve, use 00boilterplate and create CVE-NEED-0001
3. if desired, email vendor-sec@lst.de and Cc cve@mitre.org asking for a CVE
4. file a bug in LP (assign to security-team if main, motu-swat if universe

Doing '1' may be enough if its in universe and not high priority, as
it will eventually find its way back to Ubuntu.


Stable Release Actions
----------------------
When a stable release is published, the active CVEs need to be adjusted to
reflect the new stable release.  e.g. when trusty was published:
  perl -pi -e 's/^((#?)devel_(.*))/$2trusty_$3\n$1/g' active/{CVE-,00boilerplate}*
The script tools will need to be adjusted as well.  There is usually some
lag time between the new devel archive opening and the stable release
getting published.  This means that "devel" will disappear from ubuntu-table
briefly:
  scripts/cve_lib.py should have the new release timestamp (in UTC) added
    to 'release_stamps'.
  scripts/cve_lib.py should have an empty 'devel_release'.

Move all active CVEs and boilerplates from "devel" to release state:
./scripts/release-cycle-released $RELEASE


Development Release Actions
---------------------------
Fill in releases and devel_release in ubuntu-cve-tools/scripts/cve_lib.py

Move all active CVEs and boilerplates from latest release to devel state:
./scripts/release-cycle-devel-opens $LATEST_STABLE_RELEASE

Add release to non-ports and ports section of
ubuntu-cve-tools/scripts/packages-mirror

End of Life
-----------
When a releases is end-of-lifed, all CVEs for that release must be updated.
Here is how:

1. add the release to the list of releases in 'eol_releases' in
   scripts/cve_lib.py

2. update the CVEs:
$ sed -i 's/^<release>_\(.*\): \(needed\|needs\-triage\)/<release>_\1: \2 (reached end-of-life)/g' ./active/CVE-*
$ sed -i '/^<release>_\(.*\): /d' ./active/00boilerplate*

3. retire the CVEs (see 'Retiring items', above)

This should generally be done as 1 commit with no other changes, for easier
review.


Ubuntu variants with PPA overlays
---------------------------------
Ubuntu Touch and Ubuntu Core use a ppa overlay on top of an Ubuntu base
release. cve_lib.py will list these as '<base>/<ppa name>' in all_releases
and release_names. The <base>-<ppa name>-supported.txt files list the source
packages that comprise these variant releases. CVEs will use '<base>/<ppa
name>' as the release name for CVE tracking purposes.

Variants where the Ubuntu base release is EOL are possible, as with Ubuntu
Touch 15.04. In these instances, the base release is marked end of life while
the variant release is still active and updates will only be provided to the
variant release via its ppa overlay.

Currently used variant overlay PPAs:
 * Ubuntu Touch: https://launchpad.net/~ci-train-ppa-service/+archive/ubuntu/stable-phone-overlay/+packages
 * Ubuntu Core 15.04: https://launchpad.net/~snappy-dev/+archive/ubuntu/image/+packages

Images built from these PPAs employ batched OTA (over the air) updates. As
such, when a package is uploaded to the PPA (or the base release distro version
if before the EOL) it should be marked as 'pending (<version>)' and only after
the stable image (ie, the one users use) has the package should the CVE be
marked as 'released'.  The following will report CVEs in this pending state for
overlay PPAs:
$ ./scripts/cve-alert.sh ubuntu-core
$ ./scripts/cve-alert.sh ubuntu-touch

Once the stable images have been updated, use the mass-cve-edit tool to mark
the CVEs as 'released (<version>)'.


New Backport Kernel
-------------------
When a new backport kernel is added, update scripts/cve_lib.py's
kernel_srcs and description_overrides.

Then update the 00boilerplate.linux with its entry, add that entry to each
CVE with an entry for the LTS the kernel was added to, and update all the
statuses for the newly added kernel, based off the version it will be
branched from. For example, to add a new kernel backported from Vivid to
the Trusty LTS release, run the following command:

./scripts/add-backport-kernel -l trusty -b vivid

You must also add the new backport kernel to the end of the
$UQT/security-tools/kernel-abi-check script.

Devel Release Closed
--------------------
During the week between a new release and when devel opens, the pre-filled
CVE templates from ./scripts/process_cves will give devel_foo: DNE without
actually knowing if the package exists or not. Change DNE to the proper
value for what the new devel release will need once it is opened.

Devel Release Opens
-------------------
Changes to tools:
  scripts/cve_lib.py should have new release added to 'releases',
    'devel_release', and 'release_names'.


Vim syntax highlighting/checking
--------------------------------
$UCT/scripts/cve.vim can perform rudimentary syntax checking for CVEs. To use:
$ mkdir -p ~/.vim/syntax
$ ln -s $UCT/scripts/cve.vim ~/.vim/syntax/cve.vim

Then add to ~/.vimrc something like:
autocmd BufNewFile,BufRead CVE-[0-9][0-9][0-9][0-9]-[0-9][0-9][0-9][0-9] set syntax=cve

You can also option turn it on within vim using:
:set syntax=cve

Pre-commit Syntax Checking
--------------------------
To perform pre-commit syntax checking on only the files that have been
modified in the Bazaar tree, the following hook can be used. If an error is
discovered, the check-syntax output will be printed and then you'll have the
opportunity to ignore the errors or to exit immediately. To preemptively
avoid running check-syntax, you can set UCT_IGNORE_CHECK_SYNTAX=1 in the
environment.

mkdir -p ~/.bazaar/plugins/hooks
cat > ~/.bazaar/plugins/hooks/__init__.py <<EOM
#!/usr/bin/python
from bzrlib.branch import Branch

def run_tests(local, master, old_revno, old_revid, new_revno, new_revid, tree_delta, new_tree):
    #print local, master, old_revno, old_revid, new_revno, new_revid, tree_delta, new_tree
    import os
    if 'ubuntu-cve-tracker' in master.base and not os.environ.has_key('UCT_IGNORE_CHECK_SYNTAX'):
        import subprocess
        print ''

        if local is not None:
            local_branch = local
        else:
            local_branch = master
        old_tree = local_branch.basis_tree()

        changed_files = []

        old_tree.lock_read()
        new_tree.lock_read()

        try:
            iterator = new_tree.iter_changes(old_tree)
            for (file_id, paths, changed_content, versioned, parent,
                name, kind, executable) in iterator:
                if (changed_content and paths[1] is not None):
                    changed_files.append(paths[1])

        finally:
            old_tree.unlock()
            new_tree.unlock()

        if len(changed_files) > 0:
            import tempfile
            (fd, filename) = tempfile.mkstemp(prefix="ucthook-")

            handle = os.fdopen(fd, 'w')
            handle.write('\n'.join(changed_files))

            rc = subprocess.call(['./scripts/check-syntax', '--verbose',
                                  '--filelist', filename])

            if os.path.exists(filename):
                os.unlink(filename)

            if rc != 0:
                import sys

                sys.stdout.write('Found syntax errors. Enter "ignore" to continue or anything else to exit: ')
                if raw_input() != 'ignore':
                    sys.exit(1)

Branch.hooks.install_named_hook('pre_commit', run_tests, 'CVE Tracker tests')
EOM

Package Notes
-------------
Packages that share a codebase or are rebranded share the same vulnerabilities.
Eg:

firefox-3.0 and xulrunner 1.9 share codebase
iceweasel = rebranded firefox

Mass-update kernel team "released" tags to use correct rc versions:

	sed -i -e 's/^\(upstream_linux.*: released (2\.6\...\)-\(rc.*)\)/\1~\2/g' active/CVE-*


Triage Frequency
----------------
It is important to run check-cves on a regular basis. A suggested schedule is:

Monday:    check-cves nvdcve-2*.xml ; check-cves --import-missing-debian
Wednesday: check-cves allitems.xml ; check-cves --refresh nvdcve-*.xml
Friday:    check-cves nvdcve-recent.xml

MITRE is the definitive CVE database, but NVD contains the same information,
updates their database more frequently and also lists the PublicDate. Using the
above schedule allows our database to stay up-to-date with the NVD, and ensure
that our descriptions are also up to date.  For CVEs where the PublicDate is
wrong, we can override the value using the "CRD" field instead.

Since we want to keep a record of what the PublicDate was when the
USN published, USN publication should include the addition of the
"PublicDateAtUSN" field, which duplicates PublicDate, in case it changes
later. When answering the question "When did a CVE go public?" the
scripts will use CRD over PublicDateAtUSN over PublicDate. Times should
include the timezone; UTC is preferred. If no time is specified during
coordination, 14:00:00 UTC is recommended.

Eg: PublicDateAtUSN: 2009-03-27 23:45:00 UTC

Individual CVEs not in the databases yet can also be added with
scripts/active_edit. Be sure to use the '--embargoed' flag when adding CVEs
that are not public yet (and double check that it wasn't accidentally added
to the public tracker).

If you setup mitre_loc and nvd_loc in ~/.ubuntu-cve-tracker.conf, then this
can all be taken care of for you with:
$ ./scripts/process_cves
$ ./scripts/process_cves [Mon|Wed|Fri]

Specifying anything other than Mon, Wed, or Fri simply runs check-cves on
nvdcve-recent.xml.

Example ~/.ubuntu-cve-tracker.conf setup for the security team:
mitre_loc="people.canonical.com:/home/ubuntu-security/cache"
nvd_loc="people.canonical.com:/home/ubuntu-security/cache"

Sometimes MITRE is behind in getting CVEs into the database. It is recommended
that http://www.openwall.com/lists/oss-security/ also be reviewed for new CVE
assignments for packages in main.


Status Transitions
------------------
For stable releases, "scripts/sync-from-usns.py" should flip statuses to
"released". "scripts/sync-from-versions.py" can also be used for stable
releases when specifying '-r <release>', which might be useful for things
that are in universe but get new upstream versions.

For the devel release, only "scripts/sync-from-versions.py" should flip
statuses to "released". (The "--stable-pending" option will break this rule
if you need it to.)

The version (status comment) on a "pending" or "released" is considered
canonical. The USN database, however, will show when a fix was first
published to the -security pocket, and may have a later version.

Notes about what to do with kernel bugs and the tracker:

UCT			  	LP
---                  	        ------------
bug			->	lp (and if it's a dup, update to master)

When description has "Placeholder" only (i.e. new bug):
DNE			->	(New) -> Invalid
not-affected		->	(New) -> Invalid
pending			->	(New) -> Fix Committed
released		->	(New) -> Fix Released
Patches:...upstream:	->	"Break-Fix: BROKEN_SHA FIXED_SHA" (multiple)

Once alive:
* phase 1:
DNE			->	Invalid
pending			->	(Invalid,New,Confirmed,Triaged,In Progress)->Fix Committed
released		->	Fix Released
not-affected		->	(New)->Invalid
ignored			->	(New)->Invalid
[priority]		->	Importance
deferred			skipped (ie no processing)
* phase 2:
needed			<-	Confirmed/Triaged/In Progress
needs-triage		<-	New
not-affected		<-	Invalid (double-check with DNE)
Patches:...upstream:	<-	"Add-Break-Fix: SHA SHA" (multiple)
Patches:...upstream:	<-	"Del-Break-Fix: SHA SHA" (multiple)
deferred			skipped (ie no processing)
* phase 3:
needs-triage		->	New
Description:		->	Description with shas...
Patches:...upstream:	->	"Break-Fix: BROKEN_SHA FIXED_SHA" (multiple)

Closed bug that needs a SHA change:
	- create new bug
	- dup old bug to new bug

TODO/THINK-ABOUT:
	- sha1 introduced CVE, sha1 fixes it (if one sha1, needs another)
	- "Patches_linux" new identifier "break-fix" with introduced/fixed-by shas
		- "introduced-by" can be either sha or rc1 tag ("v2.6.35-rc1") or "-" (infinity)