Contents of /doc/narrative_introduction

        A Narrative Introduction to the Debian Security Tracker 


About
-----

Everything in the Debian Security Tracker is publicly available, as in
"Debian doesn't hide problems" available. 

The best thing about our tracking 'system' is that it is very basic.
There is no horrible overhead of web-based ticket/issue trackers, its
just a subversion repository and some text files that we
collaboratively edit and then some scripts to parse these files and
generate useful reports available online. Everything is designed to be
very simple to use, transparent and easy to see what other people are
working on so you can work on other things.

Gentle Introduction
--------------------

This following will give you a basic walk-through of how the files are
structured and how we do our work tracking issues. 

The best way to understand is to check out our repository from
subversion so you have the files on your computer and can follow along
at home. To do this you just need to do the following:

svn co svn+ssh://<alioth user name>@svn.debian.org/svn/secure-testing

This will check out the working repository (given that you already have
an alioth account and public key authentication already set up, see
http://wiki.debian.org/Alioth/SSH).  After successfully downloading,
you will have a new directory called secure-testing.  Inside this directory
are a number of subdirectories.  The data directory is where we do most of
our work.  Note that the name of the Subversion repository is historical;
the tracker is not specially related to testing-security, but for Debian
security at large.

If you don't have an Alioth account, you can create one at:

https://alioth.debian.org/account/register.php

You can then join the team by clicking the 'Request to join' link at:

https://alioth.debian.org/projects/secure-testing

If you don't need write access, you can of course check out our files
without an Alioth account as well:

svn co svn://svn.debian.org/svn/secure-testing

If you are a git fan, you can also use git-svn. Once you have the
git-svn package installed, you can clone the subversion repository into
your own local git repository with:

git svn clone svn+ssh://<alioth user account>@svn.debian.org/svn/secure-testing

Note that this will take a very long time (expect over two hours) since
every commit from the very beginning (over 12,000 at this point) is
checked out individually and merged into your git repository.

Subversion and git-svn Crash Course
-----------------------------------

The following table lists the most common/useful commands for working
with the secure-testing repository:

  subversion       | git-svn           | action
  -----------------+-------------------+------------------------------
  svn update       | git svn rebase    | sync your local repo from
                   |                   | remote secure-testing repo
  -----------------+-------------------+------------------------------
  svn commit       | git commit -a     | commit your changes to the
                   | git svn dcommit   | remote secure-testing repo
                   |                   | (note that 'git commit -a'
                   |                   | only updates your local repo)
  -----------------+-------------------+------------------------------
  svn diff         | git diff          | compare your local repo to
                   |                   | remote secure-testing repo
  -----------------+-------------------+------------------------------

Automatic Issue Updates
-----------------------

Twice a day a cronjob runs that pulls down the latest full CVE lists
from Mitre, this automatically gets checked into data/CVE/list, and
also syncs that file with other lists like data/DSA/list and
data/DTSA/list.

We get notified via either email
(http://lists.alioth.debian.org/mailman/listinfo/secure-testing-commits)
of every SVN commit
or via the KGB bot on #debian-security on OFTC. For example, the bot
will say in the channel:

17:14 < KGB-0> joeyh r21191 data/CVE/list * automatic update

Most of our work is taking the new issues that Mitre releases and
processing them so that the tracking data is correct. Read on for how we
do this.

Processing TODO entries
-----------------------

The Mitre update typically manifests in new CVE entries. So what we do
is to update our svn repository and then edit data/CVE/list and look
for new TODO entries. These will often be in blocks of 10-50 or so,
depending on how many new issues they have assigned.

IMPORTANT: make sure to read: 
http://lists.alioth.debian.org/pipermail/secure-testing-team/2009-May/002394.html

Issues NOT-FOR-US (NFU)
-----------------------

Processing entries is done by first seeing if the issue is related to any
software packaged in Debian. If it isn't a package in Debian and has no
ITP then you note that in the file with a 'NOT-FOR-US:' tag. Third-party
modules not yet packaged for Debian are also tagged as NFU; even if their
parent software is packaged for Debian. The module names should be
mentioned in the NFU note in order to make issues apparent if that module
should ever receive a propper package.  Another case are meta packages
that only provide a downloader (e.g. flashplugin-nonfree). There is no
way to mark such packages as we have no influence on the version and
technically the code is not present in Debian.

Example:

CVE-2005-3018 (Apple Safari allows remote attackers to cause a denial of
service ...)
   NOT-FOR-US: Safari

Before marking a package NOT-FOR-US, the following should be done:
    - Read the full CVE description to determine the product name
    - Search for the product using apt-cache search <name>
    - If a file was referenced, search for the file using
      apt-file search <name>
    - Search the wnpp list (http://www.debian.org/devel/wnpp/) to see
      if the product has an ITP or RFP (see "ITP/RFP packages" below)
    - Search the ftp-master removal list
      (http://ftp-master.debian.org/removals-full.txt) or the Package
      Tracking System (http://packages.qa.debian.org/) to see if the
      package was present in the past but was removed (see "Removed 
      packages" below)

If there is any doubt, add a NOTE with your findings and ask others to
double check.

There is a tool that helps with sorting out all the NOT-FOR-US issues: 
See "bin/check-new-issues -h". For the search functions in 
check-new-issues to work, you need to have unstable in your 
sources.list and have done "apt-get update" and "apt-file update". 
Having libterm-readline-gnu-perl installed helps, too. If you are not
running unstable, you can search at http://packages.debian.org or
set up an unstable chroot:

http://www.debian.org/doc/manuals/reference/ch09#_chroot_system
http://wiki.debian.org/Debootstrap

Undetermined Tags
-----------------

If you don't have time to fully research an issue, but it is abundantly
clear (via CVE text or other announcement) that the issue affects a
particular package or set of packages, the <undetermined> tag can be
used.  This has the advantage of entering the issue earlier in the
output of debsecan and on the pts pages, which is useful for the small
set of proactive maintainers paying attention to these information
sources.  Getting the maintainer involved hopefully prompts faster
fixes.  This also allows enables tracking of multiple packages, some
of which may already be fixed.  

<undetermined> can also be used when there simply is not enough
information disclosed in the existing known references about the
issue.  Essentially, <undetermined> indicates that someone needs
to come back and revisit the issue.  An example undetermined
entry is:

CVE-2011-2351 (Use-after-free vulnerability in Google Chrome before 12.0.742.112 ...)
        - chromium-browser 12.0.742.112~r90304-1
        - webkit <undetermined>
        NOTE: webkit commit #123456

The list of all of currently undetermined issues is aggregated at:
http://security-tracker.debian.org/tracker/status/undetermined

This is a good place for new contributors to get started since these
are issues that can be pruned quickly for new information that may
not have been known during the initial disclosure, and thus marked
<unfixed> for further work or closed with a version number.  Please
add notes if you do change an undetermined issue to unfixed (unless
you're also fixing the issue in the process, which is of course the
ideal way to help/contribute).

Issues in ITP and/or RFP packages
---------------------------------

If an issue is discovered in a package that has an RFP or ITP already filed,
then that is also noted in order to track the problem, and make sure it is
resolved before the package enters the archive.  These issues are marked with
the <itp> tag.  Note this includes both ITPs and RFPs since (from a security
tracking standpoint) there is no advantage in tracking them in separate ways.
An example entry for an ITP/RFP package is:

CVE-2004-2525 (Cross-site scripting (XSS) vulnerability in compat.php
in Serendipity ...)
        - serendipity <itp> (bug #312413)

Reserved entries
----------------

Several security problems have coordinated dates of public disclosure,
i.e. a CVE identifier has been assigned to a problem, but it's not
public yet. Also, several vendors have a pool of CVE ids they can
assign to problems that are detected in their products. Such entries
are marked as RESERVED in the tracker:

CVE-2005-1432
        RESERVED

Rejected entries
----------------

Sometimes there are CVE assignments that later turn out to be duplicates,
mistakes or non-issues. These items are reverted and turned into REJECTED
entries:

CVE-2005-4129
        REJECTED

Packages in the archive
-----------------------

If it is a package in Debian, look to see if the package is affected or 
not (sometimes newer versions that have the fixes have already been 
uploaded). 

If the version has been fixed already, note the package name and the
Debian version that fixes it and assign a severity level to it, for
example:

CVE-2005-2596 (User.php in Gallery, as used in Postnuke, allows users
with any Admin ...)
   - gallery 1.5-2 (medium)

Even if the CVE description mentions it is fixed as of a particular
version, double-check the Debian package yourself (because sometimes 
the CVE descriptions or information from databases like Secunia is 
incorrect).

If it hasn't been fixed, we determine if there has been a bug filed
about the issue, and if not, file one and then note it in the list
(again with a severity level):

CVE-2005-3054 (fopen_wrappers.c in PHP 4.4.0, and possibly other
versions, does not ...)
   - php4 <unfixed> (bug #353585; medium)
   - php5 <unfixed> (bug #353585; medium)

Bug numbers can be added as in the example above. To avoid duplicate bugs,
"bug filed" can be added instead of "bug #123456" when the bug report has
been sent but the bug number is not yet known (however, it is more 
desirable to file the bug, wait for the BTS to assign a number, then update 
the entry in the CVE list so that complete information is always available
in the tracker).  The bug number is important because it makes it clear
that the maintainer has been contacted about the problem, and that they are 
aware of their responsibility to work swiftly toward a fix.

Since CVEs often drop in bulk, submission of multiple CVEs in a single bug
report is permissable and encouraged.  However, some maintainers have 
indicated a preference for only one issue per bug report.  The following 
is a list of packages for which each CVE should be reported separately:
    - php5
    - libav

A special exception is made for kernel related issues. The kernel-sec group
will take care of them. It is not necessary to file bugs in the BTS for kernel
security issues, it only causes overhead.

If you want to report a bug, bin/report-vuln might be helpful in creating
the bug report.

If a vulnerability does not affect Debian, e.g. because the vulnerable
code is not contained, it is marked as <not-affected>:

CVE-2004-2628 (Multiple directory traversal vulnerabilities in thttpd 2.07 beta 0.4, ...)
        - thttpd <not-affected> (Windows-specific vulnerabilities)

<not-affected> is also used if a vulnerability was fixed before a
package was uploaded into the Debian archive.

Removed packages
----------------

Sometimes there are cases, where a vulnerability hasn't been fixed with
a code change, but simply by deciding that a package is that broken that
it needs to be removed from the archive entirely. This is tracked with
the <removed> tag:

CVE-2005-1435 (Open WebMail (OWM) before 2.51 20050430 allows remote authenticated ...)
        - openwebmail <removed>

Also note that it is sufficient to mark a package as removed in unstable.
The tracker is aware of which package is present in which distribution
and marks other distributions that still contain the package automagically
as unfixed.  For example, if libxml is in oldstable, but not stable or
unstable, then:

        - libxml <removed>

will track oldstable as affected, but stable and unstable as not-affected.

Once a package has been completely removed from all currently supported
debian releases, it should be tracked in the data/packages/removed-packages
file.  This file lists all packages (one source package per line) that were
at one time in a debian release, but no longer exist in any supported
version.  Additions to this file can be used to address failing consistency
checks after a new release.

Severity levels
---------------

These levels are mostly used to prioritize the order in which security
problems are resolved. Anyway, we have a rough overview on how you should
assess these levels. 

unimportant: This problem does not affect the Debian binary package, e.g.
             a vulnerable source file, which is not built, a vulnerable file
             in doc/foo/examples/, PHP Safe mode bugs, path disclosure (doesn't
             matter on Debian).
             All "non-issues in practice" fall also into this category, like
             issues only "exploitable" if the code in question is setuid root,
             exploits which only work if someone already has administrative
             privileges or similar.

low        : A security problem, which has only mild security implications
             (local DoS, /tmp file races and so on).

medium     : For anything which permits code execution after user interaction.
             Local privilege escalation vulnerabilities are in this category as
             well, or remote privilege escalation if it's constrained to the
             application (i.e. no shell access to the underlying system, such
             as simple cross-site scripting). Most remote DoS vulnerabilities
             fall into this category, too.

high       : A typical, exploitable security problem, which you'll really
             like to fix or at least implement a workaround. This could
             be because the vulnerable code is very broadly used, because
             an exploit is in the wild or because the attack vector is
             very wide. 
             Should be put into that category anything that permits an attacker
             to execute arbitrary code on the vulnerable system (with or
             without root privileges) and high-impact denial-of-service bugs
             (for instance, an IPv4 forwarding path vulnerability which
             requires only very few packets to exploit).
             Significant defects in security software can be rated "high" as
             well (for instance, a vulnerability in a piece of cryptographic
             software which flags forged digital signatures as genuine).


Certain packages may get higher or lower rating than usual, based on
their importance.


NOTE and TODO entries
---------------------

There are many instances where more work has to be done to determine
if something is affected, and you might not be able to do this at the
time. These entries can have their TODO line changed to something
descriptive so that it is clear what remains to be done. For example:

CVE-2005-3990 (Directory traversal vulnerability in FastJar 0.93
allows remote ...)
        TODO: check, whether fastjar from the gcc source packages is affected

It is also useful to add information to issues as you find it, so that
when others go to look at an issue and want to know why you marked it
as you did, or need a reference, it will be there. The more
information left, the better. For example, the following entry lets
you know that CVE-2005-3258 doesn't affect the squid that we have
because the issue was introduced in a patch that was never applied to
the Debian package:

CVE-2005-3258 (The rfc1738_do_escape function in ftp.c for Squid 2.5
STABLE11 and ...)
        - squid <not-affected> (bug #334882; medium)
        NOTE: Bug was introduced in a patch to squid-2.5.STABLE10,
        NOTE: this patch was never applied to the Debian package.

CVE assignments
---------------

Debian can only assign CVE names from its own pool for issues which
are not public.  To request a CVE from the Debian pool, write to
<team@security.debian.org> and include a description which follows CVE
conventions.  To request a CVE for public issues, write to Mitre and
possibly to the moderated oss-security list.  In the meantime, you can
add an entry of the form

CVE-2009-XXXX [optipng array overflow]
        - optipng 0.6.2.1-1 (low)
        NOTE: http://secunia.com/advisories/34035/

in the data/CVE/list file.  It is desirable to include references
which uniquely identify the issue, such as a permanent link to an
entry in the upstream bug tracker, or a bug in the Debian BTS.  If the
issue is likely present in unstable, a bug should be filed to help the
maintainer to track it.

Lack of CVE entries should not block advisory publication which are
otherwise ready, but we should strieve to release fully
cross-referenced advisories nevertheless.

Distribution tags
-----------------

Our data is primarily targeted at sid, as we track the version that
a certain issue was fixed in sid. The Security Tracker web site (see
below) derives information about the applicability of a vulnerability
to stable and oldstable from the list of DSAs issued by the security
team and the fact that a source package is part of a release.
Distribution tags can be used to denote information about a vulnerability
for the version of a package in a specific release. An example:

CVE-2005-3974 (Drupal 4.5.0 through 4.5.5 and 4.6.0 through 4.6.3, when running on ...)
        - drupal 4.5.6-1 (low)
        [sarge] - drupal <not-affected> (Only vulnerable if running PHP 5)

Drupal has been fixed since 4.5.6, however Drupal from Sarge still isn't
vulnerable as the vulnerability is only effective when run under PHP 5,
which isn't part of Sarge.

When a vulnerability is fixed in (oldstable-)proposed-updates, it is added
to next-(oldstable-)point-update.txt and only added to CVE/list after the
point release (during which the no-dsa entry is removed).

Generated Reports
-----------------

All of this tracking information gets automatically parsed and
compared against madison to determine what has been fixed and what is
still waiting, this results in this website:

http://security-tracker.debian.org/

It incorporates package lists and parses distribution lists and can
thus be used to
- Present the security history of a package
- Provide overviews of vulnerable packages in stable, testing, sid and
  oldstable (it still has some false positives, wrt packages in
  stable that are present in stable, but not vulnerable, these need to
  be triaged individually).
- Generate a list of packages that are subject to security problems, but
  stuck in testing migration due to problems with the dependency chain
  and thus candidates for a DTSA
- Generate a list of TODO issues that need to be addressed
- Generate a list of packages that will enter Debian soon and need to
  be checked for security problems
- Generate a list of provisional IDs that need to be turned into proper
  CVE entries
- Show some potential problems in the data pool (e.g. misspelled package
  names not found in the packages list, or potentially missing epochs)

For every security problem it displays
- The CVE information
- A severity assessment by NVD
- Cross references to DTSAs, DSAs and bugs in the BTS
- The status of a security problem in stable, oldstable, testing and sid
- Additional notes from our tracker

The DSA list
------------

We maintain a list of all DSA advisories issued by the stable security
team. This information is used to derive information about the state
of security problems for the stable and oldstable distribution. An
entry for a DSA looks like this:

[21 Nov 2005] DSA-903-1 unzip - race condition
        {CVE-2005-2475}
        [woody] - unzip 5.50-1woody4
        [sarge] - unzip 5.52-1sarge2
        NOTE: fixed in testing at time of DSA

The first line tracks the date, when a DSA was issued, the DSA
identifier, the affected source package and the type of vulnerability.
The second line performs a cross-reference to the entry in CVE/list
that maintains the state of the vulnerability in sid. Every entry that
is added like this to DSA/list is parsed by a script and automatically
added to CVE/list.  The next lines contain the fixes for stable and
optionally oldstable, addressed with distribution tags.  You may add
NOTE: entries freely, we use a NOTE entry for statistical purposes
that tracks, when a fix has reached testing relative to the time when
it hit stable.

There is no need to add anything to CVE/list for a DSA, the DSA
cross-reference will be added automatically by the cron job. However,
you do need to add [lenny] or [squeeze] entries to CVE/list when there
is a 'no-dsa' or 'not-affected' condition.

Checking in your changes
------------------------

After thoroughly researching each issue (as described above) and editing
the relevant files, commit your changes. Peer review is done via the 
mailing list and IRC notifications (see "Automatic Issue Updates" above).
However, changes to the tracker website itself (e.g. the files in lib/*
and bin/tracker_service.py) should be vetted and approved before being
committed. The preferred way to do this is to send a patch to: 
debian-security-tracker@lists.debian.org mailing list.

Commits are checked for syntax errors before they are actually committed,
and you'll receive an error and your commit is aborted if it is in error.
To check your changes yourself beforehand, use "make check-syntax" from
the root of the svn directory.

Following up on security issues
-------------------------------

By simply loading this page and doing a little gardening of the
different issues many things can be done. One thing is that you can
read all the bug reports of each issue and see if new information has
been added to the end that might provide updated or changed
information (such as if an issue has been closed, or a version of the
package has been uploaded that contains the fix). It is also useful to
follow-up on the issues to prod the maintainer to deal with the issue,
which they may have forgotten about.


Tracking of security bugs in the BTS and linking them to a user tag by CVE
--------------------------------------------------------------------------

There's an automated tagging of security-related bugs to CVE IDs through
the user tag security for the user debian-security@lists.debian.org

All bugs added to the tracker are automatically tagged. You can use
the search
http://bugs.debian.org/cgi-bin/pkgreport.cgi?tag=security;users=debian-security@lists.debian.org;exclude=tracked 
to find all bugs not yet present in the tracker.

All bug numbers added to the tracked are automatically associated
to the relevant user tag.

If you checked an issue which doesn't need to be added to the tracked
(e.g. because it's not security-relevant or otherwise bogus you can either
remove the security tag from the bugs or send a mail to control@bugs.debian.org
with the following content:

user debian-security@lists.debian.org
usertag $BUGNUM + tracked

IRC Channel
-----------

We hang-out on #debian-security on OFTC, stop by the IRC channel if
you'd like, also we can add you to the alioth project so you have svn
write permission and you can test drive it on the testing issues for
however long you like to get an idea or feel comfortable (and hey it
helps!)