[secure-testing] Log of /lib/python

Revision 2135 - Directory Listing
Modified Fri Sep 23 12:45:15 2005 UTC (7 years, 7 months ago) by fw

lib/python/security_db.py (DB.getBinaryPackageVersions):
  Include architecture information in the result.

Revision 2132 - Directory Listing
Modified Fri Sep 23 11:29:09 2005 UTC (7 years, 7 months ago) by fw

lib/python/bugs.py (FileBase.__iter__):
  Assign "unimportant" urgency to not-affected packages.
lib/python/security_db.py (DB.getBugsForSourcePackage,
  DB.getBugsForBinaryPackage):
  Filter out unimportant bugs.
(DB.getNonBugsForBinaryPackage):
  New.

Revision 2118 - Directory Listing
Modified Fri Sep 23 07:55:31 2005 UTC (7 years, 7 months ago) by fw

lib/python/bugs.py (PackageNoteParsed.__iter__):
  Permit "bug filed" in package notes.
data/README:
  Document it.

Revision 2101 - Directory Listing
Modified Thu Sep 22 21:05:28 2005 UTC (7 years, 7 months ago) by fw

lib/python/bugs.py (FileBase.__iter__):
  Accept "- PACKAGE not-affected" (i.e. without comment).

Revision 2088 - Directory Listing
Modified Thu Sep 22 12:00:31 2005 UTC (7 years, 7 months ago) by fw

lib/python/security_db.py (DB.initSchema):
  Add removed_packages table.
(DB.readRemovedPackages, DB.getUnknownPackages):
  New.

bin/update-db:
  Read removed packages.

data/packages/removed-packages:
  New file.

Revision 2084 - Directory Listing
Modified Thu Sep 22 10:52:10 2005 UTC (7 years, 7 months ago) by fw

lib/python/security_db.py (DB.__init__):
  Create user-defined functions at an earlier point, before the schema.
(DB.initSchema):
  Add testing_status view.

Revision 2079 - Directory Listing
Modified Thu Sep 22 10:04:07 2005 UTC (7 years, 7 months ago) by fw

Remove cruft from the database schema.  Make status tracking
explicitly per-bug and not per-package note.

lib/python/security_db.py (DB.__init__):
  Bump schema version.
(DB.initSchema):
  In tables source_package_status and binary_package_status,
  make (bug_name, package) the primary key.  Remove the note
  and release columns.
(DB.calculateVulnerabilities):
  Update accordingly.  Remove code which sets the dead variables
  package_by_release and binary_package_status.
(DB._calcUnstable, DB._calcTesting):
  Update.
(DB.getSourcePackages, DB.getBinaryPackages, DB.getBugsForBinaryPackage,
DB.getBugsForSourcePackage):
  Update to new database schema.
(DB.getBugXrefs):
  New method, which works with xrefs in both directions.

Revision 2072 - Directory Listing
Modified Wed Sep 21 17:46:59 2005 UTC (7 years, 7 months ago) by fw

Add "FIXES:" and "FIXED-BY:" directives.

lib/python/bugs.py (PackageNote):
  New attribute "bug_origin".
(PackageNote.writeDB):
  No longer skipr writing when self.id has been set (so that writeDB
  can be used for cloning notes).  Write the bug_origin attribute.
(PackageNoteFromDB):
  Read the bug_origin attribute.
(BugBase):
  Initialize the xref_fixes and xref_fixedby attributes.
(BugBase.writeDB):
  Write them.
(Bug):
  Pass through xref_fixes and xref_fixedby in constructor.
(BugFroMDB):
  Load them.
(FileBase):
  New regexps re_xref_fixes_required, re_xref_fixes,
  re_xref_fixedby_required, re_xref_fixedby.
(FileBase.__iter__):
  Record FIXES: and FIXED-BY:.

lib/python/security_db.py (DB):
  Bump schema version.
(DB.initSchema):
  Add bug_origin column to the packages_notes table.
  Add copy_notes column to bugs_xref.
(DB.readBugs):
  Remove incremental reading.  Add new code that copies package notes,
  as requested by the FIXES: and FIXED-BY: directives.

Revision 2071 - Directory Listing
Modified Wed Sep 21 15:15:42 2005 UTC (7 years, 7 months ago) by fw

Make (bug_name, package, release) unique in the package_notes table.
This is necessary because otherwise, the version tracking code does
not work right.  We do not lose any data by doing this; package status
was already tracked by bug and not by package note.

lib/python/bugs.py (PackageNote.merge, Bug.mergeNotes):
  New.
(CANFile.finishBugs, CVEFile.finishBugs, DSAFile.finishBugs):
  New.  Merge package notes for CAN, CVE and DSA files.

lib/python/security_db.py (DB):
  Bump schema version.
(DB.initSchema):
  Add the UNIQUE index mentioned above.

lib/python/debian_support.py (mergeAsSets):
  New.
(test):
  Test cases for mergeAsSets.

Revision 2067 - Directory Listing
Modified Wed Sep 21 08:24:06 2005 UTC (7 years, 7 months ago) by fw

lib/python/security_db.py (DB.calculateVulnerabilities):
  Check that there is a single source package for a binary package
  annotation.

Revision 2064 - Directory Listing
Modified Wed Sep 21 07:27:30 2005 UTC (7 years, 7 months ago) by fw

lib/python/security_db.py (DB):
  Remove the unused nicknames table and attribute.  (No schema version
  bump is required because a table is removed, and no other changes
  are being made.)

Revision 2063 - Directory Listing
Modified Wed Sep 21 06:38:05 2005 UTC (7 years, 7 months ago) by fw

lib/python/security_db.py (DB._calcUnstable):
  Distinguish between "not vulnerable" (we have a fixed version) and
 "not known to be vulnerable" (no matching version annotation).

Revision 2062 - Directory Listing
Modified Wed Sep 21 06:08:51 2005 UTC (7 years, 7 months ago) by fw

lib/python/security_db.py (DB.calculateVulnerabilities):
  Factor out testing and unstable code.
(DB._calcUnstable):
  New.  Mostly unchanged from the verison in calculateVulnerabilities.
(DB._calcTesting):
  Rewritten from scratch.  Now works on source packages.  Should be
  more reliable.

Revision 2058 - Directory Listing
Modified Tue Sep 20 17:59:09 2005 UTC (7 years, 8 months ago) by fw

lib/python/bugs.py (FileBase.finishBug):
  New method to apply last-minute checks to bug objects.
(FileBase.__iter__):
  Use it.
(DSAFile.finishBug):
  Override, to enforce the "etch" tag.

Revision 2055 - Directory Listing
Modified Tue Sep 20 11:49:39 2005 UTC (7 years, 8 months ago) by fw

lib/python/security_db.py (DB.calculateVulnerabilities):
  Calculate unstable status.
(DB.isBinaryPackage):
  Actually return result of query.

Revision 2045 - Directory Listing
Modified Mon Sep 19 15:12:37 2005 UTC (7 years, 8 months ago) by fw

Fix bug in the version comparison algorithm.

lib/python/debian_support.py (letterValue):
  New global variable.
(Version.__parse_1):
  Use it to sort letters before non-letters.
(test):
  New test case.

Revision 2040 - Directory Listing
Modified Sun Sep 18 13:15:48 2005 UTC (7 years, 8 months ago) by fw

lib/python/security_db.py (DB._initFunctions):
  Introduce collation function for Debian versions.
(DB.getBinaryPackagesForSource, DB.getSourcePackages,
DB.getBinaryPackages):
  Use it.

Revision 2035 - Directory Listing
Modified Sat Sep 17 22:41:49 2005 UTC (7 years, 8 months ago) by fw

data/python/security_db.py (DB.getSourcePackageVersions,
DB.getBinaryPackageVersions, DB.getBinaryPackagesForSource,
DB.getBugsFromDebianBug, DB.isSourcePackage, DB.isBinaryPackage,
DB.getBugsForSourcePackage, DB.getBugsForBinaryPackage, DB.getTODOs):
  New methods.

Revision 2034 - Directory Listing
Modified Sat Sep 17 22:38:40 2005 UTC (7 years, 8 months ago) by fw

data/security.db (BugFromDB):
  Load the bug even when an alias name is used (CAN vs. CVE, DSA
  without revision number).

Revision 2031 - Directory Listing
Modified Sat Sep 17 13:19:32 2005 UTC (7 years, 8 months ago) by fw

lib/python/security_db.py (DB._readBinaryPackages):
  Skip reading entirely if there are no changes.

Revision 2030 - Directory Listing
Modified Sat Sep 17 13:02:49 2005 UTC (7 years, 8 months ago) by fw

lib/python/bugs.py (BugFromDB.getDebianBugs):
  Optimize SELECT statement.
(BugFromDB.getSourcePackages, BugFromDB.getBinaryPackages):
  Move to security_db.DB.

lib/python/security_db.py (DB._initFunctions):
  Register user-defined SQLite functions.
(DB.getSourcePackages, DB.getBinaryPackages):
  Renamed from bugs.BugFromDB.  Implement aggregation.

Revision 2025 - Directory Listing
Modified Fri Sep 16 19:52:34 2005 UTC (7 years, 8 months ago) by fw

lib/python/security_db.py (DB._parseFile):
  Record architectures.
(DB._readSourcePackages):
  Update accordingly.
(DB._readBinaryPackages):
  Use "Architecture: all" if possible, to cut down the database size a
  bit.
(DB.calculateVulnerabilities):
  Synthesize source package status from binary package annotations.

Revision 2024 - Directory Listing
Modified Fri Sep 16 18:30:15 2005 UTC (7 years, 8 months ago) by fw

CAN-2002-1767: Fix spelling.

lib/python/bugs.py (FileBase.re_not_for_us_required):
  Catch more typos.

Revision 2015 - Directory Listing
Modified Fri Sep 16 08:14:54 2005 UTC (7 years, 8 months ago) by fw

lib/python/security_db.py (DB):
  Update schema versioning code.  Replace table bugs_status with
  bug_status.  Add bug_name and release columns to source_package_status
  and binary_package_status.
(DB.calculateVulnerabilities):
  First attempt at bug status calculation.  It's rather broken,
  unfortunately.

lib/python/bugs.py (BugFromDB.getStatus):
  New method, to get the results of the bug status calculation.

Revision 2005 - Directory Listing
Modified Thu Sep 15 15:40:40 2005 UTC (7 years, 8 months ago) by fw

lib/python/bugs.py (BugFromDB.getSourcePackages):
  Add.  Needs more work.
(BugFromDB.getBinaryPackages):
  Sort by version (and package name).  This makes the output somewhat
  more tolerable.

Revision 2004 - Directory Listing
Modified Thu Sep 15 15:39:01 2005 UTC (7 years, 8 months ago) by fw

lib/python/security_db.py (DB.calculateVulnerabilities):
  Use INSERT OR REPLACE to simplify the code considerably.

Revision 2001 - Directory Listing
Modified Thu Sep 15 15:05:18 2005 UTC (7 years, 8 months ago) by fw

lib/python/security_db.py (DB):
  Add source_version_id column to binary_packages table.
(DB._updateVersions):
  Update source_version_id, too.
(DB.calculateVulnerabilities):
  Add code to determine vulnerable binary packages, based on their
  source packages.

lib/python/bugs.py (BugFromDB.getBinaryPackages):
  New method.  Needs more work.
(test):
  Fix.

Revision 1997 - Directory Listing
Modified Thu Sep 15 11:51:26 2005 UTC (7 years, 8 months ago) by fw

Keep track of advisory release dates so that we can generate links to
Debian advisories.

lib/python/security_db.py (DB):
  Change database schema: Add release_date column to bugs table.

lib/python/security_db.py (BugBase, BugFromDB):
  Add date attribute.

Revision 1996 - Directory Listing
Modified Thu Sep 15 10:41:24 2005 UTC (7 years, 8 months ago) by fw

Record whether a package note refers to a source or binary package.

lib/python/security_db.py (DB):
  Upgrade schema.  Add package_kind column to package_notes.
(DB.calculateVulnerabilities):
  Update and use package_kind.

lib/python/bugs.py (PackageNote, PackageNoteFromDB, BugFromDB):
  Add package_kind attribute.

Revision 1994 - Directory Listing
Modified Thu Sep 15 10:11:44 2005 UTC (7 years, 8 months ago) by fw

Implement bin/update-db, to update the database with a single command.
Most processing is skipped if no input files have been modified.

lib/python/security_db.py (SchemaMismatch):
  New exception.
(DB):
  Handle schema versioning.
(DB.initSchema):
  Add subrelease column to source_packages and binary_packages.
  Set user_version.
  Remove stray commit.
(DB._parseFile):
  Return information to the caller if the file is unchanged.
(DB.readPackages):
  Move deletion code to callees.
(DB._readSourcePackages, DB._readBinaryPackages):
  Implement incremental updates.  Add subrelease.
  Need to invoke _clearVersions if any changes are made.
(DB.deleteBugs, DB.finishBugs):
  Moved into readBugs.
(DB.insertBugs):
  Rename ...
(DB.readBugs):
  ... to this one.  Implement incremental updates.
  Invoke _clearVersions if necessary.
(DB._clearVersions):
  Add.
(DB._updateVersions):
  Skip processing if _clearVersions has not been invoked.
(DB.getVersion, DB.releaseContainsPackage, DB._synthesizeReleases):
  Obsolete, remove.
(test):
  Update.

lib/python/bugs.py (CANFile, CVEFile):
  Split into two classes, which handle the differences between the two
  files.

bin/check-syntax:
  Update accordingly.

bin/update-db:
  New database update script.  Implements incremental updates.

Makefile:
  Remove references to bin/update-packages.  Simplify drastically.

Revision 1987 - Directory Listing
Modified Wed Sep 14 21:26:09 2005 UTC (7 years, 8 months ago) by fw

lib/python/security_db.py (DB.calculateVulnerabilities):
  No version information means "unfixed", not "fixed".

Revision 1983 - Directory Listing
Modified Wed Sep 14 20:48:54 2005 UTC (7 years, 8 months ago) by fw

This change groups binary packages for different architectures, as long
as they have the same version.  Synthesis of testing/stable/oldstable
has been disabled (and will be replaced with new code).

lib/python/security_db.py (mergeLists):
  New helper functions to deal with architecture lists.
(DB.initSchema):
  Add parsed column to inodeprints table.
  Update source_packages and binary_packages table.
  Split package_status into source_package_status and
  binary_package_status.
(DB.updateSources, DB.updatePackages):
  Remove and replace with ...
(DB.readPackages):
  ... new method.
(DB.availableReleases, DB._updateVersions):
  Adjust to new schema.
(DB.calculateVulnerabilities):
  Disable synthesis of testing etc.
  Adjust to new schema.
  Fix bug in large INSERT INTO statements: Need t.release = p.release
  instead of t.release = n.release.
  Add return to disable bug-specific rating code.
(test):
  Update.

lib/python/bugs.py (PackageNote.releaseStatus):
  Rename ...
(PackageNote.sourceStatus):
  ... and make it specific to source packages.
(BugFromDB.getDebianBugs):
  New method to get a list of Debian bug numbers.

bin/update-packages (import):
  Automatically reads the correct directory.

Makefile (stamps/packages-imported):
  Update accordingly.

Revision 1977 - Directory Listing
Modified Wed Sep 14 13:27:19 2005 UTC (7 years, 8 months ago) by fw

Ongoing work to implement version tracking.  The current approach does
not scale with the number of architectures, though.

lib/python/security_db.py (DB):
  Add nicknames member.
(DB.initSchema):
  Add nicknames and package_status tables.
  Add index package_notes_bug.
(DB._synthesizeReleases):
  New method to build testing etc. distributions.
(DB.calculateVulnerabilities):
  Update to use new tables.  Return list of problems detected.

lib/python/bugs.py (PackageNote.releaseStatus):
  New method to check for affected releases.

bin/update-vulnerabilities:
  Print list of detected problems.

Revision 1973 - Directory Listing
Modified Wed Sep 14 08:59:50 2005 UTC (7 years, 8 months ago) by fw

Add table version_linear_order, which will enable us to make version
comparisons in pure SQL.

lib/python/security_db.py (DB):
  Add verbose flag to constructor.
(DB.initSchema):
  Add table version_linear_order.  Add version ID fields to tables
  package_notes, source_packages, binary_packages.
(DB._maybeUpdate):
  Use self.verbose.
(DB.availableReleases):
  New method to get a least of releases in the database.
(DB._updateVersions):
  Calculate a linear order of versions.  This will be used to speed up
  the vulnerability rating process.
(DB.calculateVulnerabilities):
  Invoke _updateVersions.  Use self.verbose.

bin/update-vulnerabilities, bin/update-packages:
  Set database verbose flag.

Revision 1969 - Directory Listing
Modified Tue Sep 13 21:27:08 2005 UTC (7 years, 8 months ago) by fw

Ignore bugs which apply to packages which no longer exist.  With proper
package annotations (future patch, needs release hints), this allows us
to ignore kernel bugs which are no longer relevant.

lib/python/security_db.py (DB.releaseContainsPackage):
  New.
(DB.calculateVulnerabilities):
  Vulnerabilities which do not apply to any package are no longer
  relevant.  Ignore not-for-us DSAs.
(test):
  Update.

Revision 1968 - Directory Listing
Modified Tue Sep 13 21:21:44 2005 UTC (7 years, 8 months ago) by fw

Add a crude check to discover kernel-only bug reports.  (I found a
better way to deal with this situation, though.)

lib/python/bugs.py (PackageNote.affectsKernel, BugBase.isKernelOnly):
  Add.

Revision 1952 - Directory Listing
Modified Tue Sep 13 14:43:16 2005 UTC (7 years, 8 months ago) by fw

Add support for release annotations of the form "[woody] - PACKAGE
VERSION".  (Currently not used in any input file.)

lib/python/bugs.py (PackageNote.writeDB):
  Convert release objects to strings.
(PackageNoteParsed):
  Pass in release information.  Do not extract it from the
  package-specific notes.
(FileBase.re_package_required, FileBase.re_package):
  Detect release annotations.
(FileBase.__iter__):
  Extract them.
(DSAFile):
  Moved "!" hack to base class.

Revision 1951 - Directory Listing
Modified Tue Sep 13 14:08:22 2005 UTC (7 years, 8 months ago) by fw

First step towards calculating sets of vulnerable packages.  This is
currently directed towards testing (but does not yet process the
secure-testing archive).

A new table is added, so "make clean" is required.

The remaining problem (besides potential bugs in the code) is how to
deal with kernel updates, IOW how to detect them and ignore them.

bin/update-vulnerabilities:
  New script, updates the bugs_status table.

lib/python/bugs.py (PackageNote.affects):
  Fix all kinds of errors.  The code never ran before, it seems. 8-/
(PackageNote.fixedVersion):
  Add.
(BugBase.hasTODO):
  Add.
(BugReservedCVE, BugRejectedCVE):
  Mark as not-for-us.
(FileBase.rawRecords):
  Mark all un-annotated bugs after STOP: field as not-for-us.

lib/python/security_db.py (DB.initSchema):
  Add table bugs_status.
(DB.finishBugs):
  Run to completion even if there are conflicting CAN/CVE entries.
(DB.getVersion, calculateVulnerabilities):
  New methods.
(test):
  Update.

lib/python/debian_support.py (Version):
  Add a type check.

Makefile:
  Add stamps/calc-vulns target.

bin/update-packages:
  Fix typo in comment.

Revision 1947 - Directory Listing
Modified Tue Sep 13 09:12:19 2005 UTC (7 years, 8 months ago) by fw

Add support for downloading Packages and Sources files.

After this change, you must run "make clean update-packages all" to
download the package data.

lib/python/security_db.py(DB.initSchema):
  Add table inodeprints.
(DB.filePrint, DB._maybeUpdate, DB.maybeUpdateSources,
DB.maybeUpdatePackages):
  Add.
(test):
  Update to new file locations.

bin/update-packages:
  New file.  Used by the makefile.

data/packages:
  New directory.  Used to store the downloaded files.

Makefile:
  Add "update-packages" and "clean" targets, and the necessary targets
  to support them.

Revision 1944 - Directory Listing
Modified Tue Sep 13 08:00:21 2005 UTC (7 years, 8 months ago) by fw

lib/python/security_db.py:
  Rename "subrelease" to "archive" (although the purpose was initially
  different).  Update test case.

Revision 1943 - Directory Listing
Modified Tue Sep 13 07:45:17 2005 UTC (7 years, 8 months ago) by fw

bin/apt-update-file:
  Remove unnecessary import.

lib/python/debian_support.py (listReleases):
  Add "sid".
(replaceFile):
  Remove temporary file on exception.
(updateFile):
  The file constructor raises IOError if the file does not exist.
  urllib does not raise a proper exception on 4xx errors.
  Handle varying whitespace in SHA1-Current field.

Revision 1939 - Directory Listing
Modified Mon Sep 12 20:08:46 2005 UTC (7 years, 8 months ago) by fw

lib/python/debian_support.py:
  Add support for downloading package file diffs.

bin/apt-update-file:
  Driver script for the new functionality.

(I will use this functionality to implement package database
replication.  The goal is to keep a local copy of all the interesting
data, so that we no longer need to consult madison etc.)

Revision 1937 - Directory Listing
Modified Mon Sep 12 17:12:08 2005 UTC (7 years, 8 months ago) by fw

lib/python/debian_support.py (ParseError):
  Add class.

lib/python/debian_support.py (PackageFile.raiseSyntaxError):o
  Raise ParseError instead of SyntaxError.

bin/check-syntax, bin/update-bug-list-db:
  Handle the ParseError exception gracefully.

lib/python/bugs.py (CVEFile.matchHeader):
  Check parentheses/brackets.

data/CAN/list:
  Fix uncovered syntax errors.

Revision 1936 - Directory Listing
Modified Mon Sep 12 16:46:36 2005 UTC (7 years, 8 months ago) by fw

lib/python/security_db.py (DB.finishBugs):
   Fix reporting of consistency check failures.

bin/update-bug-list-db:
   Do not print "error: " prefixes.

Revision 1934 - Directory Listing
Added Mon Sep 12 16:32:23 2005 UTC (7 years, 8 months ago) by fw

Add list parser written in Python.

"make check" runs a syntax check (no SQLite required).  "make all"
updates the SQLite database, and performs cross-list consistency checks.

There is some support for loading Debian Package/Sources files, but this
information is currently not used by the checks.

	ViewVC Help
Powered by ViewVC 1.1.5