[secure-testing] Log of /lib

Revision 11971 - Directory Listing
Modified Sun May 24 20:22:36 2009 UTC (3 years, 11 months ago) by fw

lib/python/debian_support.py (updateFile): fix the fix

Also support both the hashlib and sha modules.

Revision 10680 - Directory Listing
Modified Fri Dec 12 19:39:29 2008 UTC (4 years, 5 months ago) by fw

lib/python/debian_support.py: Handle failure to download patches

If the data is inconsistent, use a non-diffed download.

Revision 10579 - Directory Listing
Modified Tue Dec 2 21:13:06 2008 UTC (4 years, 5 months ago) by fw

lib/python/debian_support.py: Normalize version numbers before comparison

In theory, this allows us to use the data for unstable for volatile
and backports.org.  However, more testing is required if this is
indeed effective, and volatile does not actually use the version
number scheme assumed in this change.

Revision 10567 - Directory Listing
Modified Mon Dec 1 19:05:26 2008 UTC (4 years, 5 months ago) by fw

lib/python/web_support.py: Include server port in generated URLs

This allows us to run the service on a non-default port.

Revision 10566 - Directory Listing
Modified Mon Dec 1 19:05:22 2008 UTC (4 years, 5 months ago) by fw

lib/python/web_support.py: one more typing fix for Python 2.5

We suddenly have to deal with additional Unicode strings.

Revision 10465 - Directory Listing
Modified Sun Nov 23 18:07:13 2008 UTC (4 years, 5 months ago) by fw

lib/python/bugs.py: Replace integer type asserts to with int conversion

The reason is that even in Python 2.5, numeric objects fluctuate
between int and long types.

Revision 7720 - Directory Listing
Modified Tue Dec 25 17:55:56 2007 UTC (5 years, 4 months ago) by fw

Rejected CVEs with package notes are no longer an error

Without this change, an update from the CVE database may result
in failed consistency checks, which is not desirable.

In a later commit, the web front end will be extended to list
such CVE entries.  This will provide interested parties with
a means to perform cleanups.

Revision 7718 - Directory Listing
Modified Tue Dec 25 17:55:50 2007 UTC (5 years, 4 months ago) by fw

CVE no longer uses separate names for candidates

This means that we can simplify bugs.BugBase.cveStatus a bit.

Revision 7715 - Directory Listing
Modified Tue Dec 25 16:57:24 2007 UTC (5 years, 4 months ago) by fw

Remove unused methods

The following methods are removed:

	bugs.PackageNote.affects
	bugs.PackageNote.affectsKernel
	bugs.PackageNote.fixedVersion
	bugs.PackageNote.sourceStatus
	bugs.BugBase.hasTODO
	bugs.BugBase.isKernelOnly

Revision 6994 - Directory Listing
Modified Wed Oct 17 10:15:30 2007 UTC (5 years, 7 months ago) by fw

* lib/python/debian_support.py: Remove fallback for missing python-apt

These days, we need the APT algorithm, not the one described in
policy.  Requiring python-apt leads to a clear error message up front,
instead of an obscure one much later in the process.

Revision 6061 - Directory Listing
Modified Tue Jun 26 12:15:46 2007 UTC (5 years, 10 months ago) by fw

* lib/python/nvd.py:
  Make runnable as a script, for testing purposes.
  (Parser.characters): Keep whole node string, not just the last part.

Revision 5989 - Directory Listing
Modified Sun Jun 10 12:08:53 2007 UTC (5 years, 11 months ago) by fw

* lib/python/bugs.py (PackageNote, BugBase, Bug):
  Losen type checks for string arguments, to support Unicode strings.

Revision 5988 - Directory Listing
Modified Sun Jun 10 12:08:52 2007 UTC (5 years, 11 months ago) by fw

* lib/python/security_db.py (DB._parseFile):
  Store pickled data as blob in the SQLite database.

Revision 5987 - Directory Listing
Modified Sun Jun 10 12:08:43 2007 UTC (5 years, 11 months ago) by fw

* lib/python/debian_support.py (Version):
  Support Unicode strings by converting them to UTF-8.

Revision 5986 - Directory Listing
Modified Sun Jun 10 12:08:38 2007 UTC (5 years, 11 months ago) by fw

* lib/python/security_db.py (DB.__del__):
  Add destructor to close the SQLite database object explicitly.

Revision 5785 - Directory Listing
Modified Sat May 5 08:58:05 2007 UTC (6 years ago) by fw

* lib/python/debian_support.py (PackageFile.re_field):
  Field names are not necessarily separated by a space from the
  field content.

Revision 5668 - Directory Listing
Modified Wed Apr 18 20:00:39 2007 UTC (6 years, 1 month ago) by fw

Use a separate file, data/packages/removed-packages, to list source
packages which are no longer present in the archive.

* lib/python/security_db.py
  (DB.readBugs.do_parse): Ignore duplicate packages.
  (DB.readBugs): Treat removed-packages as yet another input file.
  (DB.readRemovedPackages): Resurrect method.

Revision 5665 - Directory Listing
Modified Wed Apr 18 17:42:01 2007 UTC (6 years, 1 month ago) by fw

* lib/python/security_db.py (DB.calculateDebsecan):
  No longer create version 0 debsecan data for woody.

Revision 5639 - Directory Listing
Modified Mon Apr 9 11:27:01 2007 UTC (6 years, 1 month ago) by fw

* lib/python/security_db.py (DB._initViews):
  Adjust SQL for the testing_status view to what's actually
  in the database.

Revision 5632 - Directory Listing
Modified Fri Apr 6 10:36:58 2007 UTC (6 years, 1 month ago) by fw

After the release of etch, the DTSA file will contain historic
entries for etch, and new ones for lenny.  Our previous automatic
tagging of all entries as etch does not work anymore.  Hence,
we make the release indicator explicit.

* lib/python/bugs.py (DTSAFile.finishBug):
  Verify that a release has been specified.  No longer default to
  "etch".

* data/DTSA/list
  Mark all entries as etch.

Revision 5470 - Directory Listing
Modified Sun Feb 18 16:50:24 2007 UTC (6 years, 3 months ago) by fw

Treat packages marked as <removed> as source packages.
This allows us to track firefox issues again.

* lib/python/bugs.py (FileBase):
  Keep track of removed packages.

* lib/python/security_db.py
  (DB.readBugs): Populate removed_packages table using <removed> entries.
  (DB.readRemovedPackages): Remove method.

* bin/update-db:
  Do not call readRemovedPackages anymore.

Revision 5103 - Directory Listing
Modified Sun Dec 10 18:36:34 2006 UTC (6 years, 5 months ago) by fw

Various changes to switch from FAKE- to TEMP- prefixes

(Yeah, less than stellar engineering that this isn't concentrated
in a single place.)

Revision 5101 - Directory Listing
Modified Sun Dec 10 17:44:41 2006 UTC (6 years, 5 months ago) by fw

lib/python/security_db.py (DB.getBugsForBinaryPackage,
    DB.getBugsForSourcePackage):
  Ignore bugs in woody.

Revision 5100 - Directory Listing
Modified Sun Dec 10 17:35:42 2006 UTC (6 years, 5 months ago) by fw

Add a no-dsa filter to the "testing" web page

lib/python/security_db.py (DB._initViews):
  Add no_dsa column to the testing_status table.

bin/tracker_service.py (TrackerService.page_status_release_testing):
  Use it to implement the filter in the same way as for "stable".

Revision 4773 - Directory Listing
Modified Wed Sep 27 18:13:34 2006 UTC (6 years, 7 months ago) by fw

* lib/python/security_db.py (DB._parseFile):
  Relax regexp for source versions, to support the new ~ syntax.

Revision 4738 - Directory Listing
Modified Sun Sep 17 20:54:22 2006 UTC (6 years, 8 months ago) by fw

Do not enforce version ordering between different suites.
The archive currently violates this constraint.

Revision 4236 - Directory Listing
Modified Thu Jun 15 18:17:18 2006 UTC (6 years, 11 months ago) by fw

* lib/python/debian_support.py
(Version): Implement using apt_pkg if APT is available.
(version_compare): Add.

* lib/python/security_db.py
(DB._initFunctions): Directly invoke debian_support.version_compare.

Revision 4110 - Directory Listing
Modified Fri Jun 2 04:03:24 2006 UTC (6 years, 11 months ago) by fw

* lib/python/bugs.py (FileBase.__iter__):
  Fix name generation for bugs involving <no-dsa> notes.

Revision 4005 - Directory Listing
Modified Sat May 20 17:08:37 2006 UTC (7 years ago) by fw

* lib/python/security_db.py (DB.getTODOs):
  Add hide_check parameter.

* bin/tracker_service.py (TrackerService.page_status_todo):
  Use it.

Revision 3858 - Directory Listing
Modified Sun Apr 23 12:46:41 2006 UTC (7 years, 1 month ago) by fw

Add support for no-dsa handling to the database.  This is still
preliminary.

lib/python/bugs.py (PackageNoteNoDSA):
  New class.
(BugBase.__init__):
  Losen assert to include PackageNoteNoDSA.
(FileBase.__iter__):
  Create PackageNoteNoDSA objects.

lib/python/security_db.py (DB):
  Bump schema version to 21.  Add package_notes_nodsa table.
  Add schema migration code.
(DB.readBugs):
  Clear package_notes_nodsa table.

Revision 3856 - Directory Listing
Modified Sun Apr 23 12:37:55 2006 UTC (7 years, 1 month ago) by fw

lib/python/web_support.py (URLFactory.updateParamsDict):
  New method.
(URLFactory.updateParams):
  Implement using updateParamsDict.

Revision 3179 - Directory Listing
Modified Thu Dec 29 20:47:30 2005 UTC (7 years, 4 months ago) by fw

lib/python/security_db.py (DB.calculateDebsecan1):
  Record versions of source packages from binary packages, too,
  to include versions from binary-only NMUs.  This is expected
  to fix Debian bug #345158, reported against debsecan.

Revision 3147 - Directory Listing
Modified Sat Dec 24 09:37:23 2005 UTC (7 years, 4 months ago) by fw

lib/python/security_db.py (DB.calculateDebsecan1):
  Only include CVE-* and FAKE-* vulnerabilities in version 1 data.
  (The other data is redundant anyway, and no unstable vulnerability
  status information is available.)

Revision 3145 - Directory Listing
Modified Fri Dec 23 22:35:45 2005 UTC (7 years, 5 months ago) by fw

lib/python/security_db.py (DB.calculateDebsecan1):
  Existence of a release-specific fix means that sid was vulnerable at
  some point (this is central to our tracking model).

Revision 3133 - Directory Listing
Modified Fri Dec 23 15:39:13 2005 UTC (7 years, 5 months ago) by fw

lib/python/security_db.py (DB.calculateDebsecan1):
  Yet another fix for urgency calculation.  Blecch.

Revision 3132 - Directory Listing
Modified Fri Dec 23 15:22:49 2005 UTC (7 years, 5 months ago) by fw

lib/python/security_db.py (DB.calculateDebsecan1):
  Optimize vulnerability list.  Further tweaks to urgency handling:
  'unimportant' does not trump 'unknown'.

Revision 3131 - Directory Listing
Modified Fri Dec 23 14:10:50 2005 UTC (7 years, 5 months ago) by fw

lib/python/security_db.py (DB.calculateDebsecan1):
  Rework maximum urgency calculation.  "unknown" no longer
  overrides other urgencies.

Revision 3129 - Directory Listing
Modified Fri Dec 23 13:15:25 2005 UTC (7 years, 5 months ago) by fw

lib/python/security_db.py (DB.initSchema):
  Add index on package_notes(package) (no schema version bump needed).
(DB.calculateDebsecan0):
  Renamed from DB.calculateDebsecan.
(DB.calculateDebsecan1):
  New method which generates version 1 format (with pinning support
  wtc.).
(DB.calculateDebsecan):
  Invokes both the version 0 and version 1 methods.

bin/update-db:
  Adjust accordingly.

Revision 3122 - Directory Listing
Modified Thu Dec 22 10:19:06 2005 UTC (7 years, 5 months ago) by fw

lib/python/security_db.py (DB.calculateDebsecan):
  Check that a fixed package is actually available in sid, and do not
  trust the list files.

Revision 3078 - Directory Listing
Modified Sat Dec 17 11:17:21 2005 UTC (7 years, 5 months ago) by fw

Store CVE descriptions in the nvd_data table.  Enable incremental
NVD updates.

lib/python/security_db.py (DB):
  Bump schema version.  Add cve_desc column to the nvd_data table.
(DB.updateNVD):
  New method.

bin/update-nvd:
  If the -i option is specified, use updateNVD instead of replaceNVD.

lib/python/nvd.py (_Parser):
  Add new member variable path.
(_Parser.characters):
  New method.
(_Parser.endElement):
  Store cve_desc variable.

bin/tracker_service.py (TrackerService.page_bug):
  Use NVD description if available.

Revision 3069 - Directory Listing
Modified Fri Dec 16 09:23:57 2005 UTC (7 years, 5 months ago) by fw

lib/python/security_db.py (DB):
  Bump schema version.  Add oldstable_status view.

bin/tracker_service.py (TrackerService):
  Add oldstable page.

Revision 3051 - Directory Listing
Modified Thu Dec 15 11:37:40 2005 UTC (7 years, 5 months ago) by fw

lib/python/security_db.py (DB):
  Bump schema version.
(DB.initSchema):
  Add debsecan_data table.
(DB.calculateDebsecan, DB.getDebsecan):
  New methods.

bin/update-db:
  Invoke calculateDebsecan.

bin/tracker_service.py (TrackerService):
  Add support for debsecan/* pages.
(TrackerService.page_debsecan):
  New method.

Revision 2554 - Directory Listing
Modified Mon Oct 24 14:43:04 2005 UTC (7 years, 6 months ago) by fw

Add a summary page for tracked bugs without a CVE name.

lib/python/security_db.py (DB.getFakeBugs):
  New method.

bin/tracker_service.py (TrackerService.page_data_fake_names):
  New method.
(TrackerService.__intit__, TrackerService.page_home):
  Use it.

Revision 2509 - Directory Listing
Modified Thu Oct 20 12:24:28 2005 UTC (7 years, 7 months ago) by fw

lib/python/web_support.py (make_pre);
  Fix bug which caused make_pre to include only the last element of
  the passed list in its output.
(__test);
  Regression test.

Revision 2507 - Directory Listing
Modified Thu Oct 20 12:20:09 2005 UTC (7 years, 7 months ago) by fw

lib/python/security_db.py (DB.readBugs):
  Add code to copy notes from DSAs to CVEs.
  Correctly handle <unfixed> notes.

Revision 2497 - Directory Listing
Modified Thu Oct 20 10:55:10 2005 UTC (7 years, 7 months ago) by fw

Remove support for FIXES: and FIXED-BY:.  Instead, automatically copy
notes from DTSA to CVE if there is a cross-reference.

The copying code is updated so that it can handle conflicting
annotations.  If there is a conflict, the later version wins.

lib/python/bugs.py (BugBase, Bug):
  Remove xref_fixes and xref_fixedby members.
(FileBase):
  Remove FIXES:/FIXED-BY: regexps and corresponding code.

lib/python/security_db.py (DB):
  Bump schema version.
(DB.initSchema):
  Remove normalized_target and copy_notes field from the
  bugs_xref table.
(DB.readBugs):
  No need to maintain the normalized_target column anymore.
  Update the code which copies the notes.  The code is much simpler
  now because we do not handle recursive copies.
(DB.getBugXrefs):
  Use target instead of normalized_target.

Revision 2490 - Directory Listing
Modified Thu Oct 20 09:04:01 2005 UTC (7 years, 7 months ago) by fw

 r772@deneb:  fw | 2005-10-20 10:41:31 +0200
 lib/python/web_support.py (URLFactory.updateParams):
   New method.
 (charToHTML, charToHTMLattr):
   Replaces stringToHTML.

Revision 2488 - Directory Listing
Modified Thu Oct 20 09:03:39 2005 UTC (7 years, 7 months ago) by fw

 r638@deneb:  fw | 2005-10-14 15:43:12 +0200
 bin/tracker_service.py (TrackerService.page_home):
   Document external interfaces.
 (TrackerService.page_bug):
   Add NVD references.
 (TrackerService.page_status_release_stable,
  TrackerService.page_status_release_testing):
   Show NVD remote attack range if present.
 (TrackerService.url_nvd, TrackerService.make_nvd_ref):
   New.
 
 lib/python/security_db.py (NVDEntry):
   New class.
 (DB.initSchema):
   New nvd_data table.  Update stable_status and testing_status views.
 (DB.replaceNVD, DB.getNVD):
   New methods.
 
 bin/update-nvd, lib/python/nvd.py:
   New files.

Revision 2487 - Directory Listing
Modified Thu Oct 20 09:03:27 2005 UTC (7 years, 7 months ago) by fw

 r637@deneb:  fw | 2005-10-14 15:38:48 +0200
 lib/python/web_support.py (EM):
   New.
 (CODE):
   Accept multiple arguments.

Revision 2486 - Directory Listing
Modified Thu Oct 20 09:03:14 2005 UTC (7 years, 7 months ago) by fw

 r629@deneb:  fw | 2005-10-14 11:01:25 +0200
 lib/python/security_db.py (DB._calcTesting):
   Change so that it works for stable as well.
 (DB.calculateVulnerabilities):
   Invoke _calcTesting for both stable and testing.

Revision 2484 - Directory Listing
Modified Thu Oct 20 09:02:46 2005 UTC (7 years, 7 months ago) by fw

 r627@deneb:  fw | 2005-10-14 10:33:41 +0200
 lib/python/web_support.py (RedirectResult):
   Support non-permanent redirections.

Revision 2483 - Directory Listing
Modified Thu Oct 20 09:02:32 2005 UTC (7 years, 7 months ago) by fw

 r626@deneb:  fw | 2005-10-14 10:33:18 +0200
 lib/python/security_db.py (DB.refresh):
   Reinitialize user-defined functions if the database is reopened.

Revision 2482 - Directory Listing
Modified Thu Oct 20 09:02:12 2005 UTC (7 years, 7 months ago) by fw

 r614@deneb:  fw | 2005-10-13 22:12:28 +0200
 Add new web front end.
 
 bin/tracker_service.py, lib/python/web_support.py:
   New files.

Revision 2479 - Directory Listing
Modified Thu Oct 20 09:01:34 2005 UTC (7 years, 7 months ago) by fw

 r611@deneb:  fw | 2005-10-13 22:01:11 +0200
 lib/python/bugs.py (BugFromDB.__init__):
   Fix bogus result for DSA lookup.

Revision 2478 - Directory Listing
Modified Thu Oct 20 09:01:15 2005 UTC (7 years, 7 months ago) by fw

 r304@deneb:  fw | 2005-10-01 11:19:27 +0200
 Add overview page for the stable suite.
 
 bin/tracker.cgi (print_stable_status):
   New function.
 lib/python/security_db.py (DB.initSchema):
   Add stable_status view.

Revision 2477 - Directory Listing
Modified Thu Oct 20 09:00:51 2005 UTC (7 years, 7 months ago) by fw

 r262@deneb:  fw | 2005-09-29 21:08:13 +0200
 lib/python/bugs.py (Bugs.mergeNotes):
   Deal with "None" releases in the sorting code.

Revision 2476 - Directory Listing
Modified Thu Oct 20 08:57:39 2005 UTC (7 years, 7 months ago) by fw

 r261@deneb:  fw | 2005-09-29 21:07:03 +0200
 lib/python/security_db.py (DB.readBugs):
   Fix typo.
 (DB.calculateVulnerabilities):
   No version number checks between etch and unstable.

Revision 2475 - Directory Listing
Modified Thu Oct 20 08:28:24 2005 UTC (7 years, 7 months ago) by fw

First round of updates for the CAN -> CVE transition.

lib/python/bugs.py (CANFile):
  Remove, merge with CVEFile.
(BugBase.re_cve_name, BugBase.__init__, FileBase.re_*,
 FileBase.__iter__):
  Remove CAN-specific parts.
(test):
  Update.

lib/python/security_db.py (DB.readBugs):
  Do not read CAN/list.  No longer normalize cross-references
(test):
  Update.

Revision 2178 - Directory Listing
Modified Sun Sep 25 17:55:07 2005 UTC (7 years, 7 months ago) by fw

lib/python/bugs.py (FileBase):
  Make package note regexps more strict.
(FileBase.rawRecords):
  Patch in the new-format "NOT-FOR-US:" entries for old entries.

Revision 2156 - Directory Listing
Modified Sat Sep 24 15:43:04 2005 UTC (7 years, 7 months ago) by fw

Implement NOT-FOR-US:, <unfixed>, <not-affected> and <itp>.

lib/python/bugs.py (PackageNote.writeDB):
  Store package_kind attribute in the database, so that we can detect
  notes for ITPs.
(PackageNoteParsed.__init__):
  'unfixed' is no longer a valid tag.
(FileBase):
  Update regular expressions.
(FileBase.__iter__):
  Parse new "- PACKAGE <TAG>" annotations.  Implement NOT-FOR-US:.
  Caclulate the Debian bug number for a FAKE name when it is needed.

lib/python/security_db.py (DB.calculateVulnerabilities):
  Do not overrite "itp" values in the package_kind column of
  package_notes.  Check that ITPed packages are not present in the
  archive.
(DB.getITPs):
  New method.

Revision 2135 - Directory Listing
Modified Fri Sep 23 12:45:15 2005 UTC (7 years, 8 months ago) by fw

lib/python/security_db.py (DB.getBinaryPackageVersions):
  Include architecture information in the result.

Revision 2132 - Directory Listing
Modified Fri Sep 23 11:29:09 2005 UTC (7 years, 8 months ago) by fw

lib/python/bugs.py (FileBase.__iter__):
  Assign "unimportant" urgency to not-affected packages.
lib/python/security_db.py (DB.getBugsForSourcePackage,
  DB.getBugsForBinaryPackage):
  Filter out unimportant bugs.
(DB.getNonBugsForBinaryPackage):
  New.

Revision 2118 - Directory Listing
Modified Fri Sep 23 07:55:31 2005 UTC (7 years, 8 months ago) by fw

lib/python/bugs.py (PackageNoteParsed.__iter__):
  Permit "bug filed" in package notes.
data/README:
  Document it.

Revision 2101 - Directory Listing
Modified Thu Sep 22 21:05:28 2005 UTC (7 years, 8 months ago) by fw

lib/python/bugs.py (FileBase.__iter__):
  Accept "- PACKAGE not-affected" (i.e. without comment).

Revision 2088 - Directory Listing
Modified Thu Sep 22 12:00:31 2005 UTC (7 years, 8 months ago) by fw

lib/python/security_db.py (DB.initSchema):
  Add removed_packages table.
(DB.readRemovedPackages, DB.getUnknownPackages):
  New.

bin/update-db:
  Read removed packages.

data/packages/removed-packages:
  New file.

Revision 2084 - Directory Listing
Modified Thu Sep 22 10:52:10 2005 UTC (7 years, 8 months ago) by fw

lib/python/security_db.py (DB.__init__):
  Create user-defined functions at an earlier point, before the schema.
(DB.initSchema):
  Add testing_status view.

Revision 2079 - Directory Listing
Modified Thu Sep 22 10:04:07 2005 UTC (7 years, 8 months ago) by fw

Remove cruft from the database schema.  Make status tracking
explicitly per-bug and not per-package note.

lib/python/security_db.py (DB.__init__):
  Bump schema version.
(DB.initSchema):
  In tables source_package_status and binary_package_status,
  make (bug_name, package) the primary key.  Remove the note
  and release columns.
(DB.calculateVulnerabilities):
  Update accordingly.  Remove code which sets the dead variables
  package_by_release and binary_package_status.
(DB._calcUnstable, DB._calcTesting):
  Update.
(DB.getSourcePackages, DB.getBinaryPackages, DB.getBugsForBinaryPackage,
DB.getBugsForSourcePackage):
  Update to new database schema.
(DB.getBugXrefs):
  New method, which works with xrefs in both directions.

Revision 2072 - Directory Listing
Modified Wed Sep 21 17:46:59 2005 UTC (7 years, 8 months ago) by fw

Add "FIXES:" and "FIXED-BY:" directives.

lib/python/bugs.py (PackageNote):
  New attribute "bug_origin".
(PackageNote.writeDB):
  No longer skipr writing when self.id has been set (so that writeDB
  can be used for cloning notes).  Write the bug_origin attribute.
(PackageNoteFromDB):
  Read the bug_origin attribute.
(BugBase):
  Initialize the xref_fixes and xref_fixedby attributes.
(BugBase.writeDB):
  Write them.
(Bug):
  Pass through xref_fixes and xref_fixedby in constructor.
(BugFroMDB):
  Load them.
(FileBase):
  New regexps re_xref_fixes_required, re_xref_fixes,
  re_xref_fixedby_required, re_xref_fixedby.
(FileBase.__iter__):
  Record FIXES: and FIXED-BY:.

lib/python/security_db.py (DB):
  Bump schema version.
(DB.initSchema):
  Add bug_origin column to the packages_notes table.
  Add copy_notes column to bugs_xref.
(DB.readBugs):
  Remove incremental reading.  Add new code that copies package notes,
  as requested by the FIXES: and FIXED-BY: directives.

Revision 2071 - Directory Listing
Modified Wed Sep 21 15:15:42 2005 UTC (7 years, 8 months ago) by fw

Make (bug_name, package, release) unique in the package_notes table.
This is necessary because otherwise, the version tracking code does
not work right.  We do not lose any data by doing this; package status
was already tracked by bug and not by package note.

lib/python/bugs.py (PackageNote.merge, Bug.mergeNotes):
  New.
(CANFile.finishBugs, CVEFile.finishBugs, DSAFile.finishBugs):
  New.  Merge package notes for CAN, CVE and DSA files.

lib/python/security_db.py (DB):
  Bump schema version.
(DB.initSchema):
  Add the UNIQUE index mentioned above.

lib/python/debian_support.py (mergeAsSets):
  New.
(test):
  Test cases for mergeAsSets.

Revision 2067 - Directory Listing
Modified Wed Sep 21 08:24:06 2005 UTC (7 years, 8 months ago) by fw

lib/python/security_db.py (DB.calculateVulnerabilities):
  Check that there is a single source package for a binary package
  annotation.

Revision 2064 - Directory Listing
Modified Wed Sep 21 07:27:30 2005 UTC (7 years, 8 months ago) by fw

lib/python/security_db.py (DB):
  Remove the unused nicknames table and attribute.  (No schema version
  bump is required because a table is removed, and no other changes
  are being made.)

Revision 2063 - Directory Listing
Modified Wed Sep 21 06:38:05 2005 UTC (7 years, 8 months ago) by fw

lib/python/security_db.py (DB._calcUnstable):
  Distinguish between "not vulnerable" (we have a fixed version) and
 "not known to be vulnerable" (no matching version annotation).

Revision 2062 - Directory Listing
Modified Wed Sep 21 06:08:51 2005 UTC (7 years, 8 months ago) by fw

lib/python/security_db.py (DB.calculateVulnerabilities):
  Factor out testing and unstable code.
(DB._calcUnstable):
  New.  Mostly unchanged from the verison in calculateVulnerabilities.
(DB._calcTesting):
  Rewritten from scratch.  Now works on source packages.  Should be
  more reliable.

Revision 2058 - Directory Listing
Modified Tue Sep 20 17:59:09 2005 UTC (7 years, 8 months ago) by fw

lib/python/bugs.py (FileBase.finishBug):
  New method to apply last-minute checks to bug objects.
(FileBase.__iter__):
  Use it.
(DSAFile.finishBug):
  Override, to enforce the "etch" tag.

Revision 2055 - Directory Listing
Modified Tue Sep 20 11:49:39 2005 UTC (7 years, 8 months ago) by fw

lib/python/security_db.py (DB.calculateVulnerabilities):
  Calculate unstable status.
(DB.isBinaryPackage):
  Actually return result of query.

Revision 2045 - Directory Listing
Modified Mon Sep 19 15:12:37 2005 UTC (7 years, 8 months ago) by fw

Fix bug in the version comparison algorithm.

lib/python/debian_support.py (letterValue):
  New global variable.
(Version.__parse_1):
  Use it to sort letters before non-letters.
(test):
  New test case.

Revision 2040 - Directory Listing
Modified Sun Sep 18 13:15:48 2005 UTC (7 years, 8 months ago) by fw

lib/python/security_db.py (DB._initFunctions):
  Introduce collation function for Debian versions.
(DB.getBinaryPackagesForSource, DB.getSourcePackages,
DB.getBinaryPackages):
  Use it.

Revision 2035 - Directory Listing
Modified Sat Sep 17 22:41:49 2005 UTC (7 years, 8 months ago) by fw

data/python/security_db.py (DB.getSourcePackageVersions,
DB.getBinaryPackageVersions, DB.getBinaryPackagesForSource,
DB.getBugsFromDebianBug, DB.isSourcePackage, DB.isBinaryPackage,
DB.getBugsForSourcePackage, DB.getBugsForBinaryPackage, DB.getTODOs):
  New methods.

Revision 2034 - Directory Listing
Modified Sat Sep 17 22:38:40 2005 UTC (7 years, 8 months ago) by fw

data/security.db (BugFromDB):
  Load the bug even when an alias name is used (CAN vs. CVE, DSA
  without revision number).

Revision 2031 - Directory Listing
Modified Sat Sep 17 13:19:32 2005 UTC (7 years, 8 months ago) by fw

lib/python/security_db.py (DB._readBinaryPackages):
  Skip reading entirely if there are no changes.

Revision 2030 - Directory Listing
Modified Sat Sep 17 13:02:49 2005 UTC (7 years, 8 months ago) by fw

lib/python/bugs.py (BugFromDB.getDebianBugs):
  Optimize SELECT statement.
(BugFromDB.getSourcePackages, BugFromDB.getBinaryPackages):
  Move to security_db.DB.

lib/python/security_db.py (DB._initFunctions):
  Register user-defined SQLite functions.
(DB.getSourcePackages, DB.getBinaryPackages):
  Renamed from bugs.BugFromDB.  Implement aggregation.

Revision 2025 - Directory Listing
Modified Fri Sep 16 19:52:34 2005 UTC (7 years, 8 months ago) by fw

lib/python/security_db.py (DB._parseFile):
  Record architectures.
(DB._readSourcePackages):
  Update accordingly.
(DB._readBinaryPackages):
  Use "Architecture: all" if possible, to cut down the database size a
  bit.
(DB.calculateVulnerabilities):
  Synthesize source package status from binary package annotations.

Revision 2024 - Directory Listing
Modified Fri Sep 16 18:30:15 2005 UTC (7 years, 8 months ago) by fw

CAN-2002-1767: Fix spelling.

lib/python/bugs.py (FileBase.re_not_for_us_required):
  Catch more typos.

Revision 2015 - Directory Listing
Modified Fri Sep 16 08:14:54 2005 UTC (7 years, 8 months ago) by fw

lib/python/security_db.py (DB):
  Update schema versioning code.  Replace table bugs_status with
  bug_status.  Add bug_name and release columns to source_package_status
  and binary_package_status.
(DB.calculateVulnerabilities):
  First attempt at bug status calculation.  It's rather broken,
  unfortunately.

lib/python/bugs.py (BugFromDB.getStatus):
  New method, to get the results of the bug status calculation.

Revision 2005 - Directory Listing
Modified Thu Sep 15 15:40:40 2005 UTC (7 years, 8 months ago) by fw

lib/python/bugs.py (BugFromDB.getSourcePackages):
  Add.  Needs more work.
(BugFromDB.getBinaryPackages):
  Sort by version (and package name).  This makes the output somewhat
  more tolerable.

Revision 2004 - Directory Listing
Modified Thu Sep 15 15:39:01 2005 UTC (7 years, 8 months ago) by fw

lib/python/security_db.py (DB.calculateVulnerabilities):
  Use INSERT OR REPLACE to simplify the code considerably.

Revision 2001 - Directory Listing
Modified Thu Sep 15 15:05:18 2005 UTC (7 years, 8 months ago) by fw

lib/python/security_db.py (DB):
  Add source_version_id column to binary_packages table.
(DB._updateVersions):
  Update source_version_id, too.
(DB.calculateVulnerabilities):
  Add code to determine vulnerable binary packages, based on their
  source packages.

lib/python/bugs.py (BugFromDB.getBinaryPackages):
  New method.  Needs more work.
(test):
  Fix.

Revision 1997 - Directory Listing
Modified Thu Sep 15 11:51:26 2005 UTC (7 years, 8 months ago) by fw

Keep track of advisory release dates so that we can generate links to
Debian advisories.

lib/python/security_db.py (DB):
  Change database schema: Add release_date column to bugs table.

lib/python/security_db.py (BugBase, BugFromDB):
  Add date attribute.

Revision 1996 - Directory Listing
Modified Thu Sep 15 10:41:24 2005 UTC (7 years, 8 months ago) by fw

Record whether a package note refers to a source or binary package.

lib/python/security_db.py (DB):
  Upgrade schema.  Add package_kind column to package_notes.
(DB.calculateVulnerabilities):
  Update and use package_kind.

lib/python/bugs.py (PackageNote, PackageNoteFromDB, BugFromDB):
  Add package_kind attribute.

Revision 1994 - Directory Listing
Modified Thu Sep 15 10:11:44 2005 UTC (7 years, 8 months ago) by fw

Implement bin/update-db, to update the database with a single command.
Most processing is skipped if no input files have been modified.

lib/python/security_db.py (SchemaMismatch):
  New exception.
(DB):
  Handle schema versioning.
(DB.initSchema):
  Add subrelease column to source_packages and binary_packages.
  Set user_version.
  Remove stray commit.
(DB._parseFile):
  Return information to the caller if the file is unchanged.
(DB.readPackages):
  Move deletion code to callees.
(DB._readSourcePackages, DB._readBinaryPackages):
  Implement incremental updates.  Add subrelease.
  Need to invoke _clearVersions if any changes are made.
(DB.deleteBugs, DB.finishBugs):
  Moved into readBugs.
(DB.insertBugs):
  Rename ...
(DB.readBugs):
  ... to this one.  Implement incremental updates.
  Invoke _clearVersions if necessary.
(DB._clearVersions):
  Add.
(DB._updateVersions):
  Skip processing if _clearVersions has not been invoked.
(DB.getVersion, DB.releaseContainsPackage, DB._synthesizeReleases):
  Obsolete, remove.
(test):
  Update.

lib/python/bugs.py (CANFile, CVEFile):
  Split into two classes, which handle the differences between the two
  files.

bin/check-syntax:
  Update accordingly.

bin/update-db:
  New database update script.  Implements incremental updates.

Makefile:
  Remove references to bin/update-packages.  Simplify drastically.

Revision 1987 - Directory Listing
Modified Wed Sep 14 21:26:09 2005 UTC (7 years, 8 months ago) by fw

lib/python/security_db.py (DB.calculateVulnerabilities):
  No version information means "unfixed", not "fixed".

Revision 1983 - Directory Listing
Modified Wed Sep 14 20:48:54 2005 UTC (7 years, 8 months ago) by fw

This change groups binary packages for different architectures, as long
as they have the same version.  Synthesis of testing/stable/oldstable
has been disabled (and will be replaced with new code).

lib/python/security_db.py (mergeLists):
  New helper functions to deal with architecture lists.
(DB.initSchema):
  Add parsed column to inodeprints table.
  Update source_packages and binary_packages table.
  Split package_status into source_package_status and
  binary_package_status.
(DB.updateSources, DB.updatePackages):
  Remove and replace with ...
(DB.readPackages):
  ... new method.
(DB.availableReleases, DB._updateVersions):
  Adjust to new schema.
(DB.calculateVulnerabilities):
  Disable synthesis of testing etc.
  Adjust to new schema.
  Fix bug in large INSERT INTO statements: Need t.release = p.release
  instead of t.release = n.release.
  Add return to disable bug-specific rating code.
(test):
  Update.

lib/python/bugs.py (PackageNote.releaseStatus):
  Rename ...
(PackageNote.sourceStatus):
  ... and make it specific to source packages.
(BugFromDB.getDebianBugs):
  New method to get a list of Debian bug numbers.

bin/update-packages (import):
  Automatically reads the correct directory.

Makefile (stamps/packages-imported):
  Update accordingly.

Revision 1977 - Directory Listing
Modified Wed Sep 14 13:27:19 2005 UTC (7 years, 8 months ago) by fw

Ongoing work to implement version tracking.  The current approach does
not scale with the number of architectures, though.

lib/python/security_db.py (DB):
  Add nicknames member.
(DB.initSchema):
  Add nicknames and package_status tables.
  Add index package_notes_bug.
(DB._synthesizeReleases):
  New method to build testing etc. distributions.
(DB.calculateVulnerabilities):
  Update to use new tables.  Return list of problems detected.

lib/python/bugs.py (PackageNote.releaseStatus):
  New method to check for affected releases.

bin/update-vulnerabilities:
  Print list of detected problems.

Revision 1973 - Directory Listing
Modified Wed Sep 14 08:59:50 2005 UTC (7 years, 8 months ago) by fw

Add table version_linear_order, which will enable us to make version
comparisons in pure SQL.

lib/python/security_db.py (DB):
  Add verbose flag to constructor.
(DB.initSchema):
  Add table version_linear_order.  Add version ID fields to tables
  package_notes, source_packages, binary_packages.
(DB._maybeUpdate):
  Use self.verbose.
(DB.availableReleases):
  New method to get a least of releases in the database.
(DB._updateVersions):
  Calculate a linear order of versions.  This will be used to speed up
  the vulnerability rating process.
(DB.calculateVulnerabilities):
  Invoke _updateVersions.  Use self.verbose.

bin/update-vulnerabilities, bin/update-packages:
  Set database verbose flag.

Revision 1969 - Directory Listing
Modified Tue Sep 13 21:27:08 2005 UTC (7 years, 8 months ago) by fw

Ignore bugs which apply to packages which no longer exist.  With proper
package annotations (future patch, needs release hints), this allows us
to ignore kernel bugs which are no longer relevant.

lib/python/security_db.py (DB.releaseContainsPackage):
  New.
(DB.calculateVulnerabilities):
  Vulnerabilities which do not apply to any package are no longer
  relevant.  Ignore not-for-us DSAs.
(test):
  Update.

Revision 1968 - Directory Listing
Modified Tue Sep 13 21:21:44 2005 UTC (7 years, 8 months ago) by fw

Add a crude check to discover kernel-only bug reports.  (I found a
better way to deal with this situation, though.)

lib/python/bugs.py (PackageNote.affectsKernel, BugBase.isKernelOnly):
  Add.

Revision 1952 - Directory Listing
Modified Tue Sep 13 14:43:16 2005 UTC (7 years, 8 months ago) by fw

Add support for release annotations of the form "[woody] - PACKAGE
VERSION".  (Currently not used in any input file.)

lib/python/bugs.py (PackageNote.writeDB):
  Convert release objects to strings.
(PackageNoteParsed):
  Pass in release information.  Do not extract it from the
  package-specific notes.
(FileBase.re_package_required, FileBase.re_package):
  Detect release annotations.
(FileBase.__iter__):
  Extract them.
(DSAFile):
  Moved "!" hack to base class.

Revision 1951 - Directory Listing
Modified Tue Sep 13 14:08:22 2005 UTC (7 years, 8 months ago) by fw

First step towards calculating sets of vulnerable packages.  This is
currently directed towards testing (but does not yet process the
secure-testing archive).

A new table is added, so "make clean" is required.

The remaining problem (besides potential bugs in the code) is how to
deal with kernel updates, IOW how to detect them and ignore them.

bin/update-vulnerabilities:
  New script, updates the bugs_status table.

lib/python/bugs.py (PackageNote.affects):
  Fix all kinds of errors.  The code never ran before, it seems. 8-/
(PackageNote.fixedVersion):
  Add.
(BugBase.hasTODO):
  Add.
(BugReservedCVE, BugRejectedCVE):
  Mark as not-for-us.
(FileBase.rawRecords):
  Mark all un-annotated bugs after STOP: field as not-for-us.

lib/python/security_db.py (DB.initSchema):
  Add table bugs_status.
(DB.finishBugs):
  Run to completion even if there are conflicting CAN/CVE entries.
(DB.getVersion, calculateVulnerabilities):
  New methods.
(test):
  Update.

lib/python/debian_support.py (Version):
  Add a type check.

Makefile:
  Add stamps/calc-vulns target.

bin/update-packages:
  Fix typo in comment.

Revision 1947 - Directory Listing
Modified Tue Sep 13 09:12:19 2005 UTC (7 years, 8 months ago) by fw

Add support for downloading Packages and Sources files.

After this change, you must run "make clean update-packages all" to
download the package data.

lib/python/security_db.py(DB.initSchema):
  Add table inodeprints.
(DB.filePrint, DB._maybeUpdate, DB.maybeUpdateSources,
DB.maybeUpdatePackages):
  Add.
(test):
  Update to new file locations.

bin/update-packages:
  New file.  Used by the makefile.

data/packages:
  New directory.  Used to store the downloaded files.

Makefile:
  Add "update-packages" and "clean" targets, and the necessary targets
  to support them.

Revision 1944 - Directory Listing
Modified Tue Sep 13 08:00:21 2005 UTC (7 years, 8 months ago) by fw

lib/python/security_db.py:
  Rename "subrelease" to "archive" (although the purpose was initially
  different).  Update test case.

Revision 1943 - Directory Listing
Modified Tue Sep 13 07:45:17 2005 UTC (7 years, 8 months ago) by fw

bin/apt-update-file:
  Remove unnecessary import.

lib/python/debian_support.py (listReleases):
  Add "sid".
(replaceFile):
  Remove temporary file on exception.
(updateFile):
  The file constructor raises IOError if the file does not exist.
  urllib does not raise a proper exception on 4xx errors.
  Handle varying whitespace in SHA1-Current field.

Revision 1939 - Directory Listing
Modified Mon Sep 12 20:08:46 2005 UTC (7 years, 8 months ago) by fw

lib/python/debian_support.py:
  Add support for downloading package file diffs.

bin/apt-update-file:
  Driver script for the new functionality.

(I will use this functionality to implement package database
replication.  The goal is to keep a local copy of all the interesting
data, so that we no longer need to consult madison etc.)

Revision 1937 - Directory Listing
Modified Mon Sep 12 17:12:08 2005 UTC (7 years, 8 months ago) by fw

lib/python/debian_support.py (ParseError):
  Add class.

lib/python/debian_support.py (PackageFile.raiseSyntaxError):o
  Raise ParseError instead of SyntaxError.

bin/check-syntax, bin/update-bug-list-db:
  Handle the ParseError exception gracefully.

lib/python/bugs.py (CVEFile.matchHeader):
  Check parentheses/brackets.

data/CAN/list:
  Fix uncovered syntax errors.

Revision 1936 - Directory Listing
Modified Mon Sep 12 16:46:36 2005 UTC (7 years, 8 months ago) by fw

lib/python/security_db.py (DB.finishBugs):
   Fix reporting of consistency check failures.

bin/update-bug-list-db:
   Do not print "error: " prefixes.

Revision 1934 - Directory Listing
Added Mon Sep 12 16:32:23 2005 UTC (7 years, 8 months ago) by fw

Add list parser written in Python.

"make check" runs a syntax check (no SQLite required).  "make all"
updates the SQLite database, and performs cross-list consistency checks.

There is some support for loading Debian Package/Sources files, but this
information is currently not used by the checks.

	ViewVC Help
Powered by ViewVC 1.1.5