quick-reference/asciidoc/02_package.txt

== Debian package management

// vim: set sts=2 expandtab:
// Use ":set nowrap" to edit table

NOTE:  This chapter is written assuming the latest stable release is codename: `@-@codename-stable@-@`.

http://www.debian.org[Debian] is a volunteer organization which builds **consistent** distributions of pre-compiled binary packages of free software and distributes them from its archive.

http://ftp.us.debian.org/debian/[The Debian archive] is offered by http://www.debian.org/mirror/[many remote mirror sites] for access through HTTP and FTP methods. It is also available as http://www.debian.org/CD/[CD-ROM/DVD].

The Debian package management system, **when used properly**, offers the user to install **consistent sets of binary packages** to the system from the archive.  Currently, there are @-@all-packages@-@ packages available for the @-@arch@-@ architecture.

The Debian package management system has a rich history and many choices for the front end user program and back end archive access method to be used.  Currently, we recommend `aptitude`(8) as the main front end program for the Debian package management activity.

.List of Debian package management tools
[grid="all"]
`---------------------`-------------`------------`----------------------------------------------------------------------------------------
package               popcon        size         description
------------------------------------------------------------------------------------------------------------------------------------------
`aptitude`            @-@popcon1@-@ @-@psize1@-@ terminal-based package manager (current standard, front-end for `apt`)
`apt`                 @-@popcon1@-@ @-@psize1@-@ Advanced Packaging Tool (APT), front-end for `dpkg` providing "`http`", "`ftp`", and "`file`" archive access methods (`apt-get`/`apt-cache` commands included)
`tasksel`             @-@popcon1@-@ @-@psize1@-@ tool for selecting tasks for installation on Debian system (front-end for APT)
`unattended-upgrades` @-@popcon1@-@ @-@psize1@-@ enhancement package for APT to enable automatic installation of security upgrades
`dselect`             @-@popcon1@-@ @-@psize1@-@ terminal-based package manager (previous standard, front-end for APT and other old access methods)
`dpkg`                @-@popcon1@-@ @-@psize1@-@ package management system for Debian
`dpkg-ftp`            @-@popcon1@-@ @-@psize1@-@ older ftp method for `dselect`
`synaptic`            @-@popcon1@-@ @-@psize1@-@ graphical package manager (GNOME front-end for APT)
`kpackage`            @-@popcon1@-@ @-@psize1@-@ graphical package manager (KDE front-end for APT)
`apt-utils`           @-@popcon1@-@ @-@psize1@-@ APT utility programs: `apt-extracttemplates`(1), `apt-ftparchive`(1), and `apt-sortpkgs`(1)
`apt-listchanges`     @-@popcon1@-@ @-@psize1@-@ package change history notification tool
`apt-listbugs`        @-@popcon1@-@ @-@psize1@-@ lists critical bugs before each APT installation
`apt-file`            @-@popcon1@-@ @-@psize1@-@ APT package searching utility -- command-line interface
`apt-rdepends`        @-@popcon1@-@ @-@psize1@-@ recursively lists package dependencies
------------------------------------------------------------------------------------------------------------------------------------------

// Removed
// || {{{adept-manager}}} || || - || graphical package manager (KDE front-end for APT) ||

NOTE: The annoying http://bugs.debian.org/411123[bug #411123] for the mixed use of `aptitude`(8) and `apt-get`(8) commands has been resolved.  If this kept you from using `aptitude`, please reconsider.

=== Debian package management prerequisites

==== Package configuration

Here are some key points for package configuration on the Debian system.

- The manual configuration by the system administrator is respected. In other words, the package configuration system makes no intrusive configuration for the sake of convenience.
- Each package comes with its own configuration script with standardized user interface called `debconf`(7) to help initial installation process of the package.
- Debian Developers try their best to make your upgrade experience flawless with package configuration scripts.
- Full functionalities of packaged software are available to the system administrator.  But ones with security risks are disabled in the default installation.
- If you manually activate a service with some security risks, you are responsible for the risk containment.
- Esoteric configuration may be manually enabled by the system administrator.  This may creates interference with popular generic helper programs for the system configuration.

// Some warning for configuration script and their limitation under customized environment.
// /!\ There are some system configuration programs packaged.  They require invocation by the system administrator.  The system administrator should be aware of the possibility of interferences once administrator made his own customization to the package.  

==== Basic precautions

WARNING: Do not install packages from random mixture of suites.  It probably breaks the package consistency which requires deep system management knowledge, such as compiler http://en.wikipedia.org/wiki/Application_binary_interface[ABI], http://en.wikipedia.org/wiki/Library_(computing)[library] version, interpreter features, etc.

The http://en.wikipedia.org/wiki/Newbie[newbie] Debian system administrator should stay with the **`stable`** release of Debian while applying only security updates.  I mean that some of the following valid actions are better avoided, as a precaution, until you understand the Debian system very well.  Here are some reminders.

- Do not include **`testing`** or **`unstable`** in "`/etc/apt/sources.list`".
- Do not mix standard Debian with other non-Debian archives such as Ubuntu in "`/etc/apt/sources.list`".
- Do not create "`/etc/apt/preferences`".
- Do not change default behavior of package management tools through configuration files without knowing their full impacts.
- Do not install random packages by "`dpkg -i <random_package>`".
- Do not ever install random packages by  "`dpkg --force-all -i <random_package>`".
- Do not erase or alter files in "`/var/lib/dpkg/`".
- Do not overwrite system files by installing software programs directly compiled from source.
 * Install them into "`/usr/local`" or "`/opt`", if needed.

The non-compatible effects caused by above actions to the Debian package management system may leave your system unusable.

The serious Debian system administrator who runs mission critical servers, should use extra precautions.

- Do not install any packages including security updates from Debian without thoroughly testing them with your particular configuration under safe conditions.
 * You as the system administrator are responsible for your system in the end.
 * The long stability history of Debian system is no guarantee by itself.

==== Life with eternal upgrades

Despite my warnings above, I know many readers of this document wish to run the `testing` or `unstable` suites of Debian as their main system for **self-administered Desktop environments**. This is because they work very well, are updated frequently, and offer the latest features.  

CAUTION: For your **production server**, the `stable` suite with the security updates is recommended.  The same can be said for desktop PCs on which you can spend limited administration efforts, e.g. for your mother@@@sq@@@s PC.

It takes no more than simply setting the distribution string in the "`/etc/apt/sources.list`" to the suite name: "`testing`" or "`unstable`"; or the codename:  "`@-@codename-testing@-@`" or "`@-@codename-unstable@-@`". This makes you live **the life of eternal upgrades**.

The use of `testing` or `unstable` is **a lot of fun** but comes with some risks.  Even though the `unstable` suite of Debian system looks very stable for most of the times, there have been some package problems on the `testing` and `unstable` suite of Debian system and a few of them were not so trivial to resolve. It may be **quite painful** for you. Sometimes, you may have a broken package or missing functionality for a few weeks.

Here are some ideas to ensure quick and easy recovery from bugs in Debian packages.

- Make the system **dual bootable** by installing the `stable` suite of Debian system to another partition
- Make the installation CD handy for the **rescue boot**
- Consider installing `apt-listbugs` to check the http://www.debian.org/Bugs/[Debian Bug Tracking System (BTS)] information before the upgrade
- Learn the package system infrastructure enough to work around the problem
- Create a chroot or similar environment and run the latest system in it in advance (optional)

(If you can not do any one of these precautionary actions, you are probably not ready for the `testing` and `unstable` suites.)

http://en.wikipedia.org/wiki/Bodhi[Enlightenment] with the following saves a person from the eternal http://en.wikipedia.org/wiki/Karma[karmic] struggle of upgrade http://en.wikipedia.org/wiki/Naraka[hell] and let him reach Debian http://en.wikipedia.org/wiki/Nirvana[nirvana].

==== Debian archive basics

Let@@@sq@@@s look into http://ftp.us.debian.org/debian/[the Debian archive] from a system user@@@sq@@@s perspective.

TIP: Official policy of the Debian archive is defined at http://www.debian.org/doc/debian-policy/ch-archive[Debian Policy Manual, Chapter 2 - The Debian Archive].

For the typical HTTP access, the archive is specified in the "`/etc/apt/sources.list`" file as the following, e.g. for the current `stable` = `@-@codename-stable@-@` system.

--------------------
deb http://ftp.XX.debian.org/debian/ @-@codename-stable@-@ main contrib non-free
deb-src http://ftp.XX.debian.org/debian/ @-@codename-stable@-@ main contrib non-free

deb http://security.debian.org/ @-@codename-stable@-@/updates main contrib
deb-src http://security.debian.org/ @-@codename-stable@-@/updates main contrib
--------------------

Please note "`ftp.XX.debian.org`" must be replaced with appropriate mirror site URL for your location, for USA "`ftp.us.debian.org`", which can be found in http://www.debian.org/mirror/list[the list of Debian worldwide mirror sites].  The status of these servers can be checked at http://www.de.debian.org/dmc/[Debian Mirror Checker site].

Here, I tend to use codename "`@-@codename-stable@-@`" instead of suite name "`stable`" to avoid surprises when the next `stable` is released.

The meaning of "`/etc/apt/sources.list`" is described in `sources.list`(5) and key points are followings.

- The "`deb`" line defines for the binary packages.
- The "`deb-src`" line defines for the source packages.
- The 1st argument is the root URL of the Debian archive.
- The 2nd argument is the distribution name: either the suite name or the codename.
- The 3rd and following arguments are the list of valid archive component names of the Debian archive.

The "`deb-src`" lines can safely be omitted (or commented out by placing "#" at the start of the line) if it is just for `aptitude` which does not access source related meta data. It speeds up the updates of the archive meta data. The URL can be "`http://`", "`ftp://`", "`file://`", ....

TIP: If "`sid`" is used in the above example instead of "`@-@codename-stable@-@`", the "`deb: http://security.debian.org/ ...`" line for security updates in the "`/etc/apt/sources.list`" is not required.  Security updates are only available for `stable` and `testing` (i.e., `@-@codename-stable@-@` and `@-@codename-testing@-@`).

Here is the list of URL of the Debian archive sites and suite name or codename used in the configuration file.

.List of Debian archive sites
[grid="all"]
`----------------------------------------------------------------------------------------`--------------------------------------`------------------
archive URL                                                                              suite name (codename)                  purpose
---------------------------------------------------------------------------------------------------------------------------------------------------
http://ftp.us.debian.org/debian/[http://ftp.XX.debian.org/debian/]                       `stable` (`@-@codename-stable@-@`)     stable (@-@codename-stable@-@) release
http://ftp.us.debian.org/debian/[http://ftp.XX.debian.org/debian/]                       `testing` (`@-@codename-testing@-@`)   testing (@-@codename-testing@-@) release
http://ftp.us.debian.org/debian/[http://ftp.XX.debian.org/debian/]                       `unstable` (`@-@codename-unstable@-@`) unstable (@-@codename-unstable@-@) release
http://ftp.us.debian.org/debian/[http://ftp.XX.debian.org/debian/]                       `experimental`                         experimental pre-release (optional, only for developer)
http://ftp.us.debian.org/debian/[http://ftp.XX.debian.org/debian/]                       `stable-proposed-updates`              Updates for the next stable point release (optional)
http://security.debian.org/[http://security.debian.org/]                                 `stable/updates`                       security updates for stable release (important)
http://security.debian.org/[http://security.debian.org/]                                 `testing/updates`                      security updates for testing release (important)
http://volatile.debian.org/debian-volatile/[http://volatile.debian.org/debian-volatile/] `volatile`                             compatible updates for spam filter, IM clients, etc.
http://volatile.debian.org/debian-volatile/[http://volatile.debian.org/debian-volatile/] `volatile-sloppy`                      non-compatible updates for spam filter, IM clients, etc.
http://backports.org/debian/[http://backports.org/debian/]                               `@-@codename-stable@-@-backports`      newer backported packages for @-@codename-stable@-@ (non-official, optional)
---------------------------------------------------------------------------------------------------------------------------------------------------

CAUTION: Only pure **`stable`** release with security updates provides the best stability. Running mostly **`stable`** release mixed with some packages from **`testing`** or **`unstable`** release is riskier than running pure **`unstable`** release.  If you really need the latest version of some programs under **`stable`** release, please use packages from http://www.debian.org/volatile/[the debian-volatile project] and http://backports.org[backports.org] (see <<_volatile_and_backports_org>>) services.  These services must be used with extra care.

CAUTION: You should basically list only one of `stable`, `testing`, or `unstable` suites in the "`deb`" line.  If you list any combination of `stable`, `testing`, and `unstable` suites in the "`deb`" line, APT programs slow down while only the latest archive is effective.  Multiple listing makes sense for these when the "`/etc/apt/preferences`" file is used with clear objectives (see <<_tweaking_candidate_version>>).

NOTE: For the Debian system with the `stable` and `testing` suites, it is a good idea to include lines with "`http://security.debian.org/`" in the "`/etc/apt/sources.list`" to enable security updates as in the example above.

// Here, I changed from section to "component" following the Release file notation.

Each Debian archive consists of 3 components. Components are alternatively called http://www.debian.org/doc/debian-policy/ch-archive#s-sections[categories in "Debian Policy"] or areas in http://www.debian.org/social_contract["Debian Social Contract"].  The component is grouped by the compliance to http://www.debian.org/social_contract#guidelines["The Debian Free Software Guidelines" (DFSG)].

.List of Debian archive components
[grid="all"]
`----------`-----------------------`-------------------------------------------------
component  number of packages      criteria of package
-------------------------------------------------------------------------------------
`main`     @-@main-packages@-@     DSFG compliant and no dependency to `non-free`
`contrib`  @-@contrib-packages@-@  DSFG compliant but having dependency to `non-free`
`non-free` @-@non-free-packages@-@ not DSFG compliant
-------------------------------------------------------------------------------------

Here the number of packages in the above is for the @-@arch@-@ architecture.  Strictly speaking, only the `main` component archive shall be considered as the Debian system.

The Debian archive organization can be studied best by pointing your browser to the each archive URL appended with `dists` or `pool`.

// the next section seems very complicated ... I know but this is life.  I keep then indirect reference for easier XML update with new release.  Please see generated HTML how they are translated.

The distribution is referred by two ways, the suite or http://www.debian.org/doc/manuals/developers-reference/resources.html#codenames[codename]. The word distribution is alternatively used as the synonym to the suite in many documentations. The relationship between the suite and the codename can be summarized as the following.

.The relationship between suite and codename
[grid="all"]
`------------------------------------------`-----------------------------------`---------------------------------------`------------------
Timing                                     suite = `stable`                    suite = `testing`                       suite = `unstable` 
------------------------------------------------------------------------------------------------------------------------------------------
after the `@-@codename-stable@-@` release  codename = `@-@codename-stable@-@`  codename = `@-@codename-testing@-@`     codename = `sid`   
after the `@-@codename-testing@-@` release codename = `@-@codename-testing@-@` codename = `@-@codename-nexttesting@-@` codename = `sid`   
------------------------------------------------------------------------------------------------------------------------------------------

// Intentinally changed for lenny release
// || current (as of @-@build-date@-@) || codename = {{{@-@codename-stable@-@}}} || codename = {{{@-@codename-testing@-@}}} || codename = {{{sid}}} ||

The history of codenames are described in http://www.debian.org/doc/manuals/debian-faq/ch-ftparchives#s-oldcodenames[Debian FAQ: 6.3.1 Which other codenames have been used in the past?]

In the stricter Debian archive terminology, the word "section" is specifically used for the categorization of packages by the application area.  (Although, the word "main section" may sometimes be used to describe the Debian archive section which provides the main component.)

Every time a new upload is done by the Debian developer (DD) to the `unstable` archive (via http://incoming.debian.org/[incoming] processing), DD is required to ensure uploaded packages to be compatible with the latest set of packages in the latest `unstable` archive.

If DD breaks this compatibility intentionally for important library upgrade etc, there is usually announcement to http://lists.debian.org/debian-devel/[the debian-devel mailing list] etc.

Before a set of packages are moved by the Debian archive maintenance script from the `unstable` archive to the `testing` archive, the archive maintenance script not only checks the maturity (about 10 days old) and the status of the RC bug reports for the packages but also tries to ensure them to be compatible with the latest set of packages in the `testing` archive. This process makes the `testing` archive very current and usable.

Through the gradual archive freeze process led by the release team, the `testing` archive is matured to make it completely consistent and bug free with some manual interventions.  Then the new `stable` release is created by assigning the codename for the old `testing` archive to the new `stable` archive and creating the new codename for the new `testing` archive.  The initial contents of the new `testing` archive is exactly the same as that of the newly released `stable` archive.

Both the `unstable` and the `testing` archives may suffer temporary glitches due to several factors.

- Broken package upload to the archive (mostly for `unstable`)
- Delay of accepting the new packages to the archive (mostly for `unstable`)
- Archive synchronization timing issue (both for `testing` and `unstable`)
- Manual intervention to the archive such as package removal (more for `testing`) etc.

So if you ever decide to use these archives, you should be able to fix or work around these kinds of glitches.

CAUTION: For about few months after a new `stable` release, most desktop users should use the `stable` archive with its security updates even if they usually use `unstable` or `testing` archives.  For this transition period, both `unstable` and `testing` archives are not good for most people. Your system is difficult to keep in good working condition with the `unstable` archive since it suffers surges of major upgrades for core packages.  The `testing` archive is not useful either since it contains mostly the same content as the `stable` archive without its security support (http://lists.debian.org/debian-testing-security-announce/2008/12/msg00019.html[Debian testing-security-announce 2008-12]). After a month or so, the `unstable` archive may be usable if you are careful.

TIP: When tracking the `testing` archive, problem caused by a removed package is usually worked around by installing corresponding package from the `unstable` archive which is uploaded for bug fix.

// Testing usability discussion may be too complicated here.
// {i} See discussion on [http://lists.debian.org/debian-devel/ debian-devel mailing list] 
// [http://lists.debian.org/debian-devel/2008/06/msg00048.html how manual intervention are managed].

See http://www.debian.org/doc/debian-policy/[Debian Policy Manual] for archive definitions.

- "http://www.debian.org/doc/debian-policy/ch-archive#s-subsections[Sections]"
- "http://www.debian.org/doc/debian-policy/ch-archive#s-priorities[Priorities]"
- "http://www.debian.org/doc/debian-policy/ch-binary#s3.7[Base system]"
- "http://www.debian.org/doc/debian-policy/ch-binary#s3.8[Essential packages]"

==== Package dependencies

The Debian system offers a consistent set of binary packages through its versioned binary dependency declaration mechanism in the control file fields.   Here is a bit over simplified definition for them.

- "Depends"
 * This declares an absolute dependency and all of the packages listed in this field must be installed at the same time or in advance.
- "Pre-Depends"
 * This is like Depends, except that it requires completed installation of the listed packages in advance.
- "Recommends"
 * This declares a strong, but not absolute, dependency.  Most users would not want the package unless all of the packages listed in this field are installed.
- "Suggests"
 * This declares a weak dependency.  Many users of this package may benefit from installing packages listed in this field but can have reasonable functions without them.
- "Enhances"
 * This declares a week dependency like Suggests but works in the opposite direction.
- "Conflicts"
 * This declares an absolute incompatibility.  All of the packages listed in this field must be removed to install this package.
- "Replaces"
 * This is declared when files installed by this package replace files in the listed packages.
- "Provides"
 * This is declared when this package provide all of the files and functionality in the listed packages.

NOTE: Please note that defining, "Provides", "Conflicts" and "Replaces" simultaneously to an virtual package is the sane configuration.  This ensures that only one real package providing this virtual package can be installed at any one time.

The official definition including source dependency can be found in http://www.debian.org/doc/debian-policy/ch-relationships[the Policy Manual: Chapter 7 - Declaring relationships between packages].

==== The event flow of the package management

Here is a summary of the simplified event flow of the package management by APT.

- **Update** ("`aptitude update`" or "`apt-get update`"):
 1. Fetch archive metadata from remote archive
 2. Reconstruct and update local metadata for use by APT

- **Upgrade** ("`aptitude safe-upgrade`" and "`aptitude full-upgrade`", or "`apt-get upgrade`" and "`apt-get dist-upgrade`"):
 1. Chose candidate version which is usually the latest available version for all installed packages (see <<_tweaking_candidate_version>> for exception)
 2. Make package dependency resolution
 3. Fetch selected binary packages from remote archive if candidate version is different from installed version
 4. Unpack fetched binary packages
 5. Run **preinst** script
 6. Install binary files
 7. Run **postinst** script

- **Install** ("`aptitude install ...`" or "`apt-get install ...`"):
 1. Chose packages listed on the command line
 2. Make package dependency resolution
 3. Fetch selected binary packages from remote archive
 4. Unpack fetched binary packages
 5. Run **preinst** script
 6. Install binary files
 7. Run **postinst** script

- **Remove** ("`aptitude remove ...`" or "`apt-get remove ...`"):
 1. Chose packages listed on the command line
 2. Make package dependency resolution
 3. Run **prerm** script
 4. Remove installed files **except** configuration files
 5. Run **postrm** script

- **Purge** ("`aptitude purge ...`" or "`apt-get purge ...`"):
 1. Chose packages listed on the command line
 2. Make package dependency resolution
 3. Run **prerm** script
 4. Remove installed files **including** configuration files
 5. Run **postrm** script

Here, I intentionally skipped technical details for the sake of big picture.

==== First response to package management troubles

You should read the fine official documentation.  The first document to read is the Debian specific "`/usr/share/doc/<package_name>/README.Debian`".  Other documentation in "`/usr/share/doc/<package_name>/`" should be consulted too.  If you set shell as <<_customizing_bash>>, type the following.

--------------------
$ cd <package_name>
$ pager README.Debian
$ mc
--------------------

You may need to install the corresponding documentation package named with "`-doc`" suffix for detailed information.

If you are experiencing problems with a specific package, make sure to check out http://bugs.debian.org/[the Debian bug tracking system (BTS)] sites, first.

.List of key web site to resolving problems with a specific package
[grid="all"]
`--------------------------------------------------------------------------`----------------------------------------------------------
web site                                                                   command
--------------------------------------------------------------------------------------------------------------------------------------
Home page of http://bugs.debian.org/[the Debian bug tracking system (BTS)] `sensible-browser "http://bugs.debian.org/"`
The bug report of a known package name                                     `sensible-browser "http://bugs.debian.org/<package_name>"`
The bug report of known bug number                                         `sensible-browser "http://bugs.debian.org/<bug_number>"`
--------------------------------------------------------------------------------------------------------------------------------------

Search http://www.google.com[Google] with search words including "`site:debian.org`", "`site:wiki.debian.org`", "`site:lists.debian.org`", etc.

When you file a bug report, please use `reportbug`(1) command.

=== Basic package management operations

Aptitude is the current preferred package management tool for the Debian system.  It can be used as the commandline alternative to `apt-get` / `apt-cache` and also as the full screen interactive package management tool.

For the package management operation which involves package installation or updates package metadata, you need to have root privilege.

==== Basic package management operations with commandline

Here are basic package management operations with commandline using `aptitude`(8) and `apt-get`(8) /`apt-cache`(8).

.Basic package management operations with commandline using `aptitude`(8) and `apt-get`(8) /`apt-cache`(8)
[grid="all"]
`--------------------------`--------------------------------`-----------------------------------------------------------------------------
`aptitude` syntax          `apt-get`/`apt-cache` syntax     description
------------------------------------------------------------------------------------------------------------------------------------------
`aptitude update`          `apt-get update`                 update package archive metadata
`aptitude install foo`     `apt-get install foo`            install candidate version of "`foo`" package with its dependencies
`aptitude safe-upgrade`    `apt-get upgrade`                install candidate version of installed packages without removing any other packages
`aptitude full-upgrade`    `apt-get dist-upgrade <package>` install candidate version of installed packages while removing other packages if needed
`aptitude remove foo`      `apt-get remove foo`             remove "`foo`" package while leaving its configuration files
N/A                        `apt-get autoremove`             remove auto-installed packages which is no longer required
`aptitude purge foo`       `apt-get purge foo`              purge "`foo`" package with its configuration files
`aptitude clean`           `apt-get clean`                  clear out the local repository of retrieved package files completely
`aptitude autoclean`       `apt-get autoclean`              clear out the local repository of retrieved package files for outdated packages
`aptitude show foo`        `apt-cache show <package>`       display detailed information about "`foo`" package
`aptitude search <regex>`  `apt-cache search <regex>`       search packages which match <regex>
`aptitude why <regex>`     N/A                              explain the reason why <regex> matching packages should be installed
`aptitude why-not <regex>` N/A                              explain the reason why <regex> matching packages can not be installed
------------------------------------------------------------------------------------------------------------------------------------------

// You may lose the advantage of `aptitude` keeping track of which packages you have deliberately installed if other package tools are used.  

Although it is now safe to mix different package tools on the Debian system, it is best to continue using `aptitude` as much as possible.

The difference between "`safe-upgrade`"/"`upgrade`" and "`full-upgrade`"/"`dist-upgrade`" only appears when new versions of packages stand in different dependency relationships from old versions of those packages.  The "`aptitude safe-upgrade`" command does not install new packages nor remove installed packages.

The "`aptitude why <regex>`" can list more information by "`aptitude -v why <regex>`".  Similar information can be obtained by "`apt-cache rdepends <package>`".

When `aptitude` command is started in the commandline mode and faces some issues such as package conflicts, you can switch to the full screen interactive mode by pressing "`e`"-key later at the prompt.

You may provide command options right after "`aptitude`".  

.Notable command options for `aptitude`(8)
[grid="all"]
`--------------`------------------------------------------------------------------------
command option description
----------------------------------------------------------------------------------------
`-s`           simulate the result of the command
`-d`           download only but no install/upgrade
`-D`           show brief explanations before the automatic installations and removals
----------------------------------------------------------------------------------------

See `aptitude`(8) and "aptitude user@@@sq@@@s manual" at "`/usr/share/doc/aptitude/README`" for more.

TIP: The `dselect` package is still available and was the preferred full screen interactive package management tool in previous releases.

==== Interactive use of aptitude

For the interactive package management, you start `aptitude` in interactive mode from the console shell prompt as follows.

--------------------
$ sudo aptitude -u
Password:
--------------------

This updates the local copy of the archive information and display the package list in the full screen with menu.  Aptitude places its configuration at "`\~/.aptitude/config`".

TIP: If you want to use root@@@sq@@@s configuration instead of user@@@sq@@@s one, use "`sudo -H aptitude ...`" instead of "`sudo aptitude ...`" in the above expression.

TIP: `Aptitude` automatically sets **pending actions** as it is started interactively. If you do not like it, you can reset it from menu: "Action" -> "Cancel pending actions".

==== Key bindings of aptitude

Notable key strokes to browse status of packages and to set "planned action" on them in this full screen mode are the following.

.List of key bindings for aptitude
[grid="all"]
`-------------------------------------`---------------------------------------------------------------
key                                   key binding
-----------------------------------------------------------------------------------------------
`F10` or `Ctrl-t`                     menu
`?`                                   display **help** for keystroke (more complete listing)
`F10` -> Help -> User@@@sq@@@s Manual display User@@@sq@@@s Manual
`u`                                   update package archive information
`\+`                                  mark the package for the **upgrade** or the **install**
`-`                                   mark the package for the **remove** (keep configuration files)
`\_`                                  mark the package for the **purge** (remove configuration files)
`=`                                   place the package on **hold**
`U`                                   mark all upgradable packages (function as **full-upgrade**)
`g`                                   start **downloading** and **installing** selected packages
`q`                                   quit current screen and save changes
`x`                                   quit current screen and discard changes
`Enter`                               view information about a package
`C`                                   view a package@@@sq@@@s changelog
`l`                                   change the limit for the displayed packages
`/`                                   search for the first match
`\`                                   repeat the last search
------------------------------------------------------------------------------------------------

The file name specification of the command line and the menu prompt after pressing "`l`" and "`//`" take the aptitude regex as described below.  Aptitude regex can explicitly match a package name using a string started by "`\~n` and followed by the package name.

TIP: You need to press "`U`" to get all the installed packages upgraded to the **candidate version** in the visual interface.  Otherwise only the selected packages and certain packages with versioned dependency to them are upgraded to the **candidate version**.

==== Package views under aptitude

In the interactive full screen mode of `aptitude`(8), packages in the package list are displayed as the next example.

--------------------
idA   libsmbclient                             -2220kB 3.0.25a-1  3.0.25a-2
--------------------

Here, this line means from the left as the following.

- The "current state" flag (the first letter)
- The "planned action" flag (the second letter) 
- The "automatic" flag (the  third letter)
- The Package name
- The change in disk space usage attributed to "planned action"
- The current version of the package
- The candidate version of the package

TIP: The full list of flags are given at the bottom of **Help** screen shown by pressing "`?`".

The **candidate version** is chosen according to the current local preferences (see `apt_preferences`(5) and <<_tweaking_candidate_version>>).

Several types of package views are available under the menu "`Views`".

.List of views for aptitude
[grid="all"]
`-----------------------`-------------`-----------------------------------------------------------------------------------------------
view                    status        description of view
--------------------------------------------------------------------------------------------------------------------------------------
`Package View`          Good          see <<standard-package-views>> (default)
`Audit Recommendations` Good          list packages which are recommended by some installed packages but not yet installed are listed
`Flat Package List`     Good          list packages without categorization (for use with regex)
`Debtags Browser`       Very usable   list packages categorized according to their http://debtags.alioth.debian.org/[debtags] entries
`Categorical Browser`   Deprecated    list packages categorized according to their category (use `Debtags Browser`, instead)
--------------------------------------------------------------------------------------------------------------------------------------

NOTE: Please help us http://debtags.alioth.debian.org/todo.html[improving tagging packages with debtags!]

The standard "`Package View`" categorizes packages somewhat like `dselect` with few extra features.

[[standard-package-views]]
.The categorization of standard package views
[grid="all"]
`---------------------------------------`--------------------------------------------------------------------------------
category                                description of view
-------------------------------------------------------------------------------------------------------------------------
`Upgradable Packages`                   list packages organized as `section` -> `component` -> `package`
`New Packages`                          , ,
`Installed Packages`                    , ,
`Not Installed Packages`                , ,
`Obsolete and Locally Created Packages` , ,
`Virtual Packages`                      list packages with the same function
`Tasks`                                 list packages with different functions generally needed for a task
-------------------------------------------------------------------------------------------------------------------------

TIP: `Tasks` view can be used to cherry pick packages for your task.

==== Search method options with aptitude

Aptitude offers several options for you to search packages using its regex formula.

- Shell commandline:
 * "`aptitude search \'<aptitude_regex>\'`" to list installation status, package name and short description of matching packages
 * "`aptitude show \'<package_name>\'`" to list detailed description of the package
- Interactive full screen mode:
 * "`l`" to limit package view to matching packages
 * "`/`" for search to a matching package
 * "`\`" for backward search to a matching package
 * "`n`" for find-next
 * "`N`" for find-next (backward)

TIP: The string for <package_name> is treated as the exact string match to the package name unless it is started explicitly with "`\~`" to be the regex formula.

==== The aptitude regex formula

The aptitude regex formula is mutt-like extended **ERE** (see <<_regular_expressions>>) and the meanings of the `aptitude` specific special match rule extensions are as follows.

.List of the aptitude regex formula
[grid="all"]
`---------------------------------------------------------------------------------`-------------------------------------------------------
description of the extended match rule                                            regex formula
------------------------------------------------------------------------------------------------------------------------------------------
match on package name                                                             `\~n<regex_name>`
match on description                                                              `\~d<regex_description>`
match on task name                                                                `\~t<regex_task>`
match on debtag                                                                   `\~G<regex_debtag>`
match on maintainer                                                               `\~m<regex_maintainer>`
match on package section                                                          `\~s<regex_section>`
match on package version                                                          `\~V<regex_version>`
match archive                                                                     `\~A{sarge,etch,sid`}
match origin                                                                      `\~O{debian,...`}
match priority                                                                    `\~p{extra,important,optional,required,standard`}
match essential packages                                                          `\~E`
match virtual packages                                                            `\~v`
match new packages                                                                `\~N`
match with pending action                                                         `\~a{install,upgrade,downgrade,remove,purge,hold,keep`}
match installed packages                                                          `\~i`
match installed packages with **A**-mark (auto installed package)                 `\~M`
match installed packages without **A**-mark (administrator selected package)      `\~i!\~M`
match installed and upgradable packages                                           `\~U`
match removed but not purged packages                                             `\~c`
match removed, purged or can-be-removed packages                                  `\~g`
match packages with broken relation                                               `\~b`
match packages with broken depends/predepends/conflict                            `\~B<type>`
match packages from which relation <type> is defined to <term> package            `\~D[<type>:]<term>`
match packages from which **broken** relation <type> is defined to <term> package `\~DB[<type>:]<term>`
match packages to which the <term> package defines relation <type>                `\~R[<type>:]<term>`
match packages to which the <term> package defines **broken** relation <type>     `\~RB[<type>:]<term>`
match packages to which some other installed packages depend on                   `\~R\~i`
match packages to which no other installed packages depend on                     `!\~R\~i`
match packages to which some other installed packages depend or recommend on      `\~R\~i|\~Rrecommends:\~i`
match <term> package with filtered version                                        `\~S filter <term>`
match all packages (true)                                                         `\~T`
match no packages (false)                                                         `\~F`
------------------------------------------------------------------------------------------------------------------------------------------

- The regex part is the same **ERE** as the one used in typical Unix-like text tools using "`\^`", "`.\*`", "`$`" etc. as in `egrep`(1), `awk`(1) and `perl`(1).
- The relation <type> is one of (depends, predepends, recommends, suggests, conflicts, replaces, provides).
- The default relation type is "depends".

TIP: When <regex_pattern> is a null string, place "`\~T`" immediately after the command.

Here are some short cuts.

- "`\~P<term>`" == "`\~Dprovides:<term>`"
- "`\~C<term>`" == "`\~Dconflicts:<term>`"
- "`...\~W term`" == "`(...|term)`"

Users familiar with `mutt` pick up quickly, as mutt was the inspiration for the expression syntax. See "SEARCHING, LIMITING, AND EXPRESSIONS" in the "User@@@sq@@@s Manual" "`/usr/share/doc/aptitude/README`".

NOTE: With the `lenny` version of `aptitude`(8), the new **long form** syntax such as "`?broken`" may be used for regex matching in place for its old **short form** equivalent "`\~b`". Now space character "` `" is considered as one of the regex terminating character in addition to tilde character "`\~`".  See "User@@@sq@@@s Manual" for the new **long form** syntax.

==== Dependency resolution of aptitude

The selection of a package in `aptitude` not only pulls in packages which are defined in its "`Depends:`" list but also defined in the "`Recommends:`" list if the menu "`F10` -> Options -> Dependency handling" is set accordingly.  These auto installed packages are removed automatically if they are no longer needed under `aptitude`. 

NOTE: Before the `lenny` release, `apt-get` and other standard APT tools did not offer the autoremove functionality.

==== Package activity logs

You can check package activity history in the log files.

.The log files for package activities
[grid="all"]
`-----------------------`--------------------------------------------------------
file                    content
---------------------------------------------------------------------------------
`/var/log/dpkg.log`     Log of `dpkg` level activity for all package activities
`/var/log/apt/term.log` Log of generic APT activity
`/var/log/aptitude`     Log of `aptitude` command activity
---------------------------------------------------------------------------------

In reality, it is not so easy to get meaningful understanding quickly out from these logs.  See <<_recording_changes_in_configuration_files>> for easier way.

==== Aptitude advantages

Aptitude has advantages over other APT based packaging systems (apt-get, apt-cache, synaptic, ...).

- `aptitude` removes unused auto installed packages automatically using its own extra layer of package state file (`/var/lib/aptitude/pkgstates`). (For new "`lenny`", other APT does the same.)
- `aptitude` makes it easy to resolve package conflicts and to add recommended packages.
- `aptitude` makes it easy to keep track of obsolete software by listing under "Obsolete and Locally Created Packages".
- `aptitude` gives a log of its history in "`/var/log/aptitude`".
- `aptitude` offers access to all versions of the package if available.
- `aptitude` includes a fairly powerful regex based system for searching particular packages and limiting the package display.
- `aptitude` in the full screen mode has `su` functionality embedded and can be run from normal user until you really need administrative privileges.

For the old `etch` release version, `synaptic` also gives you the history log; `apt-get` did not but you can rely on the log of `dpkg`.

Anyway, `aptitude` is nice for interactive console use.

=== Examples of aptitude operations

Here are few examples of `aptitude`(8) operations.

==== Listing packages with regex matching on package names

The following command lists packages with regex matching on package names.

--------------------
$ aptitude search '~n(pam|nss).*ldap'
p libnss-ldap - NSS module for using LDAP as a naming service
p libpam-ldap - Pluggable Authentication Module allowing LDAP interfaces
--------------------

This is quite handy for you to find the exact name of a package.

==== Browsing with the regex matching

The regex "`\~dipv6`" in the "New Flat Package List" view with "`l`" prompt, limits view to packages with the matching description and let you browse their information interactively.

==== Purging removed packages for good

You can purge all remaining configuration files of removed packages.

Check results of the following command.

--------------------
# aptitude search '~c'
--------------------

If you think listed packages are OK to be purged, execute the following command.

--------------------
# aptitude purge '~c'
--------------------

You may want to do the similar in the interactive mode for fine grained control.

You provide the regex "`\~c`" in the "New Flat Package List" view with "`l`" prompt.  This limits the package view only to regex matched packages, i.e., "removed but not purged".  All these regex matched packages can be shown by pressing "`[`" at top level headings.

Then you press "`_`" at top level headings such as "Installed Packages".  Only regex matched packages under the heading are marked to be purged by this.  You can exclude some packages to be purged by pressing "`=`" interactively for each of them.

This technique is quite handy and works for many other command keys.

==== Tidying auto/manual install status

Here is how I tidy auto/manual install status for packages (after using non-aptitude package installer etc.).

1. Start `aptitude` in interactive mode as root.
2. Type "`u`", "`U`", "`f`" and "`g`" to update and upgrade package list and packages.
3. Type "`l`" to enter the package display limit as "`\~i(\~R\~i|\~Rrecommends:\~i)`" and type "`M`" over "`Installed Packages`" as auto installed.
4. Type "`l`" to enter the package display limit as "`\~prequired|\~pimportant|\~pstandard|\~E`" and type "`m`" over "`Installed Packages`" as manual installed.
5. Type "`l`" to enter the package display limit as "`\~i!\~M`" and remove unused package by typing "`-`" over each of them after exposing them by typing "`[`" over "`Installed Packages`".
6. Type "`l`" to enter the package display limit as "`\~i`" and type "`m`" over "`Tasks`" as manual installed.
7. Exit `aptitude`.
8. Start "`apt-get -s autoremove|less`" as root to check what are not used.
9. Restart `aptitude` in interactive mode and mark needed packages as "`m`".
10. Restart "`apt-get -s autoremove|less`" as root to recheck REMOVED contain only expected packages.
11. Start "`apt-get autoremove|less`" as root to autoremove unused packages.

The "`m`" action over "`Tasks`" is an optional one to prevent mass package removal situation in future.

==== System wide upgrade with aptitude

NOTE: When moving to a new release etc, you should consider to perform a clean installation of new system even though Debian is upgradable as described below.  This provides you a chance to remove garbages collected and exposes you to the best combination of latest packages.  Of course, you should make a full backup of system to a safe place (see <<_backup_and_recovery>>) before doing this. I recommend to make a dual boot configuration using different partition to have the smoothest transition.

You can perform system wide upgrade to a newer release by changing contents of the "`/etc/apt/sources.list`" file pointing to a new release and running the "`aptitude update; aptitude full-upgrade`" command.  

To upgrade from `stable` to `testing` or `unstable`, you replace "`@-@codename-stable@-@`" in the "`/etc/apt/sources.list`" example of <<_debian_archive_basics>> with "`@-@codename-testing@-@`" or "`sid`".

In reality, you may face some complications due to some package transition issues, mostly due to package dependencies.  The larger the difference of the upgrade, the more likely you face larger troubles.  For the transition from the old `stable` to the new `stable` after its release, you can read its new http://www.debian.org/releases/stable/releasenotes[Release Notes] and follow the exact procedure described in it to minimize troubles. 

When you decide to move from `stable` to `testing` before its formal release, there are no http://www.debian.org/releases/stable/releasenotes[Release Notes] to help you. The difference between `stable` and `testing` could have grown quite large after the previous `stable` release and makes upgrade situation complicated.

You should make precautionary moves for the full upgrade while gathering latest information from mailing list and using common senses.

1. Read previous "Release Notes".
2. Backup entire system (especially data and configuration information).
3. Have bootable media handy for broken bootloader.
4. Inform users on the system well in advance.
5. Record upgrade activity with `script`(1).
6. Apply "unmarkauto" to required packages, e.g., "`aptitude unmarkauto vim`", to prevent removal.
7. Minimize installed packages to reduce chance of package conflicts, e.g., remove desktop task packages.
8. Remove the "`/etc/apt/preferences`" file (disable apt-pinning).
9. Try to upgrade step wise: `oldstable` -> `stable` -> `testing` -> `unstable`.
10. Update the "`/etc/apt/sources.list`" file to point to new archive only and run "`aptitude update`".
11. Install, optionally, new **core packages** first, e.g., "`aptitude install perl`".
12. Run the "`aptitude full-upgrade -s`" command to assess impact.
13. Run the "`aptitude full-upgrade`" command at last.

CAUTION: It is not wise to skip major Debian release when upgrading between `stable` releases.

CAUTION: In previous "Release Notes", GCC, Linux Kernel, initrd-tools, Glibc, Perl, APT tool chain, etc. have required some special attention for system wide upgrade.

For daily upgrade in `unstable`, see <<_safeguarding_for_package_problems>>.

=== Advanced package management operations

==== Advanced package management operations with commandline

Here are list of other package management operations for which `aptitude` is too high-level or lacks required functionalities.

.List of advanced package management operations
[grid="all"]
`--------------------------------------------------------------`--------------------------------------------------------------------------
command                                                        action
------------------------------------------------------------------------------------------------------------------------------------------
`COLUMNS=120 dpkg -l <package_name_pattern>`                   list status of an installed package for the bug report
`dpkg -L <package_name>`                                       list contents of an installed package
`dpkg -L <package_name> | egrep \'/usr/share/man/man.\*/.+\'`  list manpages for an installed package
`dpkg -S <file_name_pattern>`                                  list installed packages which have matching file name
`apt-file search <file_name_pattern>`                          list packages in archive which have matching file name
`apt-file list <package_name_pattern>`                         list contents of matching packages in archive
`dpkg-reconfigure <package_name>`                              reconfigure the exact package 
`dpkg-reconfigure -p=low <package_name>`                       reconfigure the exact package with the most detailed question
`configure-debian`                                             reconfigure packages from the full screen menu
`dpkg --audit`                                                 audit system for partially installed packages
`dpkg --configure -a`                                          configure all partially installed packages
`apt-cache policy <binary_package_name>`                       show available version, priority, and archive information of a binary package
`apt-cache madison <package_name>`                             show available version, archive information of a package
`apt-cache showsrc <binary_package_name>`                      show source package information of a binary package
`apt-get build-dep <package_name>`                             install required packages to build package
`apt-get source <package_name>`                                download a source (from standard archive)
`dget <URL for dsc file>`                                      download a source packages (from other archive)
`dpkg-source -x <package_name>_<version>-<debian_version>.dsc` build a source tree from a set of source packages ("`\*.tar.gz`" and "`\*.diff.gz`")
`debuild binary`                                               build package(s) from a local source tree
`make-kpkg kernel_image`                                       build a kernel package from a kernel source tree
`make-kpkg --initrd kernel_image`                              build a kernel package from a kernel source tree with initramfs enabled
`dpkg -i <package_name>_<version>-<debian_version>_<arch>.deb` install a local package to the system
`debi <package_name>_<version>-<debian_version>_<arch>.dsc`    install local package(s) to the system
`dpkg --get-selection \'\*\' >selection.txt`                   save `dpkg` level package selection state information
`dpkg --set-selection <selection.txt`                          set `dpkg` level package selection state information
------------------------------------------------------------------------------------------------------------------------------------------

// removed this since it create package under the same version || download a source and rebuild package(s) locally. || {{{apt-get -b source <package_name>}}} ||

CAUTION: Lower level package tools such as "`dpkg -i ...`" and "`debi ...`" should be carefully used by the system administrator.  It does not automatically take care required package dependencies. Dpkg@@@sq@@@s commandline options "`--force-all`" and similar (see `dpkg`(1)) are intended to be used by experts only.  Using them without fully understanding their effects may break your whole system.

Please note the following.

- All system configuration and installation commands require to be run from root.
- Unlike `aptitude` which uses regex (see <<_regular_expressions>>), other package management commands use pattern like shell glob (see <<_shell_glob>>). 
- `apt-file`(1) provided by the `apt-file` package must run "`apt-file update`" in advance.
- `configure-debian`(8) provided by the `configure-debian` package runs `dpkg-reconfigure`(8) as its backend.
- `dpkg-reconfigure`(8) runs package scripts using `debconf`(1) as its backend.
- "`apt-get build-dep`", "`apt-get source`" and "`apt-cache showsrc`" commands require "`deb-src`" entry in "`/etc/apt/sources.list`".
- `dget`(1), `debuild`(1), and `debi`(1) require `devscripts` package.
- See (re)packaging procedure using "`apt-get source`" in <<_porting_a_package_to_the_stable_system>>.
- `make-kpkg` command requires the `kernel-package` package (see <<_the_kernel>>).
- See <<_making_debian_package>> for general packaging.

TIP: The source package format described here as a set of source packages ("`\*.tar.gz`" and "`\*.diff.gz`") is format 1.0 which is still popular.  See more on `dpkg-source`(1) for other newer formats.

==== Verification of installed package files

The installation of `debsums` enables verification of installed package files against MD5sum values in the "`/var/lib/dpkg/info/\*.md5sums`" file with `debsums`(1).  See <<_the_md5_sum>> for how MD5sum works.

NOTE: Because MD5sum database may be tampered by the intruder, `debsums`(1) is of limited use as a security tool. It is only good for checking local modifications by the administrator or damage due to media errors.

==== Safeguarding for package problems

Many users prefer to follow the **unstable** release of the Debian system for its new features and packages.  This makes the system more prone to be hit by the critical package bugs.  

The installation of the `apt-listbugs` package safeguards your system against critical bugs by checking Debian BTS automatically for critical bugs when upgrading with APT system.

The installation of the `apt-listchanges` package provides important news in "`NEWS.Debian`" when upgrading with APT system.

==== Searching on the package meta data

Although visiting Debian site http://packages.debian.org/[http://packages.debian.org/] facilitates easy ways to search on the package meta data these days, let@@@sq@@@s look into more traditional ways.

The `grep-dctrl`(1), `grep-status`(1), and `grep-available`(1) commands can be used to search any file which has the general format of a Debian package control file.

The "`dpkg -S <file_name_pattern>`" can be used search package names which contain files with the matching name installed by `dpkg`.  But this overlooks files created by the maintainer scripts.

If you need to make more elaborate search on the dpkg meta data, you need to run "`grep -e regex_pattern \*`" command in the "`/var/lib/dpkg/info/`" directory.  This makes you search words mentioned in package scrips and installation query texts.

If you wish to look up package dependency recursively, you should use `apt-rdepends`(8).

=== Debian package management internals

Let@@@sq@@@s learn how the Debian package management system works internally.  This should help you to create your own solution to some package problems.

==== Archive meta data

Meta data files for each distribution are stored under "`dist/<codename>`" on each Debian mirror sites, e.g., "`http://ftp.us.debian.org/debian/`".  Its archive structure can be browsed by the web browser. There are 6 types of key meta data.

.The content of the Debian archive meta data
[grid="all"]
`-------------------------`------------------------------------------------------------------`--------------------------------------------
file                      location                                                           content
------------------------------------------------------------------------------------------------------------------------------------------
`Release`                 top of distribution                                                archive description and integrity information
`Release.gpg`             top of distribution                                                signature file for the "`Release`" file signed with the archive key
`Contents-<architecture>` top of distribution                                                list of all files for all the packages in the pertinent archive
`Release`                 top of each distribution/component/architecture combination        archive description used for the rule of `apt_preferences`(5)
`Packages`                top of each distribution/component/binary-architecture combination concatenated `debian/control` for binary packages
`Sources`                 top of each distribution/component/source combination              concatenated `debian/control` for source packages
------------------------------------------------------------------------------------------------------------------------------------------

In the recent archive, these meta data are stored as the compressed and differential files to reduce network traffic.

==== Top level "Release" file and authenticity

TIP: The top level "`Release`" file is used for signing the archive under the **secure APT** system.

Each suite of the Debian archive has a top level "`Release`" file, e.g., "`http://ftp.us.debian.org/debian/dists/unstable/Release`", as follows.

--------------------
Origin: Debian
Label: Debian
Suite: unstable
Codename: sid
Date: Sat, 26 Jan 2008 20:13:58 UTC
Architectures: alpha amd64 arm hppa hurd-i386 i386 ia64 m68k mips mipsel powerpc s390 sparc
Components: main contrib non-free
Description: Debian x.y Unstable - Not Released
MD5Sum:
 e9f11bc50b12af7927d6583de0a3bd06 22788722 main/binary-alpha/Packages
 43524d07f7fa21b10f472c426db66168  6561398 main/binary-alpha/Packages.gz
...
--------------------

NOTE: Here, you can find my rationale to use the "suite", "codeneme", and "components" in <<_debian_archive_basics>>.  The "distribution" is used when referring to both "suite" and "codeneme".

The integrity of the top level "`Release`" file is verified by cryptographic infrastructure called the http://wiki.debian.org/SecureApt[secure apt].

- The cryptographic signature file "`Release.gpg`" is created from the authentic top level "`Release`" file and the secret Debian archive key.
- The public Debian archive key can be seeded into "`/etc/apt/trusted.gpg`";
 * automatically by installing the keyring with the latest `base-files` package, or
 * manually by `gpg` or `apt-key` tool with http://ftp-master.debian.org/[the latest public archive key posted on the ftp-master.debian.org] .
- The **secure APT** system verifies the integrity of the downloaded top level "`Release`" file cryptographically by this "`Release.gpg`" file and the public Debian archive key in "`/etc/apt/trusted.gpg`".

The integrity of all the "`Packages`" and "`Sources`" files are verified by using MD5sum values in its top level "`Release`" file.  The integrity of all package files are verified by using MD5sum values in the "`Packages`" and "`Sources`" files.  See `debsums`(1) and <<_verification_of_installed_package_files>>.

Since the cryptographic signature verification is very CPU intensive process than the MD5sum value calculation, use of MD5sum value for each package while using cryptographic signature for the top level "`Release`" file provides http://www.infodrom.org/\~joey/Writing/Linux-Journal/secure-apt/[the good security with the performance] (see <<_data_security_infrastructure>>).

==== Archive level "Release" files

TIP: The archive level "`Release`" files are used for the rule of `apt_preferences`(5).

There are archive level "`Release`" files for all archive locations specified by "`deb`" line in "`/etc/apt/sources.list`", such as "`http://ftp.us.debian.org/debian/dists/unstable/main/binary-amd64/Release`" or "`http://ftp.us.debian.org/debian/dists/sid/main/binary-amd64/Release`" as follows.

--------------------
Archive: unstable
Component: main
Origin: Debian
Label: Debian
Architecture: amd64
--------------------

CAUTION: For "`Archive:`" stanza, suite names ("`stable`", "`testing`", "`unstable`", ...) are used in http://ftp.us.debian.org/debian/[the Debian archive] while codenames ("`dapper`", "`feisty`", "`gutsy`", "`hardy`", "`intrepid`", ...) are used in http://archive.ubuntu.com/ubuntu/[the Ubuntu archive].

For some archives, such as `experimental`, `volatile-sloppy`, and `@-@codename-stable@-@-backports`, which contain packages which should not be installed automatically, there is an extra line, e.g., "`http://ftp.us.debian.org/debian/dists/experimental/main/binary-amd64/Release`" as follows.

--------------------
Archive: experimental
Component: main
Origin: Debian
Label: Debian
NotAutomatic: yes
Architecture: amd64
--------------------

Please note that for normal archives without "`NotAutomatic: yes`", the default Pin-Priority value is 500, while for special archives with "`NotAutomatic: yes`", the default Pin-Priority value is 1 (see `apt_preferences`(5) and <<_tweaking_candidate_version>>). 

==== Fetching of the meta data for the package

When APT tools, such as `aptitude`, `apt-get`, `synaptic`, `apt-file`, `auto-apt`..., are used, we need to update the local copies of the meta data containing the Debian archive information. These local copies have following file names corresponding to the specified `distribution`, `component`, and `architecture` names in the "`/etc/apt/sources.list`" (see <<_debian_archive_basics>>).

- "`/var/lib/apt/lists/ftp.us.debian.org_debian_dists_<distribution>_Release`"
- "`/var/lib/apt/lists/ftp.us.debian.org_debian_dists_<distribution>_Release.gpg`"
- "`/var/lib/apt/lists/ftp.us.debian.org_debian_dists_<distribution>_<component>_binary-<architecture>_Packages`"
- "`/var/lib/apt/lists/ftp.us.debian.org_debian_dists_<distribution>_<component>_source_Sources`"
- "`/var/cache/apt/apt-file/ftp.us.debian.org_debian_dists_<distribution>_Contents-<architecture>.gz`" (for `apt-file`)

First 4 types of files are shared by all the pertinent APT commands and updated from command line by "`apt-get update`" and "`aptitude update`".  The "`Packages`" meta data are updated if there is the "`deb`" line in "`/etc/apt/sources.list`". The "`Sources`" meta data are updated if there is the "`deb-src`" line in "`/etc/apt/sources.list`".

The "`Packages`" and "`Sources`" meta data contain "`Filename:`" stanza pointing to the file location of the binary and source packages.  Currently, these packages are located under the "`pool/`" directory tree for the improved transition over the releases.  

Local copies of "`Packages`" meta data can be interactively searched with the help of `aptitude`.  The specialized search command `grep-dctrl`(1) can search local copies of "`Packages`" and "`Sources`" meta data.

Local copy of "`Contents-<architecture>`" meta data can be updated by "`apt-file update`" and its location is different from other 4 ones. See `apt-file`(1). (The `auto-apt` uses different location for local copy of "`Contents-<architecture>.gz`" as default.) 

==== The package state for APT

In addition to the remotely fetched meta data, the APT tool after `lenny` stores its locally generated installation state information in the "`/var/lib/apt/extended_states`" which is used by all APT tools to track all auto installed packages.

==== The package state for aptitude

In addition to the remotely fetched meta data, the `aptitude` command stores its locally generated installation state information in the "`/var/lib/aptitude/pkgstates`" which is used only by it.

==== Local copies of the fetched packages

All the remotely fetched packages via APT mechanism are stored in the "`/var/cache/apt/packages`" until they are cleaned.

==== Debian package file names

Debian package files have particular name structures.

.The name structure of Debian packages
[grid="all"]
`--------------------------------------------------------`--------------------------------------------------------------------------------
package type                                              name structure
------------------------------------------------------------------------------------------------------------------------------------------
The binary package (a.k.a deb)                           `<package-name>_<epoch>:<upstream-version>-<debian.version>-<architecture>.deb`
The binary package for the debian-installer (a.k.a udeb) `<package-name>_<epoch>:<upstream-version>-<debian.version>-<architecture>.udeb`
The source package (upstream source)                     `<package-name>_<epoch>:<upstream-version>-<debian.version>.tar.gz`
The source package (Debian changes)                      `<package-name>_<epoch>:<upstream-version>-<debian.version>.diff.gz`
The source package (description)                         `<package-name>_<epoch>:<upstream-version>-<debian.version>.dsc`
------------------------------------------------------------------------------------------------------------------------------------------

.The usable characters for each component in the Debian package names
[grid="all"]
`--------------------`-------------------------`---------
name component       usable characters (regex) existence
---------------------------------------------------------
`<package-name>`     `[a-z,A-Z,0-9,.,+,-]+`    required
`<epoch>:`           `[0-9]+:`                 optional
`<upstream-version>` `[a-z,A-Z,0-9,.,+,-,:]+`  required
`<debian.version>`   `[a-z,A-Z,0-9,.,+,\~]+`   optional
---------------------------------------------------------

NOTE: You can check package version order by `dpkg`(1), e.g., "`dpkg --compare-versions 7.0 gt 7.\~pre1 ; echo $?`" .

NOTE: http://www.debian.org/devel/debian-installer/[The debian-installer (d-i)] uses `udeb` as the file extension for its binary package instead of normal `deb`.  An `udeb` package is a stripped down `deb` package which removes few non-essential contents such as documentation to save space while relaxing the package policy requirements.  Both `deb` and `udeb` packages share the same package structure.  The "`u`" stands for micro. 

==== The dpkg command

`dpkg`(1) is the lowest level tool for the Debian package management.  This is very powerful and needs to be used with care.

While installing package called "`<package_name>`", `dpkg` process it in the following order.

1. Unpack the deb file ("`ar -x`" equivalent)
2. Execute "`<package_name>.preinst`" using `debconf`(1)
3. Install the package content to the system ("`tar -x`" equivalent)
4. Execute "`<package_name>.postinst`" using `debconf`(1)

The `debconf` system provides standardized user interaction with I18N and L10N (<<_i18n_and_l10n>>) supports.

.The notable files created by `dpkg`
[grid="all"]
`---------------------------------------------`--------------------------------------------------------------------------------
file                                          description of contents
-------------------------------------------------------------------------------------------------------------------------------
`/var/lib/dpkg/info/<package_name>.conffiles` list of configuration files. (user modifiable)
`/var/lib/dpkg/info/<package_name>.list`      list of files and directories installed by the package
`/var/lib/dpkg/info/<package_name>.md5sums`   list of MD5 hash values for files installed by the package
`/var/lib/dpkg/info/<package_name>.preinst`   package script run before the package installation
`/var/lib/dpkg/info/<package_name>.postinst`  package script run after the package installation
`/var/lib/dpkg/info/<package_name>.prerm`     package script run before the package removal
`/var/lib/dpkg/info/<package_name>.postrm`    package script run after the package removal
`/var/lib/dpkg/info/<package_name>.config`    package script for `debconf` system
`/var/lib/dpkg/alternatives/<package_name>`   the alternative information used by the `update-alternatives` command
`/var/lib/dpkg/available`                     the availability information for all the package
`/var/lib/dpkg/diversions`                    the diversions information used by `dpkg`(1) and set by`dpkg-divert`(8)
`/var/lib/dpkg/statoverride`                  the stat override information used by `dpkg`(1) and set by`dpkg-statoverride`(8)
`/var/lib/dpkg/status`                        the status information for all the packages
`/var/lib/dpkg/status-old`                    the first-generation backup of the "`var/lib/dpkg/status`" file
`/var/backups/dpkg.status\*`                  the second-generation backup and older ones of the "`var/lib/dpkg/status`" file
-------------------------------------------------------------------------------------------------------------------------------

The "`status`" file is also used by the tools such as `dpkg`(1), "`dselect update`" and "`apt-get -u dselect-upgrade`".

The specialized search command `grep-dctrl`(1) can search the local copies of "`status`" and "`available`" meta data.

TIP: In http://www.debian.org/devel/debian-installer/[the debian-installer] environment, the `udpkg` command is used to open `udeb` packages.  The `udpkg` command is a stripped down version of the `dpkg` command.

==== The update-alternative command

The Debian system has mechanism to install somewhat overlapping programs peacefully using `update-alternatives`(8).  For example, you can make the `vi` command select to run `vim` while installing both `vim` and `nvi` packages.

--------------------
$ ls -l $(type -p vi)
lrwxrwxrwx 1 root root 20 2007-03-24 19:05 /usr/bin/vi -> /etc/alternatives/vi
$ sudo update-alternatives --display vi
...
$ sudo update-alternatives --config vi
  Selection    Command
 ----------------------------------------------
      1        /usr/bin/vim
*+    2        /usr/bin/nvi

Enter to keep the default[*], or type selection number: 1
--------------------

The Debian alternatives system keeps its selection as symlinks in "`/etc/alternatives/`".  The selection process uses corresponding file in "`/var/lib/dpkg/alternatives/`".

==== The dpkg-statoverride command

**Stat overrides** provided by the `dpkg-statoverride`(8) command are a way to tell `dpkg`(1) to use a different owner or mode for a **file** when a package is installed. If "`--update`" is specified and file exists, it is immediately set to the new owner and mode.

CAUTION: The direct alteration of owner or mode for a **file** owned by the package using `chmod` or `chown` commands by the system administrator is reset by the next upgrade of the package. 

NOTE: I use the word **file** here, but in reality this can be any filesystem object that `dpkg` handles, including directories, devices, etc.

==== The dpkg-divert command

File **diversions** provided by the `dpkg-divert`(8) command are a way of forcing `dpkg`(1) not to install a file into its default location, but to a **diverted** location.  The use of `dpkg-divert` is meant for the package maintenance scripts.  Its casual use by the system administrator is deprecated.

=== Recovery from a broken system

When running `unstable` system, the administrator is expected to recover from broken package management situation.  

CAUTION: Some methods described here are high risk actions.  You have been warned!

==== Incompatibility with old user configuration

If a desktop GUI program experienced instability after significant upstream version upgrade, you should suspect interferences with old local configuration files created by it. If it is stable under newly created user account, this hypothesis is confirmed.  (This is a bug of packaging and usually avoided by the packager.)

To recover stability, you should move corresponding local configuration files and restart the GUI program.  You may need to read old configuration file contents to recover configuration information later.  (Do not erase them too quickly.)

==== Different packages with overlapped files

Archive level package management systems, such as `aptitude`(8) or `apt-get`(1), do not even try to install packages with overlapped files using package dependencies (see <<_package_dependencies>>).

Errors by the package maintainer or deployment of inconsistently mixed source of archives (see <<_packages_from_mixed_source_of_archives>>) by the system administrator may create situation with incorrectly defined package dependencies. When you install a package with overlapped files using `aptitude`(8) or `apt-get`(1) under such situation, `dpkg`(1) which unpacks package ensures to return error to the calling program without overwriting existing files.

CAUTION: The use of third party packages introduces significant system risks via maintainer scripts which are run with root privilege and can do anything to your system.  The `dpkg`(1) command only protects against overwriting by the unpacking.

You can work around such broken installation by removing the old offending package, `<old-package>`, first.

--------------------
$ sudo dpkg -P <old-package>
--------------------

==== Fixing broken package script

When a command in the package script returns error for some reason and the script exits with error, the package management system aborts their action and ends up with partially installed packages.  When a package contains bugs in its removal scripts, the package may become impossible to remove and quite nasty.

For the package script problem of "`<package_name>`", you should look into following package scripts.

- "`/var/lib/dpkg/info/<package_name>.preinst`"
- "`/var/lib/dpkg/info/<package_name>.postinst`"
- "`/var/lib/dpkg/info/<package_name>.prerm`"
- "`/var/lib/dpkg/info/<package_name>.postrm`"

Edit the offending package script from the root using following techniques.

- disable the offending line by preceding "`#`"
- force to return success by appending the offending line with "`|| true`"

Configure all partially installed packages with the following command.

--------------------
# dpkg --configure -a
--------------------

==== Rescue with the dpkg command

Since `dpkg` is very low level package tool, it can function under the very bad situation such as unbootable system without network connection.  Let@@@sq@@@s assume `foo` package was broken and needs to be replaced.

You may still find cached copies of older bug free version of `foo` package in the package cache directory: "`/var/cache/apt/archives/`".  (If not, you can download it from archive of http://snapshot.debian.net/ or copy it from package cache of a functioning machine.)  

If you can boot the system, you may install it by the following command.

--------------------
# dpkg -i /path/to/foo_<old_version>_<arch>.deb
--------------------

TIP: If system breakage is minor, you may alternatively downgrade the whole system as <<_emergency_downgrading>> using the higher level APT system.

If your system is unbootable from hard disk, you should seek other ways to boot it.

1. Boot the system using the debian-installer CD in rescue mode.
2. Mount the unbootable system on the hard disk to "`/target`".
3. Install older version of `foo` package by the following.

--------------------
# dpkg --root /target -i /path/to/foo_<old_version>_<arch>.deb
--------------------

This example works even if the `dpkg` command on the hard disk is broken.

TIP: Any GNU/Linux system started by another system on hard disk, live GNU/Linux CD, bootable USB-key drive, or netboot can be used similarly to rescue broken system.

If attempting to install a package this way fails due to some dependency violations and you really need to do this as the last resort, you can override dependency using `dpkg`@@@sq@@@s "`--ignore-depends`", "`--force-depends`" and other options.    If you do this, you need to make serious effort to restore proper dependency later. See `dpkg`(8) for details.

NOTE: When your system is seriously broken, you should make a full backup of system to a safe place (see <<_backup_and_recovery>>) and should perform a clean installation.  This is less time consuming and produces better results in the end.

==== Recovering package selection data

If "`/var/lib/dpkg/status`" becomes corrupt for any reason, the Debian system loses package selection data and suffers severely.  Look for the old "`/var/lib/dpkg/status`" file at "`/var/lib/dpkg/status-old`" or "`/var/backups/dpkg.status.\*`".

Keeping "`/var/backups/`" in a separate partition may be a good idea since this directory contains lots of important system data.

For serious breakage, I recommend to make fresh re-install after making backup of the system. Even if everything in "`/var/`" is gone, you can still recover some information from directories in "`/usr/share/doc/`" to guide your new installation.

Reinstall minimal (desktop) system.

--------------------
# mkdir -p /path/to/old/system
--------------------

Mount old system at "`/path/to/old/system/`".

--------------------
# cd /path/to/old/system/usr/share/doc
# ls -1 >~/ls1.txt
# cd /usr/share/doc
# ls -1 >>~/ls1.txt
# cd
# sort ls1.txt | uniq | less
--------------------

Then you are presented with package names to install.  (There may be some non-package names such as "`texmf`".)

=== Tips for the package management

==== How to pick Debian packages

You can seek packages which satisfy your needs with `aptitude` from the package description or from the list under "Tasks".

When you encounter more than 2 similar packages and wonder which one to install without "trial and error" efforts, you should use some **common sense**.  I consider following points are good indications of preferred packages.

- Essential: yes > no
- Component: main > contrib > non-free
- Priority: required > important > standard > optional > extra
- Tasks: packages listed in tasks such as "Desktop environment"
- Packages selected by the dependency package (e.g., `python2.4` by `python`)
- Popcon: higher in the vote and install number
- Changelog: regular updates by the maintainer
- BTS: No RC bugs (no critical, no grave, and no serious bugs)
- BTS: responsive maintainer to bug reports
- BTS: higher number of the recently fixed bugs
- BTS: lower number of remaining non-wishlist bugs 

Debian being a volunteer project with distributed development model, its archive contains many packages with different focus and quality.  You must make your own decision what to do with them.

==== Packages from mixed source of archives

CAUTION: Installing packages from mixed source of archives is not supported by the official Debian distribution except for officially supported particular combinations of archives such as `stable` with http://www.debian.org/security/[security updates] and http://www.debian.org/volatile/[volatile updates].

Here is an example of operations to include specific newer upstream version packages found in `unstable` while tracking `testing` for single occasion.

1. Change the "`/etc/apt/sources.list`" file temporarily to single "`unstable`" entry.
2. Run "`aptitude update`".
3. Run "`aptitude install <package-name>`".
4. Recover the original "`/etc/apt/sources.list`" file for `testing`.
5. Run "`aptitude update`".

You do not create the "`/etc/apt/preferences`" file nor need to worry about apt-pinning with this manual approach.  But this is very cumbersome.

CAUTION: When using mixed source of archives, you must ensure compatibility of packages by yourself since the Debian does not guarantee it. If package incompatibility exists, you may break system. You must be able to judge these technical requirements. The use of mixed source of random archives is completely optional operation and its use is not something I encourage you to use.

General rules for installing packages from different archives are followings.

- Non-binary packages ("`Architecture: all`") are **safer** to install.
 * documentation packages: no special requirements
 * interpreter program packages: compatible interpreter must be available
- Binary packages (non "`Architecture: all`") usually face many road blocks and **unsafe** to install.
 * library version compatibility (including "`libc`")
 * related utility program version compatibility
 * Kernel http://en.wikipedia.org/wiki/Application_binary_interface[ABI] compatibility
 * C++ http://en.wikipedia.org/wiki/Application_binary_interface[ABI] compatibility
 * ...

NOTE: In order to make a package to be **safer** to install, some commercial non-free binary program packages may be provided with completely statically linked libraries.  You should still check http://en.wikipedia.org/wiki/Application_binary_interface[ABI] compatibility issues etc. for them.

NOTE: Except to avoid broken package for a short term, installing binary packages from officially unsupported archives is generally bad idea.  This is true even if you use apt-pinning (see <<_tweaking_candidate_version>>).  You should consider chroot or similar techniques (see <<_virtualized_system>>) to run programs from different archives.

==== Tweaking candidate version

WARNING: In `lenny`, `aptitude`(8) has a bug for handling "`/etc/apt/preferences`" file. (http://bugs.debian.org/514930[Bug#514930])

Without the "`/etc/apt/preferences`" file, APT system choses the latest available version as the **candidate version** using the version string.  This is the normal state and most recommended usage of APT system.  All officially supported combinations of archives do not require the "`/etc/apt/preferences`" file since some archives which should not be used as the automatic source of upgrades are marked as **NotAutomatic** and dealt properly.

TIP: The version string comparison rule can be verified with, e.g., "`dpkg  --compare-versions ver1.1 gt ver1.1\~1; echo $?`" (see `dpkg`(1)).

When you install packages from mixed source of archives (see <<_packages_from_mixed_source_of_archives>>) regularly, you can automate these complicated operations by creating the "`/etc/apt/preferences`" file with proper entries and tweaking the package selection rule for **candidate version** as described in `apt_preferences`(5).  This is called **apt-pinning**.

WARNING: Use of apt-pinning by a novice user is sure call for major troubles.  You must avoid using apt-pinning except when you absolutely need it.  

CAUTION: When using apt-pinning, you must ensure compatibility of packages by yourself since the Debian does not guarantee it.  The apt-pinning is completely optional operation and its use is not something I encourage you to use.

CAUTION: Archive level Release files (see <<_archive_level_release_files>>) are used for the rule of `apt_preferences`(5).  Thus apt-pinning works only with "suite" name for http://ftp.us.debian.org/debian/dists/[normal Debian archives] and http://security.debian.org/dists/[security Debian archives]. (This is different from http://www.ubuntu.com/[Ubuntu] archives).  For example, you can do "`Pin: release a=unstable`" but can not do "`Pin: release a=sid`" in the "`/etc/apt/preferences`" file.

CAUTION: When you use non-Debian archive as a part of apt-pinning, you should check what they are intended for and also check their credibility.  For example, Ubuntu and Debian are not meant to be mixed.

NOTE: Even if you do not create the "`/etc/apt/preferences`" file, you can do fairly complex system operations (see <<_rescue_with_the_dpkg_command>> and <<_packages_from_mixed_source_of_archives>>) without apt-pinning.

Here is a simplified explanation of **apt-pinning** technique.  

APT system choses highest Pin-Priority **upgrading** package from available package sources defined in the "`/etc/apt/sources.list`" file as the **candidate version** package.  If the Pin-Priority of the package is larger than 1000, this version restriction for **upgrading** is dropped to enable downgrading (see <<_emergency_downgrading>>). 

Pin-Priority value of each package is defined by "Pin-Priority" entries in the "`/etc/apt/preferences`" file or uses its default value.

.List of the default Pin-Priority value for each package source type
[grid="all"]
`--------------------`----------------------------------
default Pin-Priority package source type
--------------------------------------------------------
990                  **target release** archive 
500                  **normal** archive         
100                  **installed** package      
1                    **NotAutomatic** archive   
--------------------------------------------------------

The **target release** archive can be set by several methods.

- "`/etc/apt/apt.conf`" configuration file with "`APT::Default-Release "stable";`" line
- command line option, e.g., "`apt-get install -t testing some-package`"

The  **NotAutomatic** archive is set by archive server having its archive level Release file (see <<_archive_level_release_files>>) containing "`NotAutomatic: yes`".

The **apt-pinning situation** of <package> from multiple archive sources is displayed by "`apt-cache policy <package>`".

- A line started with "`Package pin:`" lists the package version of **pin** if association just with <package> is defined, e.g., "`Package pin: 0.190`".
- No line with "`Package pin:`" exists if no association just with <package> is defined.
- The Pin-Priority value associated just with <package> is listed right side of all version strings, e.g., "`0.181 700`".
- "`0`" is listed right side of all version strings if no association just with <package> is defined, e.g., "`0.181 0`".
- The Pin-Priority values of archives (defined as "`Package: \*`" in the "`/etc/apt/preferences`" file) are listed left side of all archive paths, e.g., "`200 http:@@@slash@@@/backports.org etch-backports/main Packages`".

Here is an example of **apt-pinning** technique to include specific newer upstream version packages found in `unstable` regularly upgraded while tracking `testing`.  You list all required archives in the "`/etc/apt/sources.list`" file as the following.

--------------------
deb http://ftp.us.debian.org/debian/ testing main contrib non-free
deb http://ftp.us.debian.org/debian/ unstable main contrib non-free
deb http://security.debian.org/ testing/updates main contrib
--------------------

Set the "`/etc/apt/preferences`" file as as the following.

--------------------
Package: *
Pin: release a=testing
Pin-Priority: 500

Package: *
Pin: release a=unstable
Pin-Priority: 200
--------------------

When you wish to install a package named "`<package-name>`" with its dependencies from `unstable` archive under this configuration, you issue the following command which switches target release with "`-t`" option (Pin-Priority of `unstable` becomes 990.).

--------------------
$ sudo apt-get install -t unstable <package-name>
--------------------

With this configuration, usual execution of "`apt-get upgrade`" and "`apt-get dist-upgrade`" (or "`aptitude safe-upgrade`" and "`aptitude full-upgrade`" for `squeeze`) upgrades packages which were installed from `testing` archive using current `testing` archive and packages which were installed from `unstable` archive using current `unstable` archive.

CAUTION: Be careful not to remove "`testing`" entry from the "`/etc/apt/sources.list`" file.  Without "`testing`" entry in it, APT system upgrades packages using newer `unstable` archive.

TIP: I usually edit the "`/etc/apt/sources.list`" file to comment out "`unstable`" archive entry right after above operation.  This avoids slow update process of having too many entries in the "`/etc/apt/sources.list`" file although this prevents upgrading packages which were installed from `unstable` archive using current `unstable` archive.

TIP: If "`Pin-Priority: 20`" is used instead of "`Pin-Priority: 200`" for the "`/etc/apt/preferences`" file, already installed packages having Pin-Priority value of 100 are not upgraded by `unstable` archive even if "`testing`" entry in the "`/etc/apt/sources.list`" file is removed.

If you wish to track particular packages in `unstable` automatically without initial "`-t unstable`" installation, you must create the "`/etc/apt/preferences`" file and explicitly list all those packages at the top of it as the following.

--------------------
Package: <package-1>
Pin: release a=unstable
Pin-Priority: 700

Package: <package-2>
Pin: release a=unstable
Pin-Priority: 700
--------------------

These set Pin-Priority value for each specific package.  For example, in order to track the latest `unstable` version of this "Debian Reference" in English, you should have following entries in the "`/etc/apt/preferences`" file.

--------------------
Package: debian-reference-en
Pin: release a=unstable
Pin-Priority: 700

Package: debian-reference-common
Pin: release a=unstable
Pin-Priority: 700
--------------------

TIP: This apt-pinning technique is valid even when you are tracking `stable` archive.  Documentation packages have been always safe to install from `unstable` archive in my experience, so far.

Here is another example of **apt-pinning** technique to include specific newer upstream version packages found in `experimental` while tracking `unstable`.  You list all required archives in the "`/etc/apt/sources.list`" file as the following.

--------------------
deb http://ftp.us.debian.org/debian/ unstable main contrib non-free
deb http://ftp.us.debian.org/debian/ experimental main contrib non-free
deb http://security.debian.org/ testing/updates main contrib
--------------------

The default Pin-Priority value for `experimental` archive is always 1 (@@@dlt@@@100) since it is **NotAutomatic** archive (see <<_archive_level_release_files>>).  There is no need to set Pin-Priority value explicitly in the "`/etc/apt/preferences`" file just to use `experimental` archive unless you wish to track particular packages in it automatically for next upgrading.

==== Volatile and Backports.org

There are http://www.debian.org/volatile/[debian-volatile project] and http://backports.org[backports.org] archives which provide updgrade packages for `stable`.

WARNING: Do not use all packages available in the **NotAutomatic** archives such as `@-@codename-stable@-@-backports` and `volatile-sloppy`.  Use only selected packages which fits your needs.

CAUTION: http://backports.org[backports.org] is a non-Debian archive, although its packages are signed by Debian developers.

CAUTION: Archive level Release files (see <<_archive_level_release_files>>) are used for the rule of `apt_preferences`(5).  Thus apt-pinning works only with "code" name for http://volatile.debian.org/debian-volatile/dists/[volatile Debian archives]. This is different from other Debian archives.  For example, you can do "`Pin: release a=@-@codename-stable@-@`" but can not do "`Pin: release a=stable`" in the "`/etc/apt/preferences`" file for volatile Debian archives.

Here is an example of **apt-pinning** technique to include specific newer upstream version packages found in `@-@codename-stable@-@-backports` while tracking `@-@codename-stable@-@` and `volatile`.  You list all required archives in the "`/etc/apt/sources.list`" file as the following.

--------------------
deb http://ftp.us.debian.org/debian/ @-@codename-stable@-@ main contrib non-free
deb http://security.debian.org/ @-@codename-stable@-@/updates main contrib
deb http://volatile.debian.org/debian-volatile/ @-@codename-stable@-@/volatile main contrib non-free
deb http://volatile.debian.org/debian-volatile/ @-@codename-stable@-@/volatile-sloppy main contrib non-free
deb http://backports.org/debian/ @-@codename-stable@-@-backports main contrib non-free
--------------------

The default Pin-Priority value for http://backports.org[backports.org] and `volatile-sloppy` archives are always 1 (@@@dlt@@@100) since they are **NotAutomatic** archive (see <<_archive_level_release_files>>).  There is no need to set Pin-Priority value explicitly in the "`/etc/apt/preferences`" file just to use for http://backports.org[backports.org] and `volatile-sloppy` archive unless you wish to track packages automatically for next upgrading.

So whenever you wish to install a package named "`<package-name>`" with its dependency from `@-@codename-stable@-@-backports` archive, you use following command while switching target release with "`-t`" option.

--------------------
$ sudo apt-get install -t @-@codename-stable@-@-backports <package-name>
--------------------

If you wish to upgrade particular packages, you must create the "`/etc/apt/preferences`" file and explicitly lists all packages in it as the following.

--------------------
Package: <package-1>
Pin: release o=Backports.org archive
Pin-Priority: 700

Package: <package-2>
Pin: release o=volatile.debian.org
Pin-Priority: 700
--------------------

Alternatively, with the "`/etc/apt/preferences`" file as the following.

--------------------
Package: *
Pin: release a=stable , o=Debian
Pin-Priority: 500

Package: *
Pin: release a=@-@codename-stable@-@, o=volatile.debian.org
Pin-Priority: 500

Package: *
Pin: release a=@-@codename-stable@-@-backports, o=Backports.org archive
Pin-Priority: 200

Package: *
Pin: release a=@-@codename-stable@-@-sloppy, o=volatile.debian.org
Pin-Priority: 200
--------------------

Execution of "`apt-get upgrade`" and "`apt-get dist-upgrade`" (or "`aptitude safe-upgrade`" and "`aptitude full-upgrade`" for `squeeze`) upgrades packages which were installed from `stable` archive using current `stable` archive and packages which were installed from other archives using current corresponding archive for all archives in the "`/etc/apt/sources.list`" file.

==== Automatic download and upgrade of packages

// APT package since 0.5.28 Ubuntu hoary and 0.5.29 Debian unstable (Sat, 13 Nov 2004) 
// has its own script /etc/cron.daily/apt

The `apt` package comes with its own cron script "`/etc/cron.daily/apt`" to support the automatic download of packages.  This script can be enhanced to perform the automatic upgrade of packages by installing the `unattended-upgrades` package. These can be customized by parameters in "`/etc/apt/apt.conf.d/02backup`" and "`/etc/apt/apt.conf.d/50unattended-upgrades`" as described in "`/usr/share/doc/unattended-upgrades/README`".

The `unattended-upgrades` package is mainly intended for the security upgrade for the `stable` system.  If the risk of breaking an existing `stable` system by the automatic upgrade is smaller than that of the system broken by the intruder using its security hole which has been closed by the security update, you should consider using this automatic upgrade with configuration parameters as the following.

--------------------
APT::Periodic::Update-Package-Lists "1";
APT::Periodic::Download-Upgradeable-Packages "1";
APT::Periodic::Unattended-Upgrade "1";
--------------------

If you are running an `unstable` system, you do not want to use the automatic upgrade since it certainly breaks system some day.  Even for such `unstable` case, you may still want to download packages in advance to save time for the interactive upgrade with configuration parameters as the following.

--------------------
APT::Periodic::Update-Package-Lists "1";
APT::Periodic::Download-Upgradeable-Packages "1";
APT::Periodic::Unattended-Upgrade "0";
--------------------

// I changed mind.  Maybe curent one is good enough.
// {i} With an improved "/etc/cron.daily/apt" file (see [http://bugs.debian.org/478904]), 
// you can safely mark "Option -> Preferences -> [X] Remove obsolete package files 
// after downloading new package lists" in aptitude(8) while creating a file 
// with "APT::Periodic::BackupArchiveInterval=2;" in "/etc/apt/apt.conf.d/".


==== Limiting download bandwidth for APT

If you want to limit the download bandwidth for APT to e.g. 800Kib/sec (=100kiB/sec), you should configure APT with its configuration parameter as the following.

--------------------
APT::Acquire::http::Dl-Limit "800";
--------------------

==== Emergency downgrading

CAUTION: Downgrading is not officially supported by the Debian by design.  It should be done only as a part of emergency recovery process.  Despite of this situation, it is known to work well in many incidents.  For critical systems, You should backup all important data on the system after the recovery operation and re-install the new system from the scratch.

You may be lucky to downgrade from newer archive to older archive to recover from broken system upgrade by manipulating **candidate version** (see <<_tweaking_candidate_version>>).  This is lazy alternative to tedious actions of many "`dpkg -i <broken-package>@@@ul@@@<old-version>.deb`" commands (see <<_rescue_with_the_dpkg_command>>).

Search lines in the "`/etc/apt/sources.list`" file tracking `unstable` as the following.

--------------------
deb http://ftp.us.debian.org/debian/ @-@codename-unstable@-@ main contrib non-free
--------------------

Replace it with the following to track `testing`.

--------------------
deb http://ftp.us.debian.org/debian/ @-@codename-testing@-@ main contrib non-free
--------------------

Set the "`/etc/apt/preferences`" file as the following.

--------------------
Package: *
Pin: release a=testing
Pin-Priority: 1010
--------------------

Run "`apt-get dist-upgrade`" to force downgrading of packages across the system. 

Remove this special "`/etc/apt/preferences`" file after this emergency downgrading. 

TIP: It is good idea to remove (not purge!) as much packages to minimize dependency problems.  You may need to manually remove and install some packages to get system downgraded.  Linux kernel, bootloader, udev, PAM, APT, and networking related packages and their configuration files require special attention.

==== Who uploaded the package?

Although the maintainer name listed in "`/var/lib/dpkg/available`" and "`/usr/share/doc/package_name/changelog`" provide some information on "who is behind the packaging activity", the actual uploader of the package is somewhat obscure.  `who-uploads`(1) in the `devscripts` package identifies the actual uploader of Debian source packages.

==== The equivs package

If you are to compile a program from source to replace the Debian package, it is best to make it into a real local debianized package (`\*.deb`) and use private archive. 

If you chose to compile a program from source and to install them under "`/usr/local`" instead, you may need to use `equivs` as a last resort to satisfy the missing package dependency.

--------------------
Package: equivs
Priority: extra
Section: admin
Description: Circumventing Debian package dependencies
 This is a dummy package which can be used to create Debian
 packages, which only contain dependency information.
--------------------

==== Porting a package to the stable system

For partial upgrades of the `stable` system, rebuilding a package within its environment using the source package is desirable.  This avoids massive package upgrades due to their dependencies.

Add the following entries to the "`/etc/apt/sources.list`" of a `stable` system.

--------------------
deb-src http://http.us.debian.org/debian unstable  main contrib non-free
--------------------

Install required packages for the compilation and download the source package as the following.

--------------------
# apt-get update
# apt-get dist-upgrade
# apt-get install fakeroot devscripts build-essential
$ apt-get build-dep foo
$ apt-get source foo
$ cd foo*
--------------------

Adjust installed packages if needed.

Execute the following.

--------------------
$ dch -i
--------------------

Bump package version, e.g. one appended with "`+bp1`" in "`debian/changelog`"

Build packages and install them to the system as the following.

--------------------
$ debuild
$ cd ..
# debi foo*.changes 
--------------------

==== Proxy server for APT

Since mirroring whole subsection of Debian archive wastes disk space and network bandwidth, deployment of a local proxy server for APT is desirable consideration when you administer many systems on http://en.wikipedia.org/wiki/Local_area_network[LAN].  APT can be configure to use generic web (http) proxy servers such as `squid` (see <<_other_network_application_servers>>) as described in `apt.conf`(5) and in "`/usr/share/doc/apt/examples/configure-index.gz`".  The "`$http_proxy`" environment variable can be used to override proxy server setting in the "`/etc/apt/apt.conf`" file.

There are proxy tools specially for Debian archive.  You should check BTS before using them.

.List of the proxy tools specially for Debian archive
[grid="all"]
`---------------`-------------`------------`----------------------------------------------------------------------------------------------
package         popcon        size         description
------------------------------------------------------------------------------------------------------------------------------------------
`approx`        @-@popcon1@-@ @-@psize1@-@ caching proxy server for Debian archive files (compiled http://en.wikipedia.org/wiki/Objective_Caml[OCaml] program)
`apt-proxy`     @-@popcon1@-@ @-@psize1@-@ Debian archive proxy and partial mirror builder (Python program)
`apt-cacher`    @-@popcon1@-@ @-@psize1@-@ Caching proxy for Debian package and source files (Perl program)
`apt-cacher-ng` @-@popcon1@-@ @-@psize1@-@ Caching proxy for distribution of software packages (compiled C++ program)
`debtorrent`    @-@popcon1@-@ @-@psize1@-@ Bittorrent proxy for downloading Debian packages (Python program)
------------------------------------------------------------------------------------------------------------------------------------------

CAUTION: When Debian reorganizes its archive structure, these specialized proxy tools tend to require code rewrites by the package maintainer and may not be functional for a while.  On the other hand, generic web (http) proxy servers are more robust and easier to cope with such changes.

==== Small public package archive

// Check out : http://people.connexer.com/~roberto/howtos/debrepository

Here is an example for creating a small public package archive compatible with the modern **secure APT** system (see <<_top_level_release_file_and_authenticity>>). Let@@@sq@@@s assume few things.

- Account name: "`foo`"
- Host name: "`www.example.com`"
- Required packages:  `apt-utils`, `gnupg`, and other packages
- URL: "`http://www.example.com/\~foo/`" ( -> "`/home/foo/public_html/index.html`")
- Architecture of packages: "`amd64`"

Create an APT archive key of Foo on your server system as the following.

--------------------
$ ssh foo@www.example.com
$ gpg --gen-key
...
$ gpg -K
...
sec   1024D/3A3CB5A6 2008-08-14
uid                  Foo (ARCHIVE KEY) <foo@www.example.com>
ssb   2048g/6856F4A7 2008-08-14
$ gpg --export -a 3A3CB5A6 >foo.public.key
--------------------

Publish the archive key file "`foo.public.key`" with the key ID "`3A3CB5A6`" for Foo

Create an archive tree called "Origin: Foo" as the following.

--------------------
$ umask 022
$ mkdir -p ~/public_html/debian/pool/main
$ mkdir -p ~/public_html/debian/dists/unstable/main/binary-amd64
$ mkdir -p ~/public_html/debian/dists/unstable/main/source
$ cd ~/public_html/debian
$ cat > dists/unstable/main/binary-amd64/Release << EOF
Archive: unstable
Version: 4.0
Component: main
Origin: Foo
Label: Foo
Architecture: amd64
EOF
$ cat > dists/unstable/main/source/Release << EOF
Archive: unstable
Version: 4.0
Component: main
Origin: Foo
Label: Foo
Architecture: source
EOF
$ cat >aptftp.conf <<EOF
APT::FTPArchive::Release {
  Origin "Foo";
  Label "Foo";
  Suite "unstable";
  Codename "sid";
  Architectures "amd64";
  Components "main";
  Description "Public archive for Foo";
};
EOF
$ cat >aptgenerate.conf <<EOF
Dir::ArchiveDir ".";
Dir::CacheDir ".";
TreeDefault::Directory "pool/";
TreeDefault::SrcDirectory "pool/";
Default::Packages::Extensions ".deb";
Default::Packages::Compress ". gzip bzip2";
Default::Sources::Compress "gzip bzip2";
Default::Contents::Compress "gzip bzip2";

BinDirectory "dists/unstable/main/binary-amd64" {
  Packages "dists/unstable/main/binary-amd64/Packages";
  Contents "dists/unstable/Contents-amd64";
  SrcPackages "dists/unstable/main/source/Sources";
};

Tree "dists/unstable" {
  Sections "main";
  Architectures "amd64 source";
};
EOF
--------------------

You can automate repetitive updates of APT archive contents on your server system by configuring `dupload`.

Place all package files into "`\~foo/public_html/debian/pool/main/`" by executing "`dupload -t foo changes_file`" in client while having "`\~/.dupload.conf`" containing the following.

--------------------
$cfg{'foo'} = {
  fqdn => "www.example.com",
  method => "scpb",
  incoming => "/home/foo/public_html/debian/pool/main",
  # The dinstall on ftp-master sends emails itself
  dinstall_runs => 1,
};

$cfg{'foo'}{postupload}{'changes'} = "
  echo 'cd public_html/debian ;
  apt-ftparchive generate -c=aptftp.conf aptgenerate.conf;
  apt-ftparchive release -c=aptftp.conf dists/unstable >dists/unstable/Release ;
  rm -f dists/unstable/Release.gpg ;
  gpg -u 3A3CB5A6 -bao dists/unstable/Release.gpg dists/unstable/Release'| 
  ssh foo@www.example.com  2>/dev/null ;
  echo 'Package archive created!'";
--------------------

The **postupload** hook script initiated by `dupload`(1) creates updated archive files for each upload.

You can add this small public archive to the apt-line of your client system by the following.

--------------------
$ sudo bash
# echo "deb http://www.example.com/~foo/debian/ unstable main" \
   >> /etc/apt/sources.list
# apt-key add foo.public.key
--------------------

TIP: If the archive is located on the local filesystem, you can use "`deb file:/@@@slash@@@/home/foo/debian/ ...`" instead.

==== Recording and copying system configuration

You can make a local copy of the package and debconf selection states by the following.

--------------------
# dpkg --get-selections '*' > selection.dpkg
# debconf-get-selections    > selection.debconf
--------------------

Here, "`\*`" makes "`selection.dpkg`" to include package entries for "purge" too.

You can transfer these 2 files to another computer, and install there with the following.

--------------------
# dselect update
# debconf-set-selections < myselection.debconf
# dpkg --set-selections  < myselection.dpkg
# apt-get -u dselect-upgrade    # or dselect install
--------------------

If you are thinking about managing many servers in a cluster with practically the same configuration, you should consider to use specialized package such as `fai` to manage the whole system.

==== Converting or installing an alien binary package

`alien`(1) enables the conversion of binary packages provided in Red Hat `rpm`, Stampede `slp`, Slackware `tgz`, and Solaris `pkg` file formats into a Debian `deb` package.  If you want to use a package from another Linux distribution than the one you have installed on your system, you can use `alien` to convert it from your preferred package format and install it.  `alien` also supports LSB packages.

WARNING: `alien`(1) should not be used to replace essential system packages, such as `sysvinit`, `libc6`, `libpam-modules`, etc.  Practically, `alien`(1) should only used for **non-free** binary-only packages which are LSB compliant or statically linked.  For free softwares, you should use their source packages to make real Debian packages.

==== Extracting package without dpkg

The current "`\*.deb`" package contents can be extracted without using `dpkg`(1) on any http://en.wikipedia.org/wiki/Unix-like[Unix-like] environment using standard `ar`(1) and `tar`(1).

--------------------
# ar x /path/to/dpkg_<version>_<arch>.deb
# ls
total 24
-rw-r--r-- 1 bozo bozo  1320 2007-05-07 00:11 control.tar.gz
-rw-r--r-- 1 bozo bozo 12837 2007-05-07 00:11 data.tar.gz
-rw-r--r-- 1 bozo bozo     4 2007-05-07 00:11 debian-binary
# mkdir control
# mkdir data
# tar xvzf control.tar.gz -C control
# tar xvzf data.tar.gz -C data
--------------------

You can also browse package content using the `mc` command.

==== More readings for the package management

You can learn more on the package management from following documentations.

- Primary documentations on the package management:
 * `aptitude`(8), `dpkg`(1), `tasksel`(8), `apt-get`(8), `apt-config`(8), `apt-key`(8), `sources.list`(5), `apt.conf`(5), and `apt_preferences`(5);
 * "`/usr/share/doc/apt-doc/guide.html/index.html`" and "`/usr/share/doc/apt-doc/offline.html/index.html`" from the `apt-doc` package; and
 * "`/usr/share/doc/aptitude/html/en/index.html`" from the `aptitude-doc-en` package.
- Official and detailed documentations on the Debian archive:
 * http://www.debian.org/doc/debian-policy/ch-archive["Debian Policy Manual Chapter 2 - The Debian Archive"],
 * http://www.debian.org/doc/manuals/developers-reference/resources.html#archive["Debian Developer@@@sq@@@s Reference, Chapter 4 - Resources for Debian Developers 4.6 The Debian archive"], and
 * http://www.debian.org/doc/FAQ/ch-ftparchives["The Debian GNU/Linux FAQ, Chapter 5 - The Debian FTP archives"].
- Tutorial for building of a Debian package for Debian users:
 * http://www.debian.org/doc/manuals/maint-guide/["Debian New Maintainers' Guide"].

// APT HOWTO REMOVED FROM LENNY
// * [http://www.debian.org/doc/manuals/apt-howto/ APT HOWTO].  (A very nice introduction for the user.)
