Debian Med Group Policy

Debian Med Group Policy Andreas Tille First review tille@debian.org David Paleino Initial writing d.paleino@gmail.com Charles Plessy Contributions in 2008 and 2011 plessy@debian.org $ policy.xml rev. @REV@ - @DATE@ (@AUTHOR@) $ Source: svn://svn.debian.org/debian-med/trunk/community/website/docs/policy.xml Debian Med Group Introduction Debian Med is a Debian Pure Blend with the aim to develop Debian into an operating system that is particularly well fit for the requirements for medical practice and research. The Debian Med project presents packages that are associated with medicine, pre-clinical research, and life sciences. Its developments are mostly focused on three areas for the moment: medical practice, imaging and bioinformatics. Over the previous years, several initiatives have spawned that address the scientific disciplines like chemistry or bioinformatics. Debian Med is not a competition to these efforts but a platform to present the packages to the community as a Debian Pure Blend. How to Contribute From the developer to the user, there is a long chain of tasks in which we always welcome participation. First we must keep ourselves informed about the software landscape in biology and medicine. Software to be packaged is chosen according to criteria such as users' need and the consistency of the distribution. Once in Debian, the software is monitored for its quality and bugs are fixed, if possible in collaboration with the upstream maintainer(s). All this work would not be very useful if it remains confidential. We also dedicate some time to advertise it to the world via www.debian.org and to ease the integration of new members. Please contact us on debian-med@lists.debian.org if you want to help to make medical and biological software available to Debian users. Read the Membership section if you're interested in joining us. If you speak a language other than English, you can contribute rightaway with translations of package descriptions at ddtp.debian.net. When working on these, you will find immediate targets for improvements of the original English versions, too. For these, though, you need access to Debian Med's source code repository. Very welcome are tutorials that guide Debian users towards the use of packages to their immediate benefit. You may also consider to write respective articles for Magazines, be they online or in print. Membership To request membership to this group, please go on our Alioth page, or directly follow this link. Remember that you must have an Alioth account before requesting membership (see here to request an Alioth account). Essential readings The Debian Policy: packages must conform to it. The Developers Reference: details best packaging practices. The New Maintainer's Guide: puts a bit of the two above in practice. The Debian Med Policy (this document): explains how the work is organised in our team.. Repositories We use Subversion (SVN) and Git repositories, hosted by Debian. You can have a look at each repository through Alioth's web interfaces: wsvn viewsvn and gitweb. The Subversion repository is the primary location for our source packages. However, the Git repository is free to use for special cases, for instance when the Subversion tags take a disproportionated size, when the maintainer is much more comfortable with Git than Subversion, when some special features of Git or git-buildpackage are desired, or more simply when the maintainer wants to take the opportunity to familiarise with Git. For most direct operations on the repositories in Alioth, an umask of 002 is necessary to avoid problems of missing write permission to the other team members. Give me the source! To check the sources of a package (referred as <package> below) in our repositories, use the debcheckout command, from the devscripts package. If you are a member of Debian Med or a Debian developer, with account name <username>, you have write permission: debcheckout <username> <package> # When the package is already in the archive debcheckout <username> ssh://svn.debian.org/debian-med/trunk/packages/<package>/trunk <package> debcheckout <username> git://git.debian.org/debian-med/<package>.git For read-only access, remove the option. SSH tips You can avoid specifying your Alioth user name by setting it in ~/.ssh/config as follows. Note that in that case, with debcheckout you will can replace the option by the option, for a shorter typing. Host *.debian.org User your-user-name You can avoid typing your SSH password again and again using the ssh-add command. On remote connections the SSH agent needs to be enabled with the command

eval
				ssh-agent

. In case of trouble getting SSH working it is a good idea to read this wiki article. Subversion repository structure The SVN repository is structured as follows: debian-med/ └ trunk/ ├ community/ │ ├ debtags/ │ ├ infrastructure/ │ └ website/ └ packages/ ├ <package A>/ │ ├ branches/ │ ├ tags/ │ └ trunk/ │ └ debian/ ├ <package B>/ │ ├ branches/ │ ├ tags/ │ └ trunk/ │ └ debian/ … We are currently considering an alternative layout in which all the trunk, tags and branches directories are grouped together, so that developers can checkout trunks without tags. Subversion tips Suggested aliases for svn-buildpackage: alias svn-b='svn-buildpackage -us -uc --svn-ignore' alias svn-br='svn-b --svn-dont-purge --svn-reuse' alias svn-bt='svn-buildpackage --svn-tag' svn-inject -o sets up the mergeWithUpstream property for the SVN directories where packages are stored. In case svn-inject was not used, you can do it by hand with the command

svn propset
				mergeWithUpstream
				1 debian

. To download the upstream sources (if there is a debian/watch file):

echo
				"origDir=.." >> .svn/deb-layout && uscan
				--force-download

. Alternatively, you can try debian/rules get-orig-source. If you're a Debian developer or a member of the Debian Med group on Alioth, you can commit your changes: svn commit (also svn ci). Otherwise, you can ask to be added to the group (see the Membership section), or send the result of svn diff to the mailing list (gzip -9 it, if it's too large). It may happen that a package version has been uploaded to Debian repositories, and you forgot to tag the last build with svn-buildpackage --svn-tag. You can tag this package also retroactively. A first step, creating the tags directory, can be achieved in two ways: either create it locally as sibling of trunk/ with svn mkdir tags, and commit with svn commit, or create it remotely with svn mkdir svn+ssh://user@svn.debian.org/svn/debian-med/trunk/packages/<package>/tags. After the tags directory has been created, you're ready to tag the package: svn-buildpackage . The avoids debian/changelog to be marked as UNRELEASED. Here is an example session where a package is prepared manually: # Check out debian-med/trunk, cd to debian-med/trunk/packages # Create new project folder with svn mkdir projectname svn mkdir projectname/trunk # Put orig.tar.gz in place mv some_version.orig.tar.gz projectname # Inform svn-buildpackage about the location of the orig.tar.gz echo "origDir=.." > projectname/trunk/.svn/deb-layout # Untar source tree for development cd projectname tar xzvf some_version.orig.tar.gz cd some-version # entering the unpackaged source tree if [ ! -d debian ]; then dh_make ; fi mv debian ../trunk ln -s ../trunk/debian . # continue development fakeroot ./debian/rules binary # until it works, finally cd .. # debian directory is the only thing that is stored in svn, to be merged with upstream svn propset mergeWithUpstream 1 trunk/debian # See if things can be compiled via svn-buildpackage cd trunk svn-buildpackage -uc -us # check build lintian ../build-area/*changes Common Git repository structures The area on Alioth that contains the Git repositories (/git/debian-med) is not itself structured, but the repositories can be arranged in different possible layouts. <command>git-buildpackage</command> Most of our packages using Git and stored in Alioth are managed with the git-buildpackage helper. The master branch contains the full source package. The upstream branch contains the upstream source, after repacking when necessary (non-free files, large convenience code copies, …). The pristine-tar contains the data necessary to recreate an original tarball from the repository with a reproducible checksum. Debian releases are tagged with names like debian/debianversion and upstream releases are tagged with names like upstream/upstreamversion. Social Git For some projects, like the ones hosted on GitHub or Gitorious, it may be easier to forward changes made in the Debian package if this one is itself hosted on the same platform, as a clone. In that case, the layouts may vary from package to packages. However, there are good chances that the branch that contains the Debian source package is called debian. Git tips Example to create a new git repository for a package where the upstream sources are distributed as file archives (tar, zip, …): mkdir package cd package git init git import-orig /path/to/package_version.orig.tar.gz dh_make The above steps will create a repository with the appropriate layout for git-buildpackage, with three branches: master (where the Debian development will happen), pristine-tar used by the pristine-tar tool during the package build process to recreate the original tarball, and upstream, which will contain the upstream source. To display your name and more nicely in the Git log, you should instruct Git to do so (the option is to say Git these are the default parameters for every Git repository you commit to, without it the settings will be per-repository only): git config git config To clone and follow every branch of a git repository containing a package that is already in the Debian archive, you can use the debcheckout command with its option. To restrict the tracked branch to the standard ones used by git-buildpackage, master upstream pristine-tar can be passed instead of the wildcard. Example commands to track new branches: git branch git branch If the devscripts variables DEBEMAIL and DEBFULLNAME are set, debcheckout will set git's options user.email and user.name accordingly. To push changes to Alioth, a remote branch needs to be configured. This is done automatically after cloning a repository, for instance with debcheckout. The default remote branch is called origin. Here are example commands to set up Alioth as a remote branch on a freshly created repository: git remote add origin git+ssh://git.debian.org/git/debian-med/package.git There is a small script called setup-repository in the /git/debian-med directory on Alioth.

./setup-repository

will create a packagename.git repository on with the proper hooks set up for our team. To push the package (make sure you've added the alioth remote!), do the following:git push . For the first push, it's necessary to specify . The next time you will push, a git push will suffice. git pushthis will only push the master branch unless it is somehow related to other branches), so be sure to also do a run with , and one with if you created new tags. To tag a release: git tag . You can also easily retroactively make tags: git tag . Remember to git push --tags. In particular for Git repositories that are not stored in Alioth, the layout can differ from git-buildpackage conventions. In that case, look for instance for a branch called debian. If upstream manages his sources with Git, the following makefile script can help producing a version number when no Git tag is available: SOURCEPKG=$(shell dpkg-parsechangelog | sed -n 's/^Source: $.*$/\1/p') UPSTREAM=$(shell dpkg-parsechangelog | sed -n 's/^Version: $.*$-[^-]*/\1/p') SHA1=$(lastword $(subst ~g, ,$(UPSTREAM))) ORIG=${SOURCEPKG}_${UPSTREAM}.orig.tar.gz describe-current-version: git describe --tags upstream | sed 's,^release-,,;s,-,+,;s,-,~,;' get-orig-source: git archive --format=tar $(SHA1) | gzip -9 > ../$(ORIG) To make git-buildpackage builds the package with a chroot, you can add the following to the configuration file ~/.gbp.conf or debian/gbp.conf: [DEFAULT] builder = ~/bin/git-pbuilder cleaner = fakeroot debian/rules clean pristine-tar = True [git-buildpackage] # use this for more svn-buildpackage like behaviour: export-dir = ../build-area/ tarball-dir = ../tarballs/ With this configuration file you're specifying that git-buildpackage will use ~/bin/git-pbuilder as the builder script. This is an example script you can use: #!/bin/sh set -e pdebuild --pbuilder cowbuilder --debbuildopts "-i\.git -I.git $*" rm ../*_source.changes This will build the package inside the default cowbuilder chroot, while passing any more parameters directly do dpkg-buildpackage. Migration of a source package from Subversion to Git. For packages were we set the mergeWithUpstream property, which excludes the upstream sources, there is no easy way to prepare a Git repository from our Subversion repository that would look like the package was always managed in Git. Nevertheless, the following recipe will generate a Git repository that contains all the history of the debian directory, plus a collection of selected upstream source releases. Start the conversion as explained in the Alioth/Git page of the Debian wiki. To be consistent with a later usage of git-buildpackage, it is preferable to prefix the tag names debian. During this conversion, you can take advantage of the file /trunk/community/infrastructure/comitters in the Subversion repository, and expand it if necessary. Import of the upstream original tarballs that you find relevant, for instance the ones that were part of a stable release). When this paragraph was written, it was necessary to follow special instruction detailed in bug 471560. Upload to git.debian.org/git/debian-med and set up the hooks and other meta-data as explained in the Alioth/Git page of the Debian wiki. Document in the Subversion repository that the package has been transferred elsewhere, for instance with a file named README.status containing the new VCS URL. Delete the package in the Subversion repository after its VCS URL in Stable has been updated by the next release. It is also possible to prepare a Git repository containing some of the package's history using the command git-import-dscs --debsnap from the helper toolkit git-buildpackage. This will download all the versions of a package available in snapshots.debian.org and create tags for each of them. Note that as this paragraph is written, the tool is not aware that backports should be a different branch Updating a source package managed with Git Most source packages maintained as Git repositories in Debian Med are using the git-buildpackage helper toolkit. In doubt, try this one first. git-buildpackage's command git-import-orig allows very easy update of the upstream branch when the original sources are distributed as a compressed archive. Its option --pristine-tar is useful for stabilizing the MD5 sum of the “orig.tar.gz” produced when building a source package from the repository alone (not doing so results in archive rejection of package updates). With recent versions of git-buildpackage, it is often unnecessary to rename the freshly downloaded original upstream archive. Packaging Announcing intent to package If you intent to work on a Debian package you should follow the normal Debian rules and file a WNPP bug report. It is a good idea to keep the Debian Med mailing list debian-med@lists.debian.org in CC and to set it as the owner of the ITP to keep your co-workers informed. This will ensure that we notice if for some reason the package has not been uploaded. In addition, please add the package to the task file where it fits the best, and document your ITP number using the WNPP field name. Backports Debian offers backports to its stable version. Backports of Debian Med packages should be kept as branches in our Subversion or Git repositories. PPA for Ubuntu Debian Med operates a Personal Package Archive (PPA) on Launchpad, where packages are backported for Ubuntu. There is currently no team policy on what to build. Please keep in mind issues like the possibility to upgrade to the next Ubuntu stable release. Packages that are backports can be made inferior in version by using a tilde. If the package contains additional development, a version number without the tilde will make it higher, but not as high as the next Debian revision. For example: 2.12.0-1~natty1 (backport in PPA) < 2.12.0-1 (from Debian in Ubuntu) < 2.12.0-1natty1 (in PPA, containing additions) < 2.12.0-2 (from Debian in Ubuntu). Packages sent to this PPA may be kept as branches in our repositories. R packages GNU R sometimes introduces backward incompatibilities, so the current practice is to make packages depend on versions equal or higher to the one against which they were built. When using r-dev, this can be acheived by adding the following command in debian/rules. Note the tilde, that ensures that pre-releases can be used as well. install/r-$(debRreposname)-$(cranName):: echo "R:Depends=r-base-core (>= $(shell R --version | head -n1 | perl -ne 'print / +([0-9]\.[0-9]+\.[0-9])/')~)" >> debian/r-$(debRreposname)-$(cranName).substvars Tasks The Debian Med Debian Pure Blend is organised by tasks, that group packages around broad themes such as medical imaging for instance. The tasks list programs that are already packaged in Debian as well as packages in preparation. The tasks files are not hosted in the Debian Med repositories, but in the Debian Blends repository. Nevertheless, all members of the Debian Med project on the Alioth forge have write access to the Blends subversion repository. You can easily check out its sources with the command debcheckout -a debian-med. The syntax of the tasks files is very similar to Debian control files, and described in the Debian Blends website. Policy <filename>debian/control</filename> Section Should be science for the source package. Priority Should be optional unless forbidden by the Debian policy (see §2.5). Packages of priority extra are excluded from some QA tests. Maintainer Maintainer should be Debian Med Packaging Team debian-med-packaging@lists.alioth.debian.org. Please subscribe to this list if you list yourself in the Uploaders: field of one of Debian Med's packages. You can refer to the QA page corresponding to this email to gather information about the packages. Upload by Debian Maintainers Should be enabled with the field DM-Upload-Allowed: yes. This means that when an Uploader becomes Debian Maintainer, he will immediately get the possibility to upload the package to Debian. Please consider this when you sponsor packages in which some Uploaders are added. Uploaders Please add yourself as an uploader when you have a significant interest in a package. Being Uploader means that you are expected to answer to the bug reports. For more occasional works, you can do a team upload. Standards-Version Please always use the latest unless there are concerns for backporting. If no changes are needed, please indicate this fact in the changelog, and increment the value of the field. Homepage should be documented whenever possible Vcs-Svn: and Vcs-Browser: Please use the following templates: Vcs-Svn: svn://svn.debian.org/debian-med/trunk/packages/<package>/trunk/ Vcs-Browser: http://svn.debian.org/wsvn/debian-med/trunk/packages/<package>/trunk/ or Vcs-Git: git://git.debian.org/debian-med/<package>.git Vcs-Browser: http://git.debian.org/?p=debian-med/<package>.git <filename>debian/copyright</filename> We use the proposed machine-readable format for the debian/copyright file. The Source field does not need to contain the full URL to the particular version that is being packaged, since this can be determined by the uscan program with the debian/watch file. Please list yourself in the Files: debian/* section if you think that your contributions are not trivial and therefore subjected to copyright. Please chose a license that is compatible with the program you package. You can also use same as if it were in the public domain or same as the packaged program itself. To verify the correct syntax of the debian/copyright file you can use: config-edit . <filename>debian/changelog</filename> Packages hosted in our Subversion repository, that have been modified but not uploaded must use UNRELEASED as a distribution name. This can be done automatically by declaring DEBCHANGE_RELEASE_HEURISTIC=changelog in ~/.devscripts and using dch. <filename>debian/gbp.conf</filename> Packages managed with git-buildpackage may mark this by including a debian/gbp.conf configuration file. The following recommended values will make pristine-tar used by default, and otherwise set a directory layout similar to the one we use with svn-buildpackage. [DEFAULT] pristine-tar = True [git-buildpackage] # use this for more svn-buildpackage like behaviour: export-dir = ../build-area/ tarball-dir = ../tarballs/ <filename>debian/README.source</filename> This file is recommended by the Policy (§ 4.14) from version 3.8.0 for documenting source package handling. Please follow the recommendation. For instance, this file is needed when we use a patch system, when the upstream sources are in another format than gzipped tar archive, when we repack the sources,… <filename>debian/README.test</filename> This file was (recommended by the Security team) for describing to others than the regular maintainer how the package's functionality can properly be tested. Debhelper Debhelper uses compatibility levels to control the behaviour of its commands. We currently recommend to use the level 8 which is available in current Stable (Squeeze) and backported to Oldstable. However, there is no urgent need to touch packages only because it has an older Debhelper version. It is strongly recommended to use the short dh notation in debian/rules files which makes code factorisation very simple and easy to understand the packaging for other members of the team. Even complex packaging becomes quite transparent this way. CDBS Before the short dh notation of debhelper existed CDBS was the only way to factorise code in debian/rules files. So it was recommended in the past but it turned out that CDBS is badly documented and recently introduced incompatible changes. So it is sometimes reasonable to switch from CDBS to dh if some problems in the packaging might occure. It is technically possible to build CDBS packages using Debhelper without the debian/compat file. Please do not, and always include such a file according to the above guidelines. Version control systems We are currently using two different version control systems, Subversion and Git. Source package stored in a Subversion repository We often use the mergeWithUpstream workflow. In that case, please keep all the modifications in the debian directory, and use the option of svn-buildpackage, as in the following example:

svn-inject
					 package.dsc svn+ssh://svn.debian.org/svn/debian-med/trunk/packages/

. Source package stored in a Git repository Git repositories should be stored in the /git/debian-med directory on Alioth and created with the setup-repository script available there. There, they must give write access to the debian-med Alioth group and all the Debian Developers, with appropriate Unix permissions (including SGID bit on directories) and ACLs. See /git/debian-med itself as an example. setup-repository does this automatically. Git repositories managed with a helper tool should announce it. For instance, to show that git-buildpackage is used, the package can contain a configuration file in debian/gbp.conf. Tags Tags indicate the revision corresponding to uploaded packages. For Subversion, the version number is used, and for Git, debian/ is added before the version number. In the Subversion repository, older tags may be deleted to save space. New package Try to inject a new package only after successfully building it with dpkg-buildpackage (or any wrapper around it). Use a file like debian/DRAFT to mention when the package is a draft. The Debian Med Blend tasks Once you injected a new package please make sure that it is mentioned in the appropriate tasks file in the SVN source of the debian-med Blend package. Some team members watch the changes in the Debian Med packaging pool but it helps if the maintainer of a new package verifies that everything is in the right place. Building and tagging the packages We prefer that uploaded packages are built in a chroot, to provide similar build environment to the whole team. After upload, please tag the Suvbersion or Git repository. Handling patches Often happens that the upstream code doesn't fit well into the Debian distribution: be this wrong paths, missing features, anything that implies editing the source files. When you directly edit upstream's source files, your changes will be put into a .diff.gz file if you use the 1.0 source format and in a monolithic patch if you use the 3.0 (quilt) format. To better organise the patches and group the by function, please use a patch handling system which keeps patches under the debian/patches directory. The 3.0 (quilt) Dpkg source format provides its own patch system. Apart from this, the most popular is quilt. simple-patchsys, from the CDBS package, is deprecated since version 0.4.85. dpatch has been popular as well, but is not compatible with the 3.0 (quilt) source format and is planned to be removed 2017. Please don't use any other patch system in Debian Med, unless absolutely necessary. Using <command>quilt</command> Using quilt is rather easy. First, make sure you have correctly setup quilt: open .quiltrc in your home directory (create it if you don't have one), and make sure it looks like this:

QUILT_DIFF_ARGS="--no-timestamps --no-index" QUILT_REFRESH_ARGS="--no-timestamps --no-index" QUILT_PATCHES="debian/patches"

After this, you're ready to start working with quilt. Creating a patch To create a patch, use the new command. Run:

quilt new <patch_name>.patch

This will create (if it doesn't exist yet) a debian/patches/series file, which contains all the patches to be applied by quilt. Moreover, the new patch is also the topmost (the currently applied). Now start editing files, with:

quilt edit <file>

and repeat the process for each file the patch is involved with. At the end, run

quilt refresh

This will compare the noted state of the edited files with the current state, and will produce a patch in debian/patches. Remember: the patch is currently applied (you can check this with quilt applied). Applying and unapplying patches Just two easy commands to do the job: quilt pop will unapply the topmost patch. quilt push will apply the next patch in debian/patches/series. You can just add a "-a" flag to the commands above, to respectively apply/unapply all patches in the series. You can check which patches are applied/unapplied with, respectively, quilt applied and quilt unapplied. Editing patches To edit a patch, first make it the topmost:

quilt push <patch_name>

If the patch is already applied, but is not the topmost, run quilt pop until it becomes the currently applied one. You can now run quilt edit on the files you want to change, and, when you're done, quilt refresh. Renaming patches Sometimes it's useful to rename a patch. Without any hassle, do:

quilt rename -P <old_name>.patch <new_name>.patch

Other commands Please see man 1 quilt to have a comprehensive list of commands. Integration in the build process Add in the very first part of debian/rules (i.e. before the targets), the line:

include /usr/share/quilt/quilt.make

Please use this to import patch and unpatch rules instead of writing them, and remember to add the needed dependencies to its targets:

... build: patch build-stamp build-stamp: configure ...

This kind of dependency will ensure that if you also patch the build system, you get a working patched build process. Don't also put configure as a dependency of build (leave it in build-stamp): that may cause problems during parallel buildings (i.e. the -j flag of make). Now add a dependency to the clean target:

... clean: unpatch ...

If you've also patched the build system, using upstream's clean target might fail. This is what you should do:

... clean: clean-patched unpatch clean-patched: ...

Obviously, you could always use an approach like this, but it's an useless complication if you don't patch the build system, and you should keep debian/rules the simplest you can. Using <command>dpatch</command> Removal of dpatch is planned for 2017. The commands below are for reference, but the use of dpatch in Debian Med is discouraged. The quilt package contains a conversion script, /usr/share/doc/quilt/examples/dpatch2quilt.sh. Creating a patch: dpatch-edit-patch Applying and unapplying patches: apply(-all), unapply(-all), status Editing patches: dpatch-edit-patch Renaming patches: is this possible? Other commands: please see man 1 dpatch for a comprehensive list of commands. Integration in the build process: follow the instructions for quilt and adapt the path of inclusion to /usr/share/dpatch/dpatch.make