| 3 |
Title: Debian Package Maintenance Hub |
Title: Debian Package Maintenance Hub |
| 4 |
DEP: 2 |
DEP: 2 |
| 5 |
State: DRAFT |
State: DRAFT |
| 6 |
Date: 2011-01-13 |
Date: 2012-01-13 |
| 7 |
Drivers: Raphael Hertzog <hertzog@debian.org> |
Drivers: Raphael Hertzog <hertzog@debian.org> |
| 8 |
URL: http://dep.debian.net/deps/dep2 |
URL: http://dep.debian.net/deps/dep2 |
| 9 |
Abstract: |
Abstract: |
| 21 |
This new package maintenance infrastructure is needed: |
This new package maintenance infrastructure is needed: |
| 22 |
|
|
| 23 |
* to fix long standing problems; |
* to fix long standing problems; |
| 24 |
* to implement new features that will help us to ensure that all Debian |
* to provide a clean base to implement new features: |
| 25 |
packages are well maintained; |
* that will help maintainers do a better job; |
| 26 |
* to be more pro-active and to detect problems sooner. |
* that will help packaging teams to organize themselves; |
| 27 |
|
* that will help the QA team to ensure that all Debian packages are |
| 28 |
|
well maintained. |
| 29 |
|
|
| 30 |
### Problems to solve |
### Problems to solve |
| 31 |
|
|
| 43 |
discussions on this topic. |
discussions on this topic. |
| 44 |
|
|
| 45 |
This makes it very painful to change/switch the Maintainer field because |
This makes it very painful to change/switch the Maintainer field because |
| 46 |
people have to update their PTS subscriptions accordingly. |
people have to update their PTS subscriptions accordingly. |
| 47 |
|
|
| 48 |
#### Duplication of work / inconsistency between the DDPO and the PTS |
#### Duplication of work / inconsistency between the DDPO and the PTS |
| 49 |
|
|
| 62 |
|
|
| 63 |
### Goals |
### Goals |
| 64 |
|
|
| 65 |
#### Tracking commitments |
#### Provide a working replacement for DDPO/PTS |
| 66 |
|
|
| 67 |
A package can only be well maintained if it has at least one or more |
Since the service aims to merge the DDPO and the PTS, it must be a working |
| 68 |
committed maintainers. This new infrastructure should let us monitor |
replacement of both services and its set of features must englobe the |
| 69 |
actions that prove or disprove the commitment of the persons who have |
features of the actual services. |
|
a relationship with the package. Contributors should also be able to |
|
|
state explicitly the nature of their relationship with the package (lead |
|
|
maintainer, co-maintainer, backup maintainer, follower/lurker, etc.). |
|
|
|
|
|
Packages without (enough) committed maintainers can then be advertised as |
|
|
needing help before it's too late and before the package suffered too |
|
|
much. Maintainers who are failing to keep up with the commitments they set |
|
|
for themselves can be automatically prodded. |
|
| 70 |
|
|
| 71 |
#### Support alternate notification systems |
#### Support alternate notification systems |
| 72 |
|
|
| 77 |
new maintainers can then have access to some historic information |
new maintainers can then have access to some historic information |
| 78 |
that used to be private for no good reasons. |
that used to be private for no good reasons. |
| 79 |
|
|
| 80 |
|
#### Enable new interactions with maintainers |
| 81 |
|
|
| 82 |
Design of the new infrastructure |
The central role of this new "communication infrastructure" makes it |
| 83 |
-------------------------------- |
possible to design new interactions with maintainers. Instead of being |
| 84 |
|
only a source of information, the infrastructure could be used to query |
| 85 |
|
package maintainers and/or let them provide supplementary information. |
| 86 |
|
|
| 87 |
|
This could be used to improve the MIA tracking process. |
| 88 |
|
|
| 89 |
|
This infrastructure would also be a more natural place to store the |
| 90 |
|
"available for Debian work" boolean flag ("vacation") that's currently |
| 91 |
|
never used because it's buried in db.debian.org and that it's not |
| 92 |
|
practical to update it. |
| 93 |
|
|
| 94 |
|
This infrastructure could also be used to let maintainers document the |
| 95 |
|
responsibilities that they have agreed to endorse, and describe the |
| 96 |
|
associated commitments. That way it would be easier to detect packages |
| 97 |
|
that cannot be well maintained because the set of maintainers do not cover |
| 98 |
|
all the tasks that must be assumed to have a properly maintained package. |
| 99 |
|
|
| 100 |
|
#### Replace maintenance mailing lists |
| 101 |
|
|
| 102 |
|
Packaging teams often separate the mailing list that gets the bug traffic and |
| 103 |
|
other notifications from their main discussion mailing list. This new |
| 104 |
|
infrastructure should entirely replace the former kind of mailing lists. |
| 105 |
|
Anybody receiving notifications and information directed to the package |
| 106 |
|
maintainer should get them via this new infrastructure. |
| 107 |
|
|
| 108 |
|
It allows us to know how many people are notified for a given problem. |
| 109 |
|
If nobody is notified, the package is effectively orphaned. A more |
| 110 |
|
interesting case to detect is when several persons are being notified but |
| 111 |
|
all of them are MIA or marked as not being available for Debian (busy/in |
| 112 |
|
vacation). |
| 113 |
|
|
| 114 |
|
#### Provide new services to packaging teams |
| 115 |
|
|
| 116 |
|
Given that this infrastructure would have native support for packaging |
| 117 |
|
teams, it would also be a good place to offer some standardized services |
| 118 |
|
for them. |
| 119 |
|
|
| 120 |
|
For example, one of the central tool for teams are their VCS and it can be |
| 121 |
|
useful for teams to be able to monitor the state of their package in the |
| 122 |
|
VCS. The [PET](http://pet.alioth.debian.org/) tool could be adapted, |
| 123 |
|
integrated and made available to all teams by default. |
| 124 |
|
|
| 125 |
|
#### Replace WNPP's RFH, RFA, O |
| 126 |
|
|
| 127 |
|
Since the infrastructure already stores the information about who is |
| 128 |
|
maintaining what, it only makes sense to extend it to provide the |
| 129 |
|
list of orphaned packages (i.e. packages without maintainers). |
| 130 |
|
|
| 131 |
|
RFH should be replaced by a system where the help request is better |
| 132 |
|
formalized so that we can better direct new contributors in places |
| 133 |
|
where their skills would be well used. Instead of just requesting |
| 134 |
|
help, you could request: |
| 135 |
|
- a new maintainer (to replace you, i.e. RFA) |
| 136 |
|
- a supplementary co-maintainer |
| 137 |
|
- a bug triager |
| 138 |
|
- a C/Python/Perl/… programmer (you should be able to choose the programming |
| 139 |
|
language) |
| 140 |
|
|
| 141 |
|
Each request also documents whether there's an associated offer of |
| 142 |
|
"mentorship" associated to the help request. Of course, there would |
| 143 |
|
also be a free form description to give more details about what's |
| 144 |
|
expected. |
| 145 |
|
|
| 146 |
|
#### Replace the LowThresholdNmu list |
| 147 |
|
|
| 148 |
|
The [LowThresholdNmu](http://wiki.debian.org/LowThresholdNmu) wiki page |
| 149 |
|
is a hack to let people know when NMU are welcome and not frowned |
| 150 |
|
upon. This information should be properly stored in the database |
| 151 |
|
and it should be associated to each package. |
| 152 |
|
|
| 153 |
|
|
| 154 |
|
High-level design of the new infrastructure |
| 155 |
|
------------------------------------------- |
| 156 |
|
|
| 157 |
### Fixing the flow of information |
### Fixing the flow of information |
| 158 |
|
|
| 169 |
orphan their packages and are still listed as maintainers in many released |
orphan their packages and are still listed as maintainers in many released |
| 170 |
packages. |
packages. |
| 171 |
|
|
| 172 |
### Replacing maintenance mailing lists |
### Leveraging UDD |
|
|
|
|
Packaging teams often separate the mailing list that gets the bug traffic and |
|
|
other notifications from their main discussion mailing list. This new |
|
|
infrastructure should entirely replace the former kind of mailing lists. |
|
|
Anybody receiving notifications and information directed to the package |
|
|
maintainer should get them via this new infrastructure. |
|
|
|
|
|
It allows us to know how many people are notified for a given problem. |
|
|
If nobody is notified, the package is effectively orphaned. A more |
|
|
interesting case to detect is when several persons are being notified but |
|
|
all of them are MIA or marked as being busy/on vacation. |
|
| 173 |
|
|
| 174 |
|
At least the PTS has been parsing Sources/Packages files by itself, as |
| 175 |
|
well as a bunch of other source of information. But many of the most |
| 176 |
|
recent developments have piggy-backed on UDD to retrieve the information |
| 177 |
|
needed, leaving to UDD the responsibility of bringing all the information |
| 178 |
|
in a single place. |
| 179 |
|
|
| 180 |
|
This principle should be generalized to avoid duplication of work and to |
| 181 |
|
make sure that all the important information are available in UDD. |
| 182 |
|
|
| 183 |
|
But we must make sure that UDD won't become a bottleneck. Either because |
| 184 |
|
we have a local (live?) replicate of the database, or because we have |
| 185 |
|
ensured that our usage of UDD is limited to batch tasks that are not |
| 186 |
|
on the critical path for all the real-time user requests. |
| 187 |
|
|
| 188 |
|
### Using a modern framework for web development |
| 189 |
|
|
| 190 |
|
DDPO is implemented in PHP. The PTS uses a mix of Perl, Python, XSLT and |
| 191 |
|
shell scripts. While both works very well and are reliable, the diversity |
| 192 |
|
of the tools and the fact that some are not widely known (e.g. XSLT) |
| 193 |
|
seriously limit the set of contributors who are able to hack on all the |
| 194 |
|
parts of the infrastructure. |
| 195 |
|
|
| 196 |
|
With a modern framework for web development, we enlarge the set of people |
| 197 |
|
who are able to help us develop and maintain this infrastructure. It also |
| 198 |
|
offers us a proper separation between presentation and code, so that it's |
| 199 |
|
easier to let web designers integrate this service with the general |
| 200 |
|
look&feel of the various Debian websites. On top of this, we get a fully |
| 201 |
|
internationalized website for free. |
| 202 |
|
|
| 203 |
|
### API for data export |
| 204 |
|
|
| 205 |
|
If the infrastructure is going to have a central role, there will be |
| 206 |
|
requests to extract data out of the system. We should cater for this by |
| 207 |
|
providing a public API (over HTTP) allowing to retrieve all the (public) |
| 208 |
|
information in some standardized manner. |
| 209 |
|
|
| 210 |
|
JSON seems to be a good option for data export. It allows other services |
| 211 |
|
to reuse information from the DPMH, and it makes it easy for various |
| 212 |
|
web services to retrieve the information dynamically via Javascript. |
| 213 |
|
|
| 214 |
|
### Native support of packaging teams |
| 215 |
|
|
| 216 |
|
Any Debian Developer must be able to create a "packaging team" in the |
| 217 |
|
system. Each packaging team has a set of packages that it maintains (or |
| 218 |
|
keeps an eye on). Anyone can "subscribe" to the team and gets (by default) |
| 219 |
|
all correspondance of all packages associated to that team. |
| 220 |
|
|
| 221 |
|
The team subscription can be tuned (much like the current PTS subscription) |
| 222 |
|
to receive only a subset of the usual mails. A direct package subscription would |
| 223 |
|
take precedence over a team subscription, thus allowing the user to |
| 224 |
|
exclude some packages from its team subscription (or get more info for |
| 225 |
|
some specific packages where they are particularly interested). |
| 226 |
|
|
| 227 |
|
|
| 228 |
|
Implementation details |
| 229 |
|
---------------------- |
| 230 |
|
|
| 231 |
|
Questions: |
| 232 |
|
|
| 233 |
|
* How do we store emails? For how long? (we store all mails except the BTS mails) |
| 234 |
|
* What language and web framework? (buxy's default choice: Python & Django) |
| 235 |
|
* How do we authenticate users? And for DD super-powers? |
| 236 |
|
* [To be completed] |
| 237 |
|
|
| 238 |
Acronyms |
Acronyms |
| 239 |
-------- |
-------- |
| 240 |
|
|
| 241 |
|
* DPMH: Debian Package Maintenance Hub (this project) |
| 242 |
* DDPO: [Debian Developer's Packages Overview](http://qa.debian.org/developer.php) |
* DDPO: [Debian Developer's Packages Overview](http://qa.debian.org/developer.php) |
| 243 |
* PTS: [Package Tracking System](http://packages.qa.debian.org) |
* PTS: [Package Tracking System](http://packages.qa.debian.org) |
| 244 |
* PTS: [Bug Tracking System](http://bugs.debian.org) |
* BTS: [Bug Tracking System](http://bugs.debian.org) |
| 245 |
|
* UDD: [Ultimate Debian Database](http://udd.debian.org) |
| 246 |
|
* DD: Debian Developer |
| 247 |
|
* WNPP: [Work Needing and Prospective Packages](http://www.debian.org/devel/wnpp/) |
| 248 |
|
* RFH: Request For Help |
| 249 |
|
* RFA: Request For Adoption |
| 250 |
|
* O: Orphaned |
| 251 |
|
* NMU: Non-Maintainer Upload |
| 252 |
|
|
| 253 |
Changes |
Changes |
| 254 |
------- |
------- |
| 255 |
|
|
| 256 |
* 2011-01-13: Initial draft by Raphaël Hertzog. |
* 2011-01-13: Initial draft by Raphaël Hertzog. |
| 257 |
|
* 2011-01-28: Integrate feedback from debian-qa@lists.debian.org. |