Otherwise we'll catch binaries from unrelated sources. This got introduced in commit 7c1a251a, so revert that, but add a fallback for Ubuntu 12.04 to re-try without --only-source.