Work around core_audiospec.t autopkgtest problems with newer Test-Simple

Newer versions of Test-Simple seem to lose the number of tests
issued in another thread.

Builds run without AUTOMATED_TESTING set, so they don't hit this.
Emulate that for autopkgtest as well for now.

Closes: #837145