@mhvk I am afraid that this may be not that simple: It is usually not a good idea to mix packages between distributions or even releases of the same distribution; often enough this ends in an unusable system.
One only reliable way would be to install f.e. Debian testing (or unstable) in a chroot and to use that. However, even then it may happen that one or the other new requirement is not available yet.
Another way would be to install the remaining packages via pip from source.

@olebole - hmm, maybe true, but since Ubuntu is just Debian with a bit of veneer, little of which is relevant for running code, it must be possible! And after about a dozen trials, indeed it is! (Though ideally I would not rely on unstable, but we need updated python3-pytest-doctestplus). And it already recovered some of the same errors you found (fix included).

To me, it seems this would be good enough to put in a cron job at least - let me ping @bsipocz

@bsipocz - obviously, the two commits with fixes can be pulled out of this PR! Though I do think there is little reason not to test on s390 as well.

@olebole - the other reason I like adding this is that it would be good to test a completely apt-based installation; it is no less valid an environment that one using pip or conda.

I've moved the milestone to 4.1 - no real hurry here.

There are some errors for installing optional dependencies, but otherwise it seems to work nicely, and the total job runtime of 8 minutes is not bad!

https://travis-ci.org/astropy/astropy/jobs/653658052#L966

Yes, I finally got it going again... Not having setup.py test was interesting - I was a bit disappointed that I seemed to need to create a virtual environment, but perhaps this is because of the extra packages that pip still installed (am trying to not let it install anything, so it also becomes a test of a non-pip install). Anyway, needs a bit more work, but at least getting there.

With @pllim excellent detective work, it seems that adding wheel as a dependency will probably solve those build issues, too. Yet, I'm not sure how not to install them, and also I have no idea yet how to make the build fail with those install issues, I would have assumed that those should not let the install and the test run.

The weird thing is wheel is specified at

so why is it missing?

Still has the bdist_wheel error, but also weird re.compile error from IPython.

Hurray, tests now pass on s390! Which brings up the question of what this should replace. In a way, by design it is somewhat orthogonal already to everything that is there, no only testing big-endian (main purpose) but also testing that standard debian (and therefore ubuntu) packages would do the trick.

But I do not want to increase the number of tests. Looking at the present set, we test all dependencies four times:

1. Initial tests - python 3.6 with all optional dependencies
2. Comprehensive tests - python 3.6 with all oldest optional dependencies
3. Comprehensive tests - python 3.7 with all optional dependencies
4. Final tests - windows python 3.7 with all optional dependencies

This seems too much and my suggestion would be to move all of these "one up" (deleting the first), i.e., have an initial test with all oldest supported versions (most likely to fail), comprensive tests on linux/windows, and final tests as they are.

Alternatively/additionally, I could move this new test to the cron section.

@bsipocz - what do you think? If it is easier, I can just move it to cron here and open a separate issue about reducing the number of tests generally.

OK, so I would do is to indeed remove the no1, and replace it with no3 but changing the python version to 3.8. I feel we have more breakage due to upstream updates rather than with old versions, so would keep the most up to date one in the initial set.

As to whether cron or not cron, I don't have strong preference. If we expect this new job to fail on ordinary PRs, then it should stay in the main matrix, but if breakage is infrequent (like for the osx one), then a cron is sufficient.

I don't think we need another issue for lowering the number of builds, we gone through it a few times as it's not really possible to squeeze them any more.

@bsipocz - decided it made more sense to just move the stage to cron for s390 - I do think it is unlikely to uncover problems with most PRs, so cron seems reasonable. It also means this PR remains nicely self-contained. I skip CI for this last commit, since the first one showed that tests passed.

Thanks @mhvk!

I'll try to backport this as a non cron job for the LTS release to help ensure we don't break anything there, too. I'm cherry-picking the commit and make the adjustments instead as part of #10059

Thanks, let me know if the backport proves troublesome - I'll keep #9670 around just to have a sense how the override used to look like before tox.