My first concern with this approach is that most ROS users on macOS do install python via Homebrew, therefore using Homebrew to install would give a more realistic test case, also changes to Python in Homebrew will affect our users, so perhaps it should also affect the tests. That obviously not ideal since the tests could break at any time, but that's what our users typically have to deal with.

Now you could say that macOS users should install Python from pyenv instead as well, but that's a whole other can of worms which would need to be proceeded with a discussion with macOS users and culminate (if everyone agrees that it is the right way forward) in changing the installation instructions as well as working on the widely used scripts for installing on macOS, found here: https://github.com/mikepurvis/ros-install-osx

Putting that aside, I think that using pyenv in the tests is ok, but the version you've picked for Python3 is unrealistic (the Python2 may also be unrealistic). In stead, starting closer to the versions in Homebrew right now would be better. This is what brew says right now:

% brew info python
python: stable 3.6.4 (bottled), devel 3.7.0b2, HEAD
...

% brew info python@2
python@2: stable 2.7.14 (bottled), HEAD [keg-only]
...

I tried python from brew. It worked fine on my computer, but I had trouble with Travis CI infrastructure. The code on https://github.com/mikepurvis/ros-install-osx looks very promising and I see it passes in Travis CI. If I use it I am pretty sure I can succeed.

But IMO predictable environment is more important for CI. I'd rather see a build break/test failure from a change 'Switch to Python 3.42', rather than build break from a comment update, because brew has switched to a different version of python and now you puzzled that maybe in the end the compiler reads your comments.

In other words, as a developer, my main question is - "Is my change breaking any of existing environments?" If yes, I've got work to do. No - I am good to go. Good CI should be about this.

Version update will follow.

"Is my change breaking any of existing environments?" If yes, I've got work to do. Good CI should be about this.

In the case of macOS machines the 'existing environment' is a moving target. When employing CI for a project that has a tightly controlled target environment you're not wrong. The test evironment should be kept in sync with the production environment. But our production environment is not tightly controlled. Travis is a lightweight solution to a complex problem here. We want to test minimal changes but we also want to test resilience to environmental change. My knowledge of Travis is limited but I'm not aware of a way to separate those two concerns.

In general environmental issues can be found by running stuff that known to be good in a changing environment. Easy solution for this is regular runs.

Here is my proposal: I will add a new job that is going to build and test on the current brewed python, but the repo maintainers will need to add a scheduled build (daily?) through Travis. Here is the link on how to do it - https://docs.travis-ci.com/user/cron-jobs/.

Why not replace fixed version Python jobs on macOS? There are still good data points that will help distinguish between brewed python vs pure macOs issues.

Verification on python from brew added

Anything else I need to fix?

I don't think so, but can you comment on the changes to the debian tests? is that due to the merge with master or something you had to fix along the way? In general I prefer a rebase to a merge when master has changed.

I set up a daily cron job on travis to build the master branch. 👍

The tests were failing on mac, therefore they needed to be changed. Usual testing approach in this project is to supply a mock as exec_fn. And it works for one level of dependencies (in the failed tests for calling dpkg_detect()), but fails if the function calls anything else. In this case dpkg_detect() calls _read_apt_cache_showpkg() that is unconditionally executing apt-cache showpkg ... This command is failing on mac. The proposed solution is to mock the function that does actual execution, so it returns expected data. If interested I can go into details on why this approach was picked up, because there are some alternatives, but this looked like the best for me.

I am new to GitHub merge process, so any guidance is appreciated. Rebase vs merge - should it be happening at the moment when I do pull request? Or there was a conflict in .travis.yml that I resolved in GitHub UI - is it the problem? As I said, happy to follow the best process, I just don't know what to do :)

Ok, sounds good.

I don't suppose it matters about rebase versus merge to be honest. I'll squash it on merge anyways. Resolving conflicts with the web interface is fine too.

Thanks for the contribution!