Hi,

one of our C++/Python libraries requires pillow. When we try to build pillow on Travis, we get a compilation error:

...
Building using 32 processes
...
Traceback (most recent call last):
  File "/opt/python/3.2.5/lib/python3.2/threading.py", line 740, in _bootstrap_inner
    self.run()
  File "/opt/python/3.2.5/lib/python3.2/threading.py", line 693, in run
    self._target(*self._args, **self._kwargs)
  File "/opt/python/3.2.5/lib/python3.2/multiprocessing/pool.py", line 357, in _handle_tasks
    debug('task handler got sentinel')
TypeError: 'NoneType' object is not callable

see:
https://travis-ci.org/bioidiap/bob.io.video/jobs/35247966

I am pretty sure that this is related to the parallel build, which is enabled by default using all possible processors (see Pillow/mp_compile.py), which on Travis seems to be 32. I guess that the parallel compilation does not like it, when there are more threads than files to be compiled.

My problem with your script is that it affects both your compilation and ours, which happens after yours. Therefore, I don't think it is a good strategy to enable the parallel build when the MAX_CONCURRENCY flag is not set. I think it would be smarter to enable parallel builds only when MAX_CONCURRENCY is set (e.g. to -1 to get cpu_count() threads), to avoid conflicts with other packages.

Note that we have implemented a similar solution for parallel building, which does not seem to have the problem when less files than threads are available. Maybe you want to have a look at the "parallelCompile" function on line 669 of:
https://github.com/bioidiap/bob.extension/blob/master/bob/extension/__init__.py#L667-686

For the time being, I found the solution to set MAX_CONCURRENCY=1 before building.

Cheers
Manuel