Skip to content

[MRG] Use ccache on Travis#9006

Merged
lesteve merged 3 commits intoscikit-learn:masterfrom
lesteve:use-ccache-on-travis
Jun 9, 2017
Merged

[MRG] Use ccache on Travis#9006
lesteve merged 3 commits intoscikit-learn:masterfrom
lesteve:use-ccache-on-travis

Conversation

@lesteve
Copy link
Copy Markdown
Member

@lesteve lesteve commented Jun 6, 2017

Use ccache and add .ccache to the Travis cache. This should save 2-4 minutes of compilation on each build that builds scikit-learn.

In some tests on Travis there seems to be some variability but here is what I found (I used time python setup.py develop to have a better idea about the timings):

  • when ccache is working python setup.py develop takes 45s-1min (cython compilation)
  • without ccache it is about 3-5 min (cython compilation + compilation of generated .c and .cpp files

I tried many different things but for some reason just using "export PATH=/usr/lib/ccache:$PATH" as I do locally does not seem to be enough ...

# Useful for debugging how ccache is used
# export CCACHE_LOGFILE=/tmp/ccache.log
# ~60M is used by .ccache when compiling from scratch at the time of writing
export CCACHE_MAXSIZE=100M
Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is to prevent the cache size to go out of hand. This should allow for a little bit less than twice the size of the cache needed.

@lesteve
Copy link
Copy Markdown
Member Author

lesteve commented Jun 6, 2017

I pushed an empty commit to test that ccache was working as it should.

@lesteve lesteve force-pushed the use-ccache-on-travis branch from 880fa31 to 8b2157d Compare June 7, 2017 09:26
# Useful for debugging how ccache is used
# export CCACHE_LOGFILE=/tmp/ccache.log
# ~60M is used by .ccache when compiling from scratch at the time of writing
ccache --max-size 100M --show-stats
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What happened to the CXX environmental variable?

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think it is needed. I certainly have come across SO answers that exactly say this, although nothing very authoritative.

side-comment: locally I don't need to set CC or CXX to have ccache working (you just add /usr/lib/ccache to PATH and that's it). I tried many different things to get it to work on Travis and that was one I found that was working.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

OK I googled again and although the situation is not very clear, I think it doesn't hurt to set CXX ...

although it does not seem to be necessary
@glemaitre
Copy link
Copy Markdown
Member

LGTM, it seems to work, nice.

@lesteve
Copy link
Copy Markdown
Member Author

lesteve commented Jun 9, 2017

I'll merge this one and promise to deal with the unexpected consequences if they arise.

@lesteve lesteve merged commit 84b36e0 into scikit-learn:master Jun 9, 2017
@lesteve lesteve deleted the use-ccache-on-travis branch June 9, 2017 09:22
Sundrique pushed a commit to Sundrique/scikit-learn that referenced this pull request Jun 14, 2017
dmohns pushed a commit to dmohns/scikit-learn that referenced this pull request Aug 7, 2017
dmohns pushed a commit to dmohns/scikit-learn that referenced this pull request Aug 7, 2017
NelleV pushed a commit to NelleV/scikit-learn that referenced this pull request Aug 11, 2017
paulha pushed a commit to paulha/scikit-learn that referenced this pull request Aug 19, 2017
AishwaryaRK pushed a commit to AishwaryaRK/scikit-learn that referenced this pull request Aug 29, 2017
maskani-moh pushed a commit to maskani-moh/scikit-learn that referenced this pull request Nov 15, 2017
jwjohnson314 pushed a commit to jwjohnson314/scikit-learn that referenced this pull request Dec 18, 2017
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants