@thomasjpfan did you have this refactoring in mind?

Beyond the point of making the relationship between upstream code and derivative code explicit, I am fine with the idea of moving ARFF parsing code in a dedicated private module. So +1 once the above comment is addressed.

do we vendor code that was originally developed outside of scikit-learn (e.g. https://github.com/renatopp/liac-arff)? If so the functions that are derivatives from outside code should mention it in their docstrings or at the top of the module, including the original author name (or copyright holder name) and the license.

Yes, we do vendor in the sklearn.externals._arff.py. This file has all the info regarding the external package LIAC-ARFF.
Here, the change is just a wrapper that calls this ARFF parser with some additional work on the output to make it consistent with our fetchers.

do we plan to resync this code with upstream from time to time in the future?

Basically, the development in the original repository is not super active: https://github.com/renatopp/liac-arff
Indeed, it was people from OpenML and scikit-learn that did the latest maintenance. So in short, our current version in scikit-learn is synchronized.

Probably not because OpenML is moving towards parquet as a replacement for ARFF so I suspect that we won't have to deal with long term maintenance of the ARFF parser.

I open the following issue to ask a rough timeline regarding the landing of parquet format in OpenML: openml/OpenML#1133
Indeed, if parquet is landing all at once (for all datasets) then the ARFF parser would be obsolete.

Merging because the CI failure is unrelated to this PR.