What happened?

On Python 3.10 it is not possible to install apache-beam==2.43.0 together with multiprocess. This is due to Python 3.10 only being supported by multiprocess>=0.70.12 which requires dill>=0.3.4 and is in conflict with the apache-beam requirement for dill>=0.3.1.1,<0.3.2.

These libraries are used together, for example, in the datasets library.

Is there a specific reason for the apache-beam version requirement of the dill package? If not, maybe this could be updated to fix the issue for Python 3.10? Although I have not tested it, the same issue might apply to Python 3.9 which is only supported by multiprocess>=0.70.11 requiring dill>=0.3.3.

Issue Priority

Priority: 2

Issue Component

Component: dependencies