Skip to content

Support for python env managers #1010

Merged
lfoppiano merged 19 commits intomasterfrom
python-envs
Jun 19, 2025
Merged

Support for python env managers #1010
lfoppiano merged 19 commits intomasterfrom
python-envs

Conversation

@lfoppiano
Copy link
Copy Markdown
Member

@lfoppiano lfoppiano commented May 6, 2023

This PR adds support for python virtual environments for JDK 11+. Should be merged after #921.
It supports venv and conda.

Update: I've noticed I'm not able to run grobid natively on linux without installing the python dependencies as a system dependencies. Usually I've correlate the use of this PR with the removal of the JEP library under the grobid-home, in favour of the JEP installed in the python library / virtual environment.

We could improve the detection of the python version.

@lfoppiano lfoppiano marked this pull request as draft May 6, 2023 09:40
@lfoppiano lfoppiano added this to the 0.7.3 milestone May 6, 2023
@kermitt2 kermitt2 modified the milestones: 0.7.3, 0.8.0 May 6, 2023
@coveralls
Copy link
Copy Markdown

coveralls commented Mar 12, 2024

Coverage Status

coverage: 40.46% (+0.002%) from 40.458%
when pulling c428014 on python-envs
into 8b9d113 on master.

@lfoppiano lfoppiano removed this from the 0.8.0 milestone Jun 9, 2024
@lfoppiano lfoppiano added this to the 0.8.2 milestone Sep 9, 2024
@lfoppiano lfoppiano marked this pull request as ready for review September 9, 2024 02:56
@lfoppiano
Copy link
Copy Markdown
Member Author

lfoppiano commented Nov 29, 2024

With this PR is possible to run a different version than 3.8 (the jep shipped with grobid requires python 3.8) and relay on conda for the environment.

Here what I did for running DL models natively using a conda environment:

  1. cd grobid
  2. Switch to this branch
  3. rm -rf grobid-home/lib/lin-64/jep
  4. cd ..
  5. git clone https://github.com/kermitt2/delft (delft should be in the parent directory)
  6. assuming conda is installed (if not, I installed this) and the environment delft has been created with either python=3.8 or python=3.9 (e.g. conda create --name delft python=3.8
  7. cd grobid
  8. pip install delft==0.3.3 (with deflt==0.3.4 you also need to install blingfire pip install blingfire
  9. conda install jep=4.1.1 (conda will install also openjdk 8)
  10. conda uninstall --force openjdk-8 - or conda uninstall --force openjdk (if you use miniconda3)
  11. conda install --force openjdk-17 or conda install --force openjdk=17 (if you use miniconda3)
  12. change any model in the grobid config to use delft instead of wapiti (e.g. header model)
  13. ./gradlew run

@lfoppiano lfoppiano modified the milestones: 0.8.2, 0.9.0 May 11, 2025
@lfoppiano lfoppiano merged commit e89768f into master Jun 19, 2025
10 checks passed
@lfoppiano lfoppiano deleted the python-envs branch June 19, 2025 09:40
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants