Skip to content

DockerfileLeanFoundation package updates and new additions #5072

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
Martin-Molinero merged 2 commits into from
Jan 29, 2021
Merged

DockerfileLeanFoundation package updates and new additions #5072

Martin-Molinero merged 2 commits into from
Jan 29, 2021

Conversation

@gsalaz98
Copy link
Contributor

@gsalaz98 gsalaz98 commented Dec 23, 2020

Description

This PR is on top of #5036

Package updates and new installs included in this PR:
seaborn (Closes #2568)
pyflux (Closes #4499)
optuna (Closes #4528)
sklearn (Updated, closes #4587)
ntlk.download('stopwords') (Closes #4588)
findiff (Closes #4624)
sktime (Closes #4627)
sktime-dl (Closes #4627)
hyperopt (Closes #4629)
bayesian-optimization (Closes #4651)
rpy2 (Closes #4746)
pingouin (Closes #4782)
quantecon (Closes #4813)
matplotlib (Updated)
torch-scatter, torch-sparse, torch-cluster, torch-spline-conv, torch-geometric (Partial of #4857)

Packages that are pending addition or are unable to be added at the moment:
pysde (TODO, unable to be installed)
pandas_market_calendars (TODO, review dependencies)
feature-selector (TODO, has strict package requirements)
tensorforce (TODO, requires tensorflow2)
tensorflow2 (TODO)
sagemaker (TODO)
nbeats (TODO, requires tensorflow2)
dgl (TODO, requires custom build w/ build-essentials)
stellargraph (TODO)
ruptures (TODO)
simpy (TODO)
coach (TODO, requires apt-get installs)
stable-baselines3 (TODO, might be incompatible)
modin (TODO)

Related Issue

Issues posted in description

Motivation and Context

Adds user requested packages

Requires Documentation Change

Yes, update package list on QC docs website

How Has This Been Tested?

Ran all tests, all passing

Might require some new package version tests

Types of changes

  • Bug fix (non-breaking change which fixes an issue)
  • Refactor (non-breaking change which improves implementation)
  • Performance (non-breaking change which improves performance. Please add associated performance test and results)
  • New feature (non-breaking change which adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to change)
  • Non-functional change (xml comments/documentation/etc)

Checklist:

  • My code follows the code style of this project.
  • I have read the CONTRIBUTING document.
  • I have added tests to cover my changes.
  • All new and existing tests passed.
  • My branch follows the naming convention bug-<issue#>-<description> or feature-<issue#>-<description>

@hohoCode
Copy link

hohoCode commented Dec 23, 2020 •
edited
Loading

Could you please update the library: mlfinlab from 0.9.1 in lean foundation to the latest version 1.5.1? Thanks.

@gsalaz98 gsalaz98 force-pushed the refactor-dockerfileleanfoundation-python-pkg-updates-2 branch from 8aa035b to e33552f Compare December 30, 2020 17:58
@Martin-Molinero
Copy link
Member

Martin-Molinero commented Dec 31, 2020

Let's test live/backtesting/research with refactor-dockerfileleanfoundation-python-pkg-updates-2-e33552f-37e65eb 10225

@Martin-Molinero Martin-Molinero force-pushed the refactor-dockerfileleanfoundation-python-pkg-updates-2 branch from e33552f to 390953e Compare January 25, 2021 13:31
@gsalaz98
Copy link
Contributor Author

gsalaz98 commented Jan 26, 2021 •
edited
Loading

Package imports:

    torch-scatter==2.0.5            \
    torch-sparse==0.6.8             \
    torch-cluster==1.5.8            \
    torch-spline-conv==1.2.0        \
    torch-geometric==1.6.3

are failing with the following error:

RuntimeError: Detected that PyTorch and torch_scatter were compiled with different CUDA versions. PyTorch has CUDA version 10.2 and torch_scatter has CUDA version 0.0. Please reinstall the torch_<pkg> that matches your PyTorch install.

Most likely can be solved w/ Conda install rather than relying on pip

@gsalaz98 @Martin-Molinero
Updates and adds new Python packages to DockerfileLeanFoundation
6ec66bc
@gsalaz98 @Martin-Molinero
Remove torch-geometric packages due to import issues
702b111 
Torch geometric packages installed from pip as they are here result in an error saying to install the CUDA version of the packages. When the CUDA version is installed, we get a symbol error when attempting to link the `_version.so` shared object.
@Martin-Molinero Martin-Molinero force-pushed the refactor-dockerfileleanfoundation-python-pkg-updates-2 branch from 1f4be58 to 702b111 Compare January 28, 2021 15:04
@Martin-Molinero
Copy link
Member

Martin-Molinero commented Jan 28, 2021

Rebuilt at refactor-dockerfileleanfoundation-python-pkg-updates-2-702b111-a7d32a9 10514

@gsalaz98
Copy link
Contributor Author

gsalaz98 commented Jan 28, 2021 •
edited
Loading

Tested cloud -- backtesting (Py/C#), backtesting debugging (Py), research (Py/C#), and live (Py/C# QC Paper and IB) - all successful

@Martin-Molinero Martin-Molinero merged commit 6cb103b into master Jan 29, 2021
@Martin-Molinero Martin-Molinero deleted the refactor-dockerfileleanfoundation-python-pkg-updates-2 branch January 29, 2021 21:26
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@Martin-Molinero Martin-Molinero Martin-Molinero approved these changes

@AlexCatarino AlexCatarino Awaiting requested review from AlexCatarino

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

4 participants

@gsalaz98 @hohoCode @Martin-Molinero