Skip to content

Conversation

@connortann
Copy link
Contributor

@connortann connortann commented Apr 18, 2023

📚 Context

Adds support for pandas 2.0. Closes #6413

Pandas 2.0 has now been released for several weeks; however, streamlit's setup.py has a pin that prevents use with pandas 2. I believe the required code changes in streamlit to allow compatibility with pandas have already been implemented by #6378.

The remaining tasks seem to be to remove the prevous dependency constraint, and to ensure the test suite runs against the new version of pandas and passes.

  • Bugfix
  • Feature
  • Refactoring
  • Other, please describe:

🧠 Description of Changes

Adds support for pandas 2.0 by removing the pin in setup.py

🧪 Testing Done

  • Screenshots included
  • Added/Updated unit tests
  • Added/Updated e2e tests

As I understand, this change will mean the py_version (MAX) CI job will test against pandas 2.X, and the py_version (MIN) job will continue testing against 1.X.

However, in future perhaps it's worth expanding these tests to ensure that streamlit will always be tested against the different major versions of pandas, regardless of which python versions are supported? Or, more generally, expanding the CI to test streamlit against the lower bound versions of all dependencies? From a previous comment:

We are checking with our Python min tests for older Pandas versions since it's still running on Python 3.7, and Pandas 2.x will not be available for 3.7. But this is only a side-effect of us supporting a mostly outdated Python version. I think we should prioritize implementing a way for automated tests with older versions of our dependency. We probably could get a simple approach implemented without too much effort.

Originally posted by @lukasmasuch in #6378 (comment)

🌐 References

Does this depend on other work, documents, or tickets?


Contribution License Agreement

By submitting this pull request you agree that all contributions to this project are made under the Apache 2.0 license.

@connortann connortann changed the title WIP: Add support for pandas 2.0 Add support for pandas 2.0 Apr 18, 2023
@connortann
Copy link
Contributor Author

connortann commented Apr 19, 2023

Some seemingly unrelated tests now fail on CI with the new updated set of dependencies.

self = <tests.streamlit.web.server.server_test.ScriptCheckEndpointExistsTest testMethod=test_deprecated_endpoint>

    def get_app(self):
        server = Server("mock/script/path", "test command line")
        server._runtime.does_script_run_without_error = (
            self.does_script_run_without_error
        )
>       server._runtime._eventloop = self.asyncio_loop
E       AttributeError: 'ScriptCheckEndpointExistsTest' object has no attribute 'asyncio_loop'

The root cause seems to be that the tests in question do not work when tornado is upgraded from 6.2 to 6.3

From a quick look at the tornado docs, the asyncio_loop attribute of tornado's AsyncTestCase is not part of the public API, and although it's present it tornado 6.2 it has been removed in tornado 6.3.

This seems like a separate issue to the pandas dependency. Help fixing this would be welcome!

@lukasmasuch
Copy link
Collaborator

@connortann Thanks a lot for the PR and the info about the CI issue 👍 we will most likely merge this PR today. I also just merged a fix for the tornado issue into develop. I assume this will also fix the CI in this PR once you merge with the latest develop.

Copy link
Collaborator

@lukasmasuch lukasmasuch left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM 👍

@connortann connortann force-pushed the feature/support-pandas-2.0 branch from 4936fe4 to 0f6c5a9 Compare April 19, 2023 18:34
Co-authored-by: Lukas Masuch <Lukas.Masuch@gmail.com>
@lukasmasuch lukasmasuch added the security-assessment-completed Security assessment has been completed for PR label Apr 20, 2023
@lukasmasuch lukasmasuch merged commit 0e7b957 into streamlit:develop Apr 21, 2023
tconkling added a commit to tconkling/streamlit that referenced this pull request Apr 24, 2023
* develop:
  Removing viz-1.8.0.min.js (streamlit#6520)
  st.experimental_connection: The big merge (streamlit#6487)
  Add additional attributions (streamlit#6536)
  Fix code block font change (streamlit#6535)
  Fully remove email from new session message (streamlit#6516)
  Clarify what telemetry data our backend stores (streamlit#6463)
  Update emojis to latest state (streamlit#6532)
  Add support for pandas 2.0 (streamlit#6507)
  Fix regression in visibility of `st.code`'s copy-to-clipboard button (streamlit#6498)
  Fix E2E image (streamlit#6524)
  Add tenacity as dependency (streamlit#6529)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

security-assessment-completed Security assessment has been completed for PR

Projects

None yet

Development

Successfully merging this pull request may close these issues.

pandas 2.0 support

3 participants