-
Notifications
You must be signed in to change notification settings - Fork 124
Integration of new PFB IO functionality and xarray compatibility #365
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Integration of new PFB IO functionality and xarray compatibility #365
Conversation
…parflow into feature/pf_xarray_integration
smithsg84
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Minor typo and would like to get rid of the hard-coded temp directory path.
| header['p'] = '8' | ||
| header['q'] = '5' | ||
| header['z'] = '1' | ||
| if not os.path.exists(TEMP_DIRECTORY): |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The TEMP_DIRECTORY should be replaced with generated value.
…parflow into feature/pf_xarray_integration
|
I noticed a few of the tests failed - I added some fixes to 2 of them, |
smithsg84
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks for all the patching.
|
Add simple and fast pure-python based readers and writers of PFB files, done by myself and Bill Hasling (@wh3248). This eliminates the need for the external ParflowIO dependency. Implemented a new backend for the xarray package that let's you open both .pfb files as well as .pfmetadata files directly into xarray datastructures. These are very useful for data wrangling and scientific analysis Basic usage of the new functionality: . |
👋 Hi Parflow maintainers! This is my first PR here - I'm Andrew Bennett, working with Laura Condon (@lecondon)& Reed Maxwell (@reedmaxwell) to develop some machine learning emulators of Parflow. In preparing the large datasets to feed the ML algorithms it became a bottleneck to read using the ParflowIO implementation of readers. This pull request implements simple and fast pure-python based readers and writers of PFB files, done by myself and Bill Hasling (@wh3248). This eliminates the need for the external ParflowIO dependency. I have also implemented a new backend for the xarray package that let's you open both
.pfbfiles as well as.pfmetadatafiles directly into xarray datastructures. These are very useful for data wrangling and scientific analysis. This PR is a merge from my separate repo pf-xarray.I recognize this is a rather large pull request so I'm happy to meet to discuss the finer points of the implementation. Currently there are some tests, but they are minimal and might be expanded to make sure things interoperate with the rest of the parflow ecosystem. Also, I'm not sure if I should be PRing against
master- but I didn't see a staging branch likedevelopornext. Let me know if this should be changed!Some basic usage of the new functionality: