Reading/writing a matrix from/to a grid file#3848
Merged
PaulWessel merged 9 commits intomasterfrom Aug 6, 2020
Merged
Conversation
Experimental. INitially it just read an ascii table but it would be convenient to recognize file extensions *.grd and *.nc plus the remote grids @earth*.
Member
|
This PR breaks a PyGMT test, but maybe we need to update PyGMT. The PyGMT test does following things:
Before this PR, it writes a table/matrix; after this PR, it writes a netCDF grid. Is this what we want? With this PR, to write a table data, PyGMT should use |
Member
Author
|
Yes, passing geometry as GMT_IS_POINT will case it to read or write an ascii table, while GMT_IS_SURFACE ends up calling import|export_grid. So this is backwards incompatible to some extent. |
Member
|
This is a good backwards incompatible to PyGMT, but I'm not sure if GMT.jl likes it. |
Member
Author
|
We need to check with @joa-quim to see if GMT.jl uses GMT_Read_Data directly on a text file to read in a GMT_IS_MATRIX. I don't think he is, but we will wait for confirmation. Depending on how long he is banned from checking email, this could be 10 days. Meanwhile, I will update the API docs. |
Member
|
Leave only tomorrow. I use but not |
Member
Author
|
OK, so it will not affect GMT.jl since GMT modules (other than a few testapi_*.c programs) never called GMT_Read_Data with intent of returning a matrix or sorts. So if @seisman is OK with this being merged then it might as well be approved now? |
seisman
approved these changes
Aug 6, 2020
Contributor
|
The backport to To backport manually, run these commands in your terminal: # Fetch latest updates from GitHub
git fetch
# Create a new working tree
git worktree add .worktrees/backport-6.1 6.1
# Navigate to the new working tree
cd .worktrees/backport-6.1
# Create a new branch
git switch --create backport-3848-to-6.1
# Cherry-pick the merged commit of this pull request and resolve the conflicts
git cherry-pick 02af19c043977c454d4fc1a29b947a0771c52974
# Push it to GitHub
git push --set-upstream origin backport-3848-to-6.1
# Go back to the original working tree
cd ../..
# Delete the working tree
git worktree remove .worktrees/backport-6.1Then, create a pull request where the |
seisman
pushed a commit
that referenced
this pull request
Aug 6, 2020
* Allow GMT_Read_Data to return a matrix from a grid file Experimental. INitially it just read an ascii table but it would be convenient to recognize file extensions *.grd and *.nc plus the remote grids @earth*. * Complete plan for read/write matrix via grids * Update testapi_grid2matrix.c * Fix the chunk issue * Create apimat2grd.ps * Update some test codes to read OPINTS not SURFACE if ascii, and update PS * Update api.rst
4 tasks
seisman
added a commit
that referenced
this pull request
Aug 6, 2020
* Reading/writing a matrix from/to a grid file (#3848) * Allow GMT_Read_Data to return a matrix from a grid file Experimental. INitially it just read an ascii table but it would be convenient to recognize file extensions *.grd and *.nc plus the remote grids @earth*. * Complete plan for read/write matrix via grids * Update testapi_grid2matrix.c * Fix the chunk issue * Create apimat2grd.ps * Update some test codes to read OPINTS not SURFACE if ascii, and update PS * Update api.rst * Fix two conflicts Co-authored-by: Paul Wessel <pwessel@hawaii.edu>
seisman
added a commit
to GenericMappingTools/pygmt
that referenced
this pull request
Aug 6, 2020
The test `test_put_matrix_grid` fails due to recent changes of GMT API in GenericMappingTools/gmt#3848. Before this PR, for a matrix grid, calling GMT_Write_Data always writes the matrix grid as a matrix, no matter the geometry is `GMT_IS_POINT` or `GMT_IS_SURFACE`. This PR declares it as a bug (or an improvement). With that PR merged, `geometry=GMT_IS_POINT` writes a matrix, `geometry=GMT_IS_SURFACE` writes a the matrix grid as a netCDF file. This test expects a matrix, but the new API writes a netCDF grid. That's why it fails. This PR fixes the issue by simply changing `GMT_IS_SURFACE` to `GMT_IS_POINT`, so that the test can pass for GMT 6.1.0, 6.1 and master branches.
5 tasks
seisman
added a commit
to GenericMappingTools/pygmt
that referenced
this pull request
Aug 7, 2020
The test `test_put_matrix_grid` fails due to recent changes of GMT API in GenericMappingTools/gmt#3848. Before this PR, for a matrix grid, calling GMT_Write_Data always writes the matrix grid as a matrix, no matter the geometry is `GMT_IS_POINT` or `GMT_IS_SURFACE`. This PR declares it as a bug (or an improvement). With that PR merged, `geometry=GMT_IS_POINT` writes a matrix, `geometry=GMT_IS_SURFACE` writes a the matrix grid as a netCDF file. This test expects a matrix, but the new API writes a netCDF grid. That's why it fails. This PR fixes the issue by simply changing `GMT_IS_SURFACE` to `GMT_IS_POINT`, so that the test can pass for GMT 6.1.0, 6.1 and master branches.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description of proposed changes
This PR started out with trying to generalize GMT_Read_Data for GMT_IS_MATRIX. Now, if you pass the geometry GMT_IS_SURFACE then we read the file as a grid but return a matrix. The conversion between grid to matrix is simple (we do this in the general import_grid anyway) but it pointed out some issues when testing via a short C code starting a session: Since no modules were involved, the remote files were unknown. So now we added a check to refresh the server if the list is NULL. The test codes now pass remote file names and GMT_IS_SURFACE and this works fine.
Then the same was applied to GMT_Write_Data, i.e., ability to write a MATRIX (camouflaging as a grid) to a grid file. This exposed another problem in gmt_nc.c in that the file-specific chunking settings were not reset, causing a second GMT_Write_Data to fail.
I have updated a few of the testapi*.c codes to indicate the geometry (GMT_IS_POINT if just a dumb text table and GMT_IS_SURFACE if a real grid), and updated some of the PS originals that changed due to the roundoff involved in the text version.
A new test program testapi_grid2matrix.c was added to test the reading of a grid into a matrix and then writing the matrix out as a grid, and the new test apimat2grd.sh runs this and makes a plot.
Because there are several bug fixes involved in this PR I have labeled this as a bug fix which requires backport.