Skip to content

Harmony 1929 - Ensure OPeNDAP downloads use POST#97

Merged
vinnyinverso merged 5 commits into
mainfrom
HARMONY-1929
Nov 22, 2024
Merged

Harmony 1929 - Ensure OPeNDAP downloads use POST#97
vinnyinverso merged 5 commits into
mainfrom
HARMONY-1929

Conversation

@vinnyinverso

@vinnyinverso vinnyinverso commented Nov 21, 2024

Copy link
Copy Markdown
Contributor

Jira Issue ID

HARMONY-1929

Description

Since opendap can have potentially very long data download URLs we need to ensure that we convert the query string to POST body parameters in those cases and submit a POST request.

NOTE: I did see OPeNDAP timing out for very long URLs.

The snyk check is failing due to an internal snyk error.

Local Test Steps

This is how I tested:

  1. With a local harmony instance running, submit a request.
  2. Once it finishes, change one of the job link data URLs in the job links table to:
    https://opendap.uat.earthdata.nasa.gov/collections/C1261703111-EEDTEST/granules/SC:ATL03.006:264549068.dap.nc4?dap4.ce=/ds_surf_type[0:1:4]
  3. Copy the job ID for this job as you will use it to make a download request with harmony-py.
  4. Create a virtual environment with this version of harmony-py installed (pip install -e ./path/to/harmony_py).
  5. In a notebook (see basic.ipynb), create a client with env=Environment.LOCAL and make a download request (e.g. harmony_client.download_all('88aebf3b-bb47-475d-8d54-05fa9812ae77', overwrite=True)).
  6. Verify that the opendap data is successfully retrieved.

PR Acceptance Checklist

  • Acceptance criteria met
  • Tests added/updated (if needed) and passing
  • Documentation updated (if needed)

@vinnyinverso vinnyinverso marked this pull request as ready for review November 22, 2024 13:47

@indiejames indiejames left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Tested locally

@vinnyinverso vinnyinverso merged commit 26f014e into main Nov 22, 2024
@vinnyinverso vinnyinverso deleted the HARMONY-1929 branch November 22, 2024 19:08
@vinnyinverso vinnyinverso restored the HARMONY-1929 branch November 26, 2024 20:52
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants