I encountered a confusing error when submitting a Harmony request where harmony-py silently ignored a portion of my temporal definition, leading to a job failure on the backend.
I accidentally defined my temporal range using the key 'end' instead of 'stop':
time_range = {
'start': datetime(2019, 11, 1),
'end': datetime(2019, 12, 30)
}
Expected Behavior:
The harmony-py client should either:
- Accept
'end' as an alias for 'stop' (since it is a common intuitive keyword and as far as I understand very similar to the actual CMR key EndDate; see here), OR
- Raise a
ValidationError or warning when Request.is_valid() is called, alerting me that the key 'end' is unknown and will be ignored.
Actual Behavior:
Request.is_valid() returned True, erroneously confirming the request was correct.
harmony-py constructed a request URL with an open-ended time range (e.g., subset=time("2019-11-01T...":*)), effectively dropping the end date.
- The job was submitted successfully.
results_json = harmony_client.result_json(job_id, show_progress=True) then returns something like the following where it is just hinted at something being wrong "Job is running with errors":
[ Processing: 9% ] |#### | [|]
[ Processing: 9% ] |#### | [/]
[ Processing: 9% ] |#### | [-]
[ Processing: 9% ] |#### | [\]
[ Processing: 9% ] |#### | [|]
Job is running with errors.
[ Processing: 10% ] |##### | [/]
[ Processing: 10% ] |##### | [-]
[ Processing: 10% ] |##### | [\]
[ Processing: 10% ] |##### | [|]
Finally, harmony_client.download_all(job_id) is then just downloading 0 files and that's it. Only when looking at results_json it becomes clear what the actual problem was... specifically nasa/harmony-trajectory-subsetter failed with the error: Invalid temporal range, both start and end required for each file that was supposed to be processed and downloaded.
So this ended up being a confusing debugging experience that might be similar for other parameters / data products and should be evaluated in my opinion.
I encountered a confusing error when submitting a Harmony request where
harmony-pysilently ignored a portion of my temporal definition, leading to a job failure on the backend.I accidentally defined my temporal range using the key
'end'instead of'stop':Expected Behavior:
The
harmony-pyclient should either:'end'as an alias for'stop'(since it is a common intuitive keyword and as far as I understand very similar to the actual CMR keyEndDate; see here), ORValidationErroror warning whenRequest.is_valid()is called, alerting me that the key'end'is unknown and will be ignored.Actual Behavior:
Request.is_valid()returnedTrue, erroneously confirming the request was correct.harmony-pyconstructed a request URL with an open-ended time range (e.g.,subset=time("2019-11-01T...":*)), effectively dropping the end date.results_json = harmony_client.result_json(job_id, show_progress=True)then returns something like the following where it is just hinted at something being wrong "Job is running with errors":Finally,
harmony_client.download_all(job_id)is then just downloading 0 files and that's it. Only when looking atresults_jsonit becomes clear what the actual problem was... specificallynasa/harmony-trajectory-subsetterfailed with the error:Invalid temporal range, both start and end requiredfor each file that was supposed to be processed and downloaded.So this ended up being a confusing debugging experience that might be similar for other parameters / data products and should be evaluated in my opinion.