Skip to content

Request.is_valid() passes with invalid temporal key "end", causing silent failure and backend errors #123

@maawoo

Description

@maawoo

I encountered a confusing error when submitting a Harmony request where harmony-py silently ignored a portion of my temporal definition, leading to a job failure on the backend.

I accidentally defined my temporal range using the key 'end' instead of 'stop':

time_range = {
    'start': datetime(2019, 11, 1),
    'end': datetime(2019, 12, 30) 
}

Expected Behavior:
The harmony-py client should either:

  1. Accept 'end' as an alias for 'stop' (since it is a common intuitive keyword and as far as I understand very similar to the actual CMR key EndDate; see here), OR
  2. Raise a ValidationError or warning when Request.is_valid() is called, alerting me that the key 'end' is unknown and will be ignored.

Actual Behavior:

  1. Request.is_valid() returned True, erroneously confirming the request was correct.
  2. harmony-py constructed a request URL with an open-ended time range (e.g., subset=time("2019-11-01T...":*)), effectively dropping the end date.
  3. The job was submitted successfully. results_json = harmony_client.result_json(job_id, show_progress=True) then returns something like the following where it is just hinted at something being wrong "Job is running with errors":
 [ Processing:   9% ] |####                                               | [|]
 [ Processing:   9% ] |####                                               | [/]
 [ Processing:   9% ] |####                                               | [-]
 [ Processing:   9% ] |####                                               | [\]
 [ Processing:   9% ] |####                                               | [|]

Job is running with errors.
 [ Processing:  10% ] |#####                                              | [/]
 [ Processing:  10% ] |#####                                              | [-]
 [ Processing:  10% ] |#####                                              | [\]
 [ Processing:  10% ] |#####                                              | [|]

Finally, harmony_client.download_all(job_id) is then just downloading 0 files and that's it. Only when looking at results_json it becomes clear what the actual problem was... specifically nasa/harmony-trajectory-subsetter failed with the error: Invalid temporal range, both start and end required for each file that was supposed to be processed and downloaded.

So this ended up being a confusing debugging experience that might be similar for other parameters / data products and should be evaluated in my opinion.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions