Skip to content

Conversation

@ocefpaf
Copy link
Member

@ocefpaf ocefpaf commented Mar 6, 2025

This server is still on ERDDAP 2.18 and has a non-standard date/time variable. The format change, but we are also experiencing erros when the variable and constraints are in a different order:

base = "https://opendap.co-ops.nos.noaa.gov/erddap/tabledap/IOOS_Hourly_Height_Verified_Water_Level.csv?"
works =  'WL_VALUE,time&DATUM="MSL"&BEGIN_DATE="20161004"&END_DATE="20161012"&STATION_ID="8651370"'
do_not = 'WL_VALUE,time&BEGIN_DATE="20161004"&END_DATE="20161012"&DATUM="MSL"&STATION_ID="8651370"'

Those two should be the same in ERDDAP, but here one works and the other fails with:

Error {
    code=500;
    message="Internal Server Error: ERROR from data source: org.postgresql.util.PSQLException: ERROR: Invalid time requestesd.
  Where: PL/pgSQL function inline_code_block line 1 at RAISE";
}

@ocefpaf ocefpaf mentioned this pull request Mar 6, 2025
@MathewBiddle
Copy link
Contributor

FYI @LindsayRAbrams. Curious if you know who manages https://opendap.co-ops.nos.noaa.gov/erddap/index.html?

@mwengren
Copy link
Member

The CO-OPS ERDDAP is very non-standard in that it requires the extra STATION_ID and possibly other parameters to return results.

It's been awhile since I've looked into it, but I recall it's a design limitation within the CO-OPS database that requires such parameters by design in order to return data. Maybe they have an alternate data source at this point that could replace this that ERDDAP could run from. Although looking at the error message you included @ocefpaf, it looks like it's PostgreSQL now, whereas last I looked it was still running Sybase, so maybe they've updated but the underlying parameter requirement is unchanged.

I got the 504 timeout error when trying to access the ERDDAP homepage link above just now. So it may be that it's just not operationally supported at the moment too.

@ocefpaf
Copy link
Member Author

ocefpaf commented Apr 14, 2025

The CO-OPS ERDDAP is very non-standard in that it requires the extra STATION_ID and possibly other parameters to return results.

Indeed. Not only that, they must be double quoted!

Although looking at the error message you included @ocefpaf, it looks like it's PostgreSQL now, whereas last I looked it was still running Sybase, so maybe they've updated but the underlying parameter requirement is unchanged.

I'm not sure if that is a red herring or a real error. Updating the date format fixes it, but there is still the matter of variable order in the URL that should not play a role in the query. (I may be wrong here. I haven't seem any ERDDAP server that requires variables to be in a fixed order.) See https://gist.github.com/ocefpaf/bb0fd4551d5c8ce91f4b778c8cc54ec3 for an example.

@ocefpaf ocefpaf force-pushed the fix_2016-10-12-fetching_data branch from 84f659a to 5f96432 Compare April 14, 2025 17:26
@ocefpaf
Copy link
Member Author

ocefpaf commented Apr 14, 2025

Needs ioos/erddapy#399

@ocefpaf
Copy link
Member Author

ocefpaf commented Apr 15, 2025

The jupyterbook/content/code_gallery/data_management_notebooks/2023-03-20-Reading_and_writing_zarr.ipynb is failing but the co-ops one is finally OK again. Let's merge this one and I'll fix the zarr syntax in another PR.

@ocefpaf ocefpaf merged commit 8e04f8e into ioos:main Apr 15, 2025
10 of 13 checks passed
@ocefpaf ocefpaf deleted the fix_2016-10-12-fetching_data branch April 15, 2025 10:03
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants