With my geodetic authority wishful thinking hat on: A broken connection between EPSG:4258 and ETRS89-XXX is a convienient "happy accident" that could be very helpful in communicating The New World Order of individually named realisations of ETRS89. More or less everyone would be better off using their local ETRS89-XXX but communicating that as well as actually making the transition is almost impossible without a good conversation starter (realistically I think the real conversation starter will be ETRS2YYY once we inevitably get it).

Wearing all sorts of other hats: This looks like a pragmatic solution that'll keep things running seamlessly.

And with the PROJ hat on I am wondering if

Until EPSG perhaps figures out a plan to restore cleanly backward
compatibility (they have been made aware of those issues)

is realistically possible? In other words, if EPSG patches things in their end will the fix in this PR be redundant?

With my geodetic authority wishful thinking hat on:

I agree that for a geodesist, things aren't broken. But for a GIS user (most of our users in term of number), they are. And there's no practical workaround we can offer. For example there's no projected CRS based on 'ETRS89-AUT [2002]'.

In other words, if EPSG patches things in their end will the fix in this PR be redundant?

Hard to tell before seeing how/if they address that. But the timing part isn't good for us. They will discuss that issue at a meeting end of May, among other "small" topics like NATRF2020, so after our 9.8.1 release.

But for a GIS user (most of our users in term of number), they are [broken]

Absolutely and we should try to unfuck the situation as best as possible, which luckily is what this PR does. But I fear that's not feasible in the long term, so I hope the EPSG guys come up with a solid plan. At the moment they're testing in prod.

Hard to tell before seeing how/if they address that. But the timing part isn't good for us. They will discuss that issue at a meeting end of May, among other "small" topics like NATRF2020, so after our 9.8.1 release.

Yeah, that's not great timing. Ideally this issue is fixed upstream but doesn't seem likely to have that sorted in time. More likely to have that land in 9.9.0.

They are expecting it uses the grid file, but it doesn't. I think this PR does not cover that problem, right?

nope, that one is trickier indeed. EPSG:3812 which used to be "ETRS89 / Belgian Lambert 2008" was modified to be 'ETRS89-BEL [BEREF2002] / Belgian Lambert 2008' . And they created EPSG:11219 which is 'ETRS89-BEL [BEREF2011] / Belgian Lambert 2008' . The bd72lb72_etrs89lb08 grid is used when using EPSG:11219, and not EPSG:3812 . Which is technically the correct behavior here.
I'd see this as a bug of EPSG. My understanding was that they were not supposed to modify existing records, at least in a significant way. Changing comments fine, but changing datum of well established CRS is a big red flag to me...

The "workaround" for users is to switch to EPSG:11219 ... I'm wondering how much national geodetic authorities have been involved in that process...

I'm wondering how much national geodetic authorities have been involved in that process...

Very little, if at all I'm afraid. At least that's the case in my end. My memory failed me, I have indeed been in contact with EPSG regarding this.

@OSGeo/proj-committers @jjimenezshaw How do we want to handle the semi-broken status of 9.8.0 ? Just marking it as "retracted" in some documentation would not be enough for all automated packaging systems that look at new github releases and automatically publish a new packaged version. I believe we need a 9.8.1. What should that 9.8.1 be:

• 9.7.1 retagged as 9.8.1 ? But there have have been API additions in 9.8.0 that if reverted would prevent us from claiming a 9.8 lineage
• 9.7.1 + all 9.7.1 to 9.8 changes , that are not database related, re-applied. Not totally sure how doable this is.
• 9.8 branch with that PR applied. But at least the Belgium issue isn't yet covered. We need to wait a bit EPSG reaction to that, but we can't differ too long having 9.8.0 as our last version.

I agree that going backwards in a revision is quite estrange. So we need 9.8.1.
As you say, if including all the changes excluding the EPSG updates (and other db related changes if any) is possible to be done, that would be probably the better. But I am also not sure if it is possible. I remember that the first changes regarding ETRS89 were done in master in the period between 9.7.0 and 9.7.1. Maybe they were just adding some new systems, that wouldn't be a problem (I hope). In any case, you didn't add those changes to 9.7.1, waiting for 9.8.0.

I would wait at least this week to see the reaction from EPSG.

• 9.7.1 + all 9.7.1 to 9.8 changes , that are not database related, re-applied. Not totally sure how doable this is.

A 4th option, equivalent in result to the above one but easier to do, would be to start from the current state of 9.8, and re-import the version of the EPSG database before the ETRS89-XXX changes (or at the least the problematic ones. No one has complained about 9.7.1, so that's probably a safe baseline for the database version). The main impact would be revert some changes in the test suite.

It sounds to me like the best approach is to

1. Await a response from EPSG. If there's no actionable answer within a week then:
2. Issue 9.8.1 with a downgraded EPSG-database [0], aiming for a release date around April 15.

I don't expect the EPSG to do magic in a short time frame so I assume 2 is inevitable.

[0] The version used in 9.7.1 is a good starting point, it should be researched if we can get away with a more recent one.

I recommend the first line of the release notes for PROJ be a warning about this here:

https://github.com/OSGeo/PROJ/releases/tag/9.8.0

I would then suggest going for a 9.8.1 downgrade of the catalog at the soonest possible date as mentioned here, but without waiting on the EPSG folks.

Issue 9.8.1 with a downgraded EPSG-database [0], aiming for a release date around April 15.

This is all in the "no fun" category :(

I recommend the first line of the release notes for PROJ be a warning about this here:

That's a good idea, Kurt.

@rouault, @jjimenezshaw the issues you've encountered generally haven't been reported here at the PROJ issue tracker. In this issue is mentioned problems with ETRS89 transformations in Austria, the Netherlands and Romania. There's email communication about problems in Belgium too. Is this the full (known) picture or are there more known cases? It would be good to be able to clearly state in warnings and release notes in which cases you need to pay extra attention.

I haven't heard of any issues involving ETRS89-DNK but I'll have to do some testing to rule it out at this point.

Some tests I've done comparing 9.8.0 with 9.7.1 below. I should stress this should be done as a "random sampling" experiment. There are far more potential combinations than the one I've tested. For Italy the testing is definitely very partial due to the lack of public availability of the grids (little poke at the eyes of our friends at IGM ;-)) and creation of 2 datums ETRS89-ITA [IGM95] and ETRS89-ITA [RDN2008] . And I haven't tested vertical transformations

KO

• Catalonia ETRS89 <--> ED50 : KO

  • with 9.8 (uses es_ign_SPED2ETV2.tif)
    echo 41.4 2.17 | PROJ_NETWORK=ON PROJ_DEBUG=2 PROJ_DATA=data bin/cs2cs "ED50" "ETRS89"
    41d23'55.955"N 2d10'7.878"E 0.000

  • with 9.7.1 (uses es_cat_icgc_100800401.tif)
    echo 41.4 2.17 | PROJ_NETWORK=ON PROJ_DEBUG=2 PROJ_DATA=data bin/cs2cs "ED50" "ETRS89"
    41d23'55.951"N 2d10'7.863"E 0.000

• Serbia ETRS89 <--> MGI 1901: KO (due to creation of 2 distinct datums ETRS89-SRB [STRS00] and ETRS89-SRB [SREF98])

  • with 9.8 (uses ballpark noop)
    echo 44.5 20.9 | PROJ_NETWORK=ON PROJ_DEBUG=2 PROJ_DATA=data bin/cs2cs ETRS89 'MGI 1901'
    44d30'N 20d54'E 0.000

  • with 9.7.1 (using 'MGI 1901 to ETRS89 (6)')
    echo 44.5 20.9 | PROJ_NETWORK=ON PROJ_DEBUG=2 PROJ_DATA=data bin/cs2cs ETRS89 'MGI 1901'
    44d29'59.771"N 20d54'19.548"E 0.000

Different result, but likely fine?

• Sweden ETRS89 <--> RT90: KO (not really, but slightly different result)

  • with 9.8 (uses RT90 to ETRS89-SWE [SWEREF 99] (1) + Inverse of ETRS89 to ETRS89-SWE [SWEREF 99])
    echo 59.4 18.1 | PROJ_NETWORK=ON PROJ_DEBUG=2 PROJ_DATA=data bin/cs2cs SWEREF99 ETRS89
    59d23'58.209"N 18d5'47.561"E 0.000

  • with 9.7.1 (using 'RT90 to ETRS89 (1)')
    echo 59.4 18.1 | PROJ_NETWORK=ON PROJ_DEBUG=2 PROJ_DATA=data bin/cs2cs RT90 ETRS89
    59d23'58.209"N 18d5'47.56"E 0.000

  but that might be a false positive given use of SWEREF99 is likely well established, and if so RT90 to SWEREF99 (1) is used in both cases (logically due to ETRS89 not being directly involved) :

  echo 59.4 18.1 | PROJ_NETWORK=ON PROJ_DEBUG=2 PROJ_DATA=data bin/cs2cs RT90 SWEREF99
  59d23'58.209"N 18d5'47.561"E 0.000

OK

• Spain ETRS89 <--> ED50 : OK (using es_ign_SPED2ETV2.tif)
  echo 40 0 | PROJ_NETWORK=ON PROJ_DEBUG=2 PROJ_DATA=data bin/cs2cs "ED50" "ETRS89"
  39d59'55.79"N 0d0'4.339"W 0.000

• Portugal ETRS89 <--> Datum 73 : OK (using pt_dgt_D73_ETRS89_geo.tif)
  echo 38.7 -9.1 | PROJ_NETWORK=ON PROJ_DEBUG=2 PROJ_DATA=data bin/cs2cs ETRS89 "Datum 73"
  38d41'57.148"N 9d6'3.059"W 0.000

• Italy ETRS89 <--> Monte Mario: OK (using Monte Mario to ETRS89 (1))
  echo 41.89 12.48 | PROJ_NETWORK=ON PROJ_DEBUG=2 PROJ_DATA=data bin/cs2cs ETRS89 "Monte Mario"
  41d53'21.684"N 12d28'48.673"E 0.000

• France ETRS89 <--> NTF: OK
  echo 49 2 | PROJ_NETWORK=ON PROJ_DEBUG=2 PROJ_DATA=data bin/cs2cs NTF ETRS89
  48d59'59.756"N 1d59'57.4"E 0.000

• France NTF <--> RGF93v2b: OK
  echo 49 2 | PROJ_NETWORK=ON PROJ_DEBUG=2 PROJ_DATA=data bin/cs2cs "RGF93 v2b" NTF
  49d0'0.244"N 2d0'2.6"E 0.000

• Germany ETRS89 <--> DHDN: OK (using de_hvbg_hessen_HeTA2010.tif)
  echo 50.1 8.6 | PROJ_NETWORK=ON PROJ_DEBUG=2 PROJ_DATA=data bin/cs2cs ETRS89 DHDN
  50d6'4.179"N 8d36'3.561"E 0.000

• Germany ETRS89 <--> Pulkovo 1942(83): OK (using 'Pulkovo 1942(83) to ETRS89 (2)')
  echo 52.6 13.3 | PROJ_NETWORK=ON PROJ_DEBUG=2 PROJ_DATA=data bin/cs2cs ETRS89 "Pulkovo 1942(83)"
  52d36'1.397"N 13d18'6.728"E 0.000

• Ireland ETRS89 <--> TM75: OK
  echo 53.2 -7.9 | PROJ_NETWORK=ON PROJ_DEBUG=2 PROJ_DATA=data bin/cs2cs ETRS89 TM75
  53d11'59.102"N 7d53'57.304"W 0.000

• UK ETRS89<--> OSGB36: OK (using uk_os_OSTN15_NTv2_OSGBtoETRS.tif)
  echo 51.7 -0.2 | PROJ_NETWORK=ON PROJ_DEBUG=2 PROJ_DATA=data bin/cs2cs OSGB36 ETRS89
  51d42'1.767"N 0d12'5.697"W 0.000

• Slovenia ETRS89 <--> MGI 1901: OK (using 'MGI 1901 to ETRS89 (3)')
  echo 46.1 14.5 | PROJ_NETWORK=ON PROJ_DEBUG=2 PROJ_DATA=data bin/cs2cs ETRS89 'MGI 1901'
  46d6'1.121"N 14d30'17.036"E 0.000

• Croatia HTRS96 <--> MGI 1901 using ETRS89 intermediate: OK (using MGI 1901 to ETRS89 (2) and HTRS96 to ETRS89 (1))
  echo 45.6 16.2 | PROJ_NETWORK=ON PROJ_DEBUG=2 PROJ_DATA=data bin/cs2cs 'MGI 1901' EPSG:3765
  476212.82 5051143.89 0.00

• Poland Pulkovo 1942(58) <--> ETRF2000-PL using ETRS89 intermediate
  echo 53.1 18.8 | PROJ_NETWORK=ON PROJ_DEBUG=2 PROJ_DATA=data bin/cs2cs "Pulkovo 1942(58)" EPSG:2177
  5885560.92 6553456.32 0.00

• Czechia ETRS89 <--> S-JTSK : OK (using cz_cuzk_table_-y-x_3_v1710)
  echo 50 14.5 | PROJ_NETWORK=ON PROJ_DATA=data bin/cs2cs ETRS89 "S-JTSK / Krovak East North"
  -738542.81 -1053416.58 0.00

• Slovakia ETRS89 <--> "S-JTSK [JTSK03]": OK (using sk_gku_JTSK03_to_JTSK)
  echo 48.8 19.5 | PROJ_NETWORK=ON PROJ_DEBUG=2 PROJ_DATA=data bin/cs2cs ETRS89 "S-JTSK [JTSK03]"
  48d48'1.607"N 19d30'5.997"E 0.000

• Hungary ETRS89 <--> HD72: OK
  echo 47.5 19.0 | PROJ_NETWORK=ON PROJ_DEBUG=2 PROJ_DATA=data bin/cs2cs HD72 ETRS89
  47d29'59.031"N 18d59'55.948"E 0.000

• Hungary ETRF2000 <--> HD72: OK
  echo 47.5 19.0 | PROJ_NETWORK=ON PROJ_DEBUG=2 PROJ_DATA=data bin/cs2cs HD72 ETRF2000
  47d29'59.033"N 18d59'55.954"E 0.000

• Albania ETRS89 <--> Albanian 1987: OK (uses Albanian 1987 to ETRS89-ALB [KRGJSH 2010] (1))
  echo 41.3 19.8 | PROJ_NETWORK=ON PROJ_DEBUG=2 PROJ_DATA=data bin/cs2cs ETRS89 'Albanian 1987'
  41d18'1.691"N 19d48'5.66"E 0.000

• Greece ETRS89 <--> GGRS87: OK (uses GGRS87 to WGS 84 (1) and ETRS89 to WGS 84 (1))
  echo 38.0 23.7 | PROJ_NETWORK=ON PROJ_DEBUG=2 PROJ_DATA=data bin/cs2cs ETRS89 GGRS87
  37d59'50.643"N 23d41'53.9"E 0.000

• Finland ETRS89 <--> KKJ: OK (uses fi_nls_ykj_etrs35fin.json)
  echo 62.6 25.5 | PROJ_NETWORK=ON PROJ_DEBUG=2 PROJ_DATA=data bin/cs2cs ETRS89 KKJ
  62d35'58.984"N 25d30'12.285"E 0.000

• Norway ETRS89 <--> NGO 1948 (uses ETRS89 to NGO 1948 (2))
  echo 60.1 10.8 | PROJ_NETWORK=ON PROJ_DEBUG=2 PROJ_DATA=data bin/cs2cs ETRS89 "NGO 1948"
  60d5'57.23"N 10d48'17.477"E 0.000

• Latvia LKS-92 <--> LKS-2020 (not very significant given that both were in fact ETRS89-based systems without saying so). Uses lv_lgia_lks92to2020.tif
  echo 56.9 21.4 | PROJ_NETWORK=ON PROJ_DEBUG=2 PROJ_DATA=data bin/cs2cs LKS-92 LKS-2020
  56d54'0.003"N 21d23'59.997"E 0.000

Not clear, but likely changing in good direction

• Kosovo ETRS89 <--> MGI 1901

  With 9.7.1, using a Helmert transformation for North Macedonia

  $ echo 42.2 20.7 | PROJ_NETWORK=ON PROJ_DEBUG=2 PROJ_DATA=data bin/cs2cs ETRS89 'MGI 1901'
  Using coordinate operation Inverse of MGI 1901 to ETRS89 (5)
  42d11'58.825"N 20d42'19.036"E 0.000

  $ echo 42.2 20.7 | PROJ_NETWORK=ON PROJ_DEBUG=2 PROJ_DATA=data bin/cs2cs ETRS89 'MGI 1901' --area Kosovo
  Using coordinate operation Inverse of MGI 1901 to ETRS89 (5)
  42d11'58.825"N 20d42'19.036"E 0.000

  With 9.8.0, likely better when specifying --area Kosovo

  $ echo 42.2 20.7 | PROJ_NETWORK=ON PROJ_DEBUG=2 PROJ_DATA=data bin/cs2cs ETRS89 'MGI 1901'
  Using coordinate operation Ballpark geographic offset from ETRS89 to MGI 1901
  42d12'N 20d42'E 0.000

  $ echo 42.2 20.7 | PROJ_NETWORK=ON PROJ_DEBUG=2 PROJ_DATA=data bin/cs2cs ETRS89 'MGI 1901' --area Kosovo
  Using coordinate operation ETRS89 to ETRS89-XKX [KOSOVAREF01] + Inverse of MGI 1901 to ETRS89-XKX [KOSOVAREF01] (1)
  42d12'4.269"N 20d41'8.749"E 0.000

Not clear if it is a bug or feature

• Estonia ETRS89 <--> NKG_ETRF14

  • With 9.7.1: Using coordinate operation Inverse of EST97 to ETRS89 (1) + Conversion from EST97 (geog2D) to EST97 (geocentric) + Inverse of NKG_ETRF14 to EST97 (1) + Conversion from NKG_ETRF14 (geocentric) to NKG_ETRF14 (geog2D)
    echo 58.4 26.8 0 2026 | PROJ_NETWORK=ON PROJ_DEBUG=2 PROJ_DATA=data bin/cs2cs ETRS89 NKG_ETRF14
    58d24'0.002"N 26d47'59.999"E 0.009 2026

  • With 9.8.0: ballpark noop
    echo 58.4 26.8 0 2026 | PROJ_NETWORK=ON PROJ_DEBUG=2 PROJ_DATA=data bin/cs2cs ETRS89 NKG_ETRF14
    58d24'N 26d48'E 0.000 2026

"Catalonia ETRS89 <--> ED50 : KO" is fixed / worked around by this PR. But Serbia ETRS89 <--> MGI 1901 not .
To have in mind our support of ETRS89 <--> MGI 1901 in the Balkans might have always been clunky due to relying on bounding boxes, that are massively overlapping, rather than country borders to select the appropriate Helmert transformation. In that respects having per country ETRS89-XXX datums might help if users use new codes (cough cough)

@rouault Is KO short for knocked out?

In general this sample looks better than I expected, but doesn't neglect the problems already identified. Do you have the transformations above in a script that's easy to run again? Could be useful in combination with git bisect to determine which revision of the EPSG database the first problems occur in.

I've poked around with the Danish transformations and as far as I can tell everything is working as expected.

but that might be a false positive given use of SWEREF99 is likely well established, and if so RT90 to SWEREF99 (1) is used in both cases (logically due to ETRS89 not being directly involved) :

It is, the Swedish realisation of ETRS89 has always been known as SWEREF99. The same goes for a number of countries across Europe and those should all be fine.

Is KO short for knocked out?

yep: non working/broken

Do you have the transformations above in a script that's easy to run again?

just pasted my above report in

#!/bin/sh
export PROJ_NETWORK=ON
# export PROJ_DEBUG=2
export PROJ_DATA=data
echo 41.4 2.17 |  bin/cs2cs "ED50" "ETRS89"
echo 44.5 20.9 |  bin/cs2cs ETRS89 'MGI 1901'
echo 59.4 18.1 |  bin/cs2cs RT90 ETRS89
echo 40 0 |  bin/cs2cs "ED50" "ETRS89"
echo 38.7 -9.1 |  bin/cs2cs ETRS89 "Datum 73"
echo 41.89 12.48 |  bin/cs2cs ETRS89 "Monte Mario"
echo 49 2 |  bin/cs2cs NTF ETRS89
echo 49 2 |  bin/cs2cs "RGF93 v2b" NTF
echo 50.1 8.6 |  bin/cs2cs ETRS89 DHDN
echo 52.6 13.3 |  bin/cs2cs ETRS89 "Pulkovo 1942(83)"
echo 53.2 -7.9 |  bin/cs2cs ETRS89 TM75
echo 51.7 -0.2 |  bin/cs2cs OSGB36 ETRS89
echo 46.1 14.5 |  bin/cs2cs ETRS89 'MGI 1901'
echo 45.6 16.2 |  bin/cs2cs 'MGI 1901' EPSG:3765
echo 53.1 18.8 |  bin/cs2cs "Pulkovo 1942(58)" EPSG:2177
echo 50 14.5 | bin/cs2cs ETRS89 "S-JTSK / Krovak East North"
echo 48.8 19.5 |  bin/cs2cs ETRS89 "S-JTSK [JTSK03]"
echo 47.5 19.0 |  bin/cs2cs HD72 ETRS89
echo 47.5 19.0 |  bin/cs2cs HD72 ETRF2000
echo 41.3 19.8 |  bin/cs2cs ETRS89 'Albanian 1987'
echo 38.0 23.7 |  bin/cs2cs ETRS89 GGRS87
echo 62.6 25.5 |  bin/cs2cs ETRS89 KKJ
echo 60.1 10.8 |  bin/cs2cs ETRS89 "NGO 1948"
echo 56.9 21.4 |  bin/cs2cs LKS-92 LKS-2020

with reference output of 9.7.1 being

41d23'55.951"N	2d10'7.863"E 0.000
44d29'59.771"N	20d54'19.548"E 0.000
59d23'58.209"N	18d5'47.56"E 0.000
39d59'55.79"N	0d0'4.339"W 0.000
38d41'57.148"N	9d6'3.059"W 0.000
41d53'21.684"N	12d28'48.673"E 0.000
48d59'59.756"N	1d59'57.4"E 0.000
49d0'0.244"N	2d0'2.6"E 0.000
50d6'4.179"N	8d36'3.561"E 0.000
52d36'1.397"N	13d18'6.728"E 0.000
53d11'59.102"N	7d53'57.304"W 0.000
51d42'1.767"N	0d12'5.697"W 0.000
46d6'1.121"N	14d30'17.036"E 0.000
476212.82	5051143.89 0.00
5885560.92	6553456.32 0.00
-738542.81	-1053416.58 0.00
48d48'1.607"N	19d30'5.997"E 0.000
47d29'59.031"N	18d59'55.948"E 0.000
47d29'59.033"N	18d59'55.954"E 0.000
41d18'1.691"N	19d48'5.66"E 0.000
37d59'50.643"N	23d41'53.9"E 0.000
62d35'58.984"N	25d30'12.285"E 0.000
60d5'57.23"N	10d48'17.475"E 0.000
56d54'0.003"N	21d23'59.997"E 0.000

I tried some vertical transformations, and they are apparently giving something not 0 for the z

#!/bin/sh
export PROJ_NETWORK=ON
export PROJ_DATA=data
echo 40 -4 0 |  bin/cs2cs EPSG:4937 EPSG:4258+5782 # Spain
# 40.00	-4.00 -51.71
echo 46 2 0 | bin/cs2cs EPSG:4937 EPSG:4258+5720 # France
# 46.00	2.00 -49.21
echo 52 11 0 | bin/cs2cs EPSG:4937 EPSG:4258+7837 # Germany
# 52.00	11.00 -43.64
echo 47 7 0 | bin/cs2cs EPSG:4937 EPSG:2056+5728 # Switzerland
# 2566639.34	1205531.92 -49.36
echo 53 6 0 | bin/cs2cs EPSG:4937 EPSG:28992+5709 # Netherlands
# 196139.43	557179.03 -41.69
echo 51 4 0 | bin/cs2cs EPSG:4937 EPSG:3812+5710 # Belgium
# 624112.73	687814.37 -42.45
echo 51 4 0 | bin/cs2cs EPSG:4937 EPSG:3447+5710
# 125112.73	188814.37 -42.45
echo 51 4 0 | bin/cs2cs EPSG:4937 EPSG:31370+5710
# 124114.92	187817.06 -42.45
echo 51 4 0 | bin/cs2cs EPSG:4937 EPSG:31370+5710
# 124114.92	187817.06 -42.45

I tried some vertical transformations, and they are apparently giving something not 0 for the z

These values all look about right to me. The difference between ellipsoid and geoid is in the range 40-50 m in Europe.

These values all look about right to me

and I've checked they are the same for 9.7.1 and 9.8 branch without this PR

I've now tried to narrow down the EPSG update that introduced most problems. Using git bisect I come to the conclusion that EPSG v. 12.033 is the culprit introduced in 047e5dbc74ef872828 . Unfortunately that's the first EPSG update in the 9.8.0 milestone, so that's about six months of EPSG updates that'll get reverted if we base 9.8.1 on the latest revision that doesn't break ETRS89.

EPSG v. 12.033 is the culprit introduced in 047e5dbc74ef872828

working on reverting to 12.029 (only in 9.8 branch. we have time to figure out a better fix for master)

working on reverting to 12.029 (only in 9.8 branch. we have time to figure out a better fix for master)

available in #4741 . I don't think we have lost that much in content. AFAICS 99% of the EPSG 12.029 to 12.054 changes were ETRS89 related. One of the loss identified is the recently added record for no_kv_HREF2025A_NN2000_EUREF89 grid, but this relies on the new NN2000:2025 height vertical CRS. Could likely be manually reapplied, but hopefully @himsve will agree that can wait for the more deep topics of this issue to have been sorted out first. Also some SIRGAS-Chile updates (new realization).
The main concern is if early adopters of 9.8.0 have started to use the new ETRS89-XXX CRS codes (I mean when a new CRS code has been allocated, not just renames of existing CRS), in which case they will be broken by this revert. But nothing we can do about when using a revert strategy...

available in #4741 . I don't think we have lost that much in content.

That's good.

The main concern is if early adopters of 9.8.0 have started to use the new ETRS89-XXX CRS codes

I suspect there's very few of those cases. I think we can take that risk.

I'm starting to loose track of the various PR's. We have this one (#4736) which I guess we can put on the backburner for a bit because the worst problems are adressed in #4741 that downgrades the EPSG 10.029. And then there's #4739 which backports this PR but conflicts somewhat with #4736. Is my understanding correct?

We have this one (#4736) which I guess we can put on the backburner for a bit because the worst problems are adressed in #4741 that downgrades the EPSG 10.029. And then there's #4739 which backports this PR but conflicts somewhat with #4736. Is my understanding correct?

For the 9.8 branch, the strategy is indeed to apply #4741 with the downgrade to EPSG 12.029, for a 9.8.1 to be published very soon, and defer on #4739 (9.8 backport of this PR) which I marked as DRAFT so it is not accidentally merged.
What we will do afterwards on the 9.8 branch remains to be decided: will we give it some extra life with a potential later 9.8.2 ? If so, we can either freeze the EPSG version to 12.029 and only apply non-database fixes, so really strict maintenance. Or... once there's an EPSG release that fixes issues, git revert PR #4741 before applying the > 12.054 update. My point of view would be that if we consider that 9.8.0 is broken, 9.8.1 will be the true 9.8 first operational release, and then the ETRS89-XXX changes are too disturbing to be adequate for a maintenance branch.

For the master branch, we can wait a bit to decide the strategy. This PR (#4736) might or might not be useful depending on what kind of changes the next EPSG release will contain.

Do we have a reason to delay the release of 9.8.1 with #4741 applied ? I can volunteer to do it

For the master branch, we can wait a bit to decide the strategy. This PR (#4736) might or might not be useful depending on what kind of changes the next EPSG release will contain.

Yeah, I agree. Let's see how things play out. Maybe put this PR as a draft as well so it doesn't get merged unintentionally?

Do we have a reason to delay the release of 9.8.1 with #4741 applied ? I can volunteer to do it

No, let's get it out there. Thanks for volunteering!

@rouault wrote

I'm wondering how much national geodetic authorities have been involved in that process...

I fear that it was the National Land Survey of Finland who asked EPSG to create ETRS89-FIN. And by the same a large bunch of new EPSG codes were created for us, and some others were updated, for example, our most common EPSG:3067 is now using ETRS89-FIN https://epsg.org/crs/wkt/id/3067.

I fear that it was the National Land Survey of Finland who asked EPSG to create ETRS89-FIN.

actually was a two step process. First EUREF-FIN was created some time ago (it is already there in 9.7.1), and then it was renamed to "ETRS89-FIN [EUREF-FIN]" in the more recent releases when the processus of creating national realizations was generalized.
I'm not sure what would have been the best way to approach this transition, but it went too silent for my taste. Only EPSG and a few national geodetic authorities have likely been aware of the big picture. Personally, I only started to understand that it was a non trivial and generalized change occurring while we were in the middle of it. Some documentation (a guidance note or such) would also likely have been welcome for integrators like us. But to be realistic, only real world testing (as we did...) could have fully uncover issues.

I'd wish there would be a @epsg.org public mailing list and a public issue tracker, to avoid all the 1-1 discussions that occur currently. That would help more effective community and shared knowledge building to happen (I wanted to CC Roger Lott about that but can't find its github handle...) .

As a suggestion to avoid future problems (maybe this should be in the mailing list), we can have a set of unit test with the expected values from the different geodetic agencies.

For instance, the transformation in Belgium between EPSG:31370 and EPSG:3812 is expected to work in a specific way by the NGI. If we have test for such specific transformation, we can detect changes both in PROJ code or in EPSG pretty early.

I would suggest that the agencies suggest themselves what they want to test as src_crs, dst_crs, src_point, dst_point, (tolerance).
As the purpose is to test the transformation selected, just one or two points is enough. Those tests should somehow include a way to contact the responsible agency in case something is detected (something future proof, please).

Also it is important to consider with (and maybe also without) access to cdn.proj.org. The Belgian problem described above is about using a grid or a Helmert transformation, something that would trigger without grid files anyhow.

Don't forget about vertical systems and geoid models.

As a suggestion to avoid future problems (maybe this should be in the mailing list), we can have a set of unit test with the expected values from the different geodetic agencies.

We need to encourage this. Yesterday I started on a Danish test setup that could be shared as an example, but I got slowed down due to the issue reported in #4740. Once that's sorted it should be easy for geodetic agencies to set up tests for their EPSG submissions.

I would suggest that the agencies suggest themselves what they want to test as src_crs, dst_crs, src_point, dst_point, (tolerance).
As the purpose is to test the transformation selected, just one or two points is enough. Those tests should somehow include a way to contact the responsible agency in case something is detected (something future proof, please).

A template gie file with a header section for contact information would go a long way. Add a "For geodetic authorities" page on the websites on how to fill it out and contribute to the project and I think we are on the right track. A campaign to get in touch with the right people would of course also be necessary but I am confident I can get the message out on the right channels.

Once that's sorted it should be easy for geodetic agencies to set up tests for their EPSG submissions.

bootstrap of the effort at #4744

bootstrap of the effort at #4744

Thanks, that's a good start! I have more thoughts on this, hopefully I can find some time later this week to act on them.

I hope the GeoTools folks are following along so they can step over the troubles when the day comes that they update their EPSG db.

I hope the GeoTools folks are following along so they can step over the troubles when the day comes that they update their EPSG db.

I sent a message to the GeoServer/GeoTools developer group https://discourse.osgeo.org/t/please-notice-the-troubles-with-epsg-version-12-033/153374
GeoTools does not update their EPSG database especially often so that day may not come very soon. Latest update from 9.6.1 into 11.0.3 happened one year ago geotools/geotools@8528e24

As a suggestion to avoid future problems (maybe this should be in the mailing list), we can have a set of unit test with the expected values from the different geodetic agencies.

We need to encourage this. Yesterday I started on a Danish test setup that could be shared as an example, but I got slowed down due to the issue reported in #4740. Once that's sorted it should be easy for geodetic agencies to set up tests for their EPSG submissions.

I would suggest that the agencies suggest themselves what they want to test as src_crs, dst_crs, src_point, dst_point, (tolerance).
As the purpose is to test the transformation selected, just one or two points is enough. Those tests should somehow include a way to contact the responsible agency in case something is detected (something future proof, please).

A template gie file with a header section for contact information would go a long way. Add a "For geodetic authorities" page on the websites on how to fill it out and contribute to the project and I think we are on the right track. A campaign to get in touch with the right people would of course also be necessary but I am confident I can get the message out on the right channels.

Hi @kbevers, I can help out with a communication campaign for the Europe region. I have a contact list of geodetic authorities. Contact me on email when you want to work on this. ~Jeffrey