GMT has ~900 tests. Some tests are known to fail and are ignored with the magic comment line "# GMT_KNOWN_FAILURE". Some tests fail due to slightly differences among different OSs and are allowed to pass with high GRAPHICSMAGICK_RMS values.

Here we list all the known failing tests to better track them.

This PR is used for general discussions only. For a specific failing test, please open a new issue for in-depth discussions.

Failing tests on all platforms:

Output from command:

find . -name "*.sh" -exec grep GMT_KNOWN_FAILURE {} + | awk -F@ '{print "- [ ]", $1 }'

• test/genper/gridlines2.sh
• test/grdcontour/periodic.sh
• test/grdview/wrap.sh
• test/mgd77/test_cm4.sh
• test/psbasemap/poption.sh
• test/psbasemap/rotalign.sh
• test/pscoast/pseudo.sh
• test/psxy/clipping4.sh
• test/psxy/clipping5.sh
• test/psxy/clipping6.sh
• test/psxy/nearpole.sh
• test/psxy/nojump.sh
• test/psxy/outline.sh
• test/psxy/wrapper.sh

Failing tests on Windows:

• doc/scripts/GMT_movie_progress.sh (Issues with doc/scripts/GMT_movie_progress.sh on Windows #6007)
• test/gmtmex/WL_example_3.sh
• test/grdvector/bothg.sh
• test/grdvector/vectors.sh (Increase GRAPHICSMAGICK_RMS for vectors.sh #6074)
• test/greenspline/gspline_5.sh
• test/mapproject/oblmerc_down.sh (Increase GRAPHICSMAGICK_RMS for oblmerc_down.sh #6073)
• test/psbasemap/map_JE.sh (Missing annotation in map_JE.sh test on Windows #6072) (Adjust map_JE.sh test to pass on windows #6076)
• grdgdal/gdal_nn.sh (Failed) (Fixed by Fix gdal_nn.sh test and add orig PS #2724)
• test/grdvector/met_uv.sh (Set met_uv.sh as a known failure on windows/linux #6457)

Failing tests on Linux:

• test/gmtmex/WL_example_3.sh
• test/grdvector/met_uv.sh (Set met_uv.sh as a known failure on windows/linux #6457)

Tests with high GRAPHICSMAGICK_RMS value:

Output from command

find . -name "*.sh" -exec grep GRAPHICSMAGICK_RMS {} + | awk -F@ '{print "- [ ]", $1}'

• test/grdvector/vectors.sh (GRAPHICSMAGICK_RMS = 0.02 due to rounding of grid points at the map edges)
• test/mapproject/oblmerc_down.sh (GRAPHICSMAGICK_RMS = 0.01 due to minor gridline differences)
• ./test/pscoast/oblsuite.sh:# GRAPHICSMAGICK_RMS = 0.02
• ./test/pscoast/oblsuite_N.sh:# GRAPHICSMAGICK_RMS = 0.004
• ./test/geodesy/case_largeR_noW.sh:# GRAPHICSMAGICK_RMS = 0.0435
• ./test/geodesy/case_largeR_withW.sh:# GRAPHICSMAGICK_RMS = 0.0435
• ./test/geodesy/case_smallR_withW.sh:# GRAPHICSMAGICK_RMS = 0.0655
• ./test/geodesy/gpsgridder1.sh:# GRAPHICSMAGICK_RMS = 0.0565
• ./test/geodesy/gpsgridder2.sh:# GRAPHICSMAGICK_RMS = 0.0565
• ./test/geodesy/case_smallR_noW.sh:# GRAPHICSMAGICK_RMS = 0.0655
• ./test/genper/snyder_map.sh:# GRAPHICSMAGICK_RMS = 0.0055
• ./test/pscontour/thickness.sh:# GRAPHICSMAGICK_RMS = 0.02
• ./test/grdcontour/grdcontour.sh:# GRAPHICSMAGICK_RMS = 0.04 (Increase limit on grdcontour tests #6348)
• ./doc/scripts/GMT_grinten.sh # GRAPHICSMAGICK_RMS = 0.0035 (Increase GRAPHICSMAGICK limit for GMT_grinten.sh #6459)