Fix/wms provider url decoding by benoitdm-oslandia · Pull Request #62446 · qgis/QGIS

benoitdm-oslandia · 2025-06-27T07:21:58Z

This closes #62310 and reopens #59144 plus some changes:

rebase from master
remove merge and revert commits
squash some commits
fix qgshttpheaders accordingly
add tests

github-actions · 2025-06-27T08:27:59Z

🍎 MacOS Qt6 builds

Download MacOS Qt6 builds of this PR for testing.
This installer is not signed, control+click > open the app to avoid the warning
(Built from commit 0d10f88)

🪟 Windows builds

Download Windows builds of this PR for testing.
Debug symbols for this build are available here.
(Built from commit 7e14472)

🪟 Windows Qt6 builds

Download Windows Qt6 builds of this PR for testing.
(Built from commit 0d10f88)

github-actions · 2025-07-16T03:07:09Z

The QGIS project highly values your contribution and would love to see this work merged! Unfortunately this PR has not had any activity in the last 14 days and is being automatically marked as "stale". If you think this pull request should be merged, please check

that all unit tests are passing
that all comments by reviewers have been addressed
that there is enough information for reviewers, in particular
- link to any issues which this pull request fixes
- add a description of workflows which this pull request fixes
- add screenshots if applicable
that you have written unit tests where possible
In case you should have any uncertainty, please leave a comment and we will be happy to help you proceed with this pull request.
If there is no further activity on this pull request, it will be closed in a week.

github-actions · 2025-07-23T03:09:14Z

While we hate to see this happen, this PR has been automatically closed because it has not had any activity in the last 21 days. If this pull request should be reconsidered, please follow the guidelines in the previous comment and reopen this pull request. Or, if you have any further questions, just ask! We love to help, and if there's anything the QGIS project can do to help push this PR forward please let us know how we can assist.

benoitdm-oslandia · 2025-07-23T06:36:09Z

ping @troopa81, for review please!

benoitdm-oslandia · 2025-07-30T13:25:29Z

rebase from master

nyalldawson · 2025-08-04T23:51:17Z

@benoitdm-oslandia does this change break stable api? I.e would existing code relying on the old behavior (such as constructing a wms layer from a hard coded source string) be impacted?

github-actions · 2025-08-19T03:00:43Z

The QGIS project highly values your contribution and would love to see this work merged! Unfortunately this PR has not had any activity in the last 14 days and is being automatically marked as "stale". If you think this pull request should be merged, please check

that all unit tests are passing
that all comments by reviewers have been addressed
that there is enough information for reviewers, in particular
- link to any issues which this pull request fixes
- add a description of workflows which this pull request fixes
- add screenshots if applicable
that you have written unit tests where possible
In case you should have any uncertainty, please leave a comment and we will be happy to help you proceed with this pull request.
If there is no further activity on this pull request, it will be closed in a week.

benoitdm-oslandia · 2025-08-20T12:18:30Z

@benoitdm-oslandia does this change break stable api? I.e would existing code relying on the old behavior (such as constructing a wms layer from a hard coded source string) be impacted?

Sorry, what do you mean by hard coded? Can you provide a more precise usecase?

kannes · 2025-08-21T07:23:26Z

I am 99% sure Nyall means URI strings that users might have saved and that are required to keep working. E. g. if you have a custom basemap plugin that contains a list of services defined by URIs. Will those keep working fine or is there potential breakage?

benoitdm-oslandia · 2025-08-21T10:19:16Z

@nyalldawson @kannes I am 99% sure the plugins will keep working fine :) The fixes brought by this PR are mainly to improve the support of url decoding and to homogeneize the uri encoding. But maybe some usecases are not tested!

nyalldawson · 2025-08-22T00:25:31Z

@benoitdm-oslandia

but there's changes in tests like this:

and

That to me suggests that the previous hardcoded URIs are now broken, which would also affect plugins and scripts =>this is an api break.

benoitdm-oslandia · 2025-08-22T07:19:14Z

@nyalldawson I add some usecases in the test you mentioned and, from what I understood, it does not generate an API break.

cioddi · 2025-08-29T12:09:28Z

I will try to explain the situation.

The current state is, parts of the source configuration is stored in the form of URL parameters. When the value of an individual key is retrieved, it is always decoded using QUrl::PrettyDecoded (https://doc.qt.io/qt-6/qurlquery.html#queryItems).

The change in die PR will make sure values that are added to the QUrlQuery object are fully URL encoded (https://doc.qt.io/qt-6/qurl.html#ComponentFormattingOption-enum) first. The retrieval function QUrlQuery::queryItems parameter was changed from the default QUrl::PrettyDecoded (from the Qt docs: "The component is returned in a "pretty form", with most percent-encoded characters decoded. The exact behavior of PrettyDecoded varies from component to component and may also change from Qt release to Qt release. This is the default.") to the reliable and predictable QUrl::FullyDecoded. This ensures that the exact value that was passed in a data source URL parameter can be retrieved from it again.

Why is this important?
Especially in more complex QGIS server deployments we often rely on the non WMS spec "map" URL parameter to specify information about which project or configuration is supposed to be processed with the request. The value often contains characters that collide with URL parameter reserved characters such as ?, &, = so the value must be URL encoded (which is fine).

Now if we consume the WMS provided by QGIS server using a QGIS client the only way to make this work with current versions of QGIS is to double URL encode the values passed to the map parameter. That way at the time QGIS assembles the URL for an HTTP request to the server the value becomes single URL encoded because that is the way QUrlQuery::queryItems works by default.

(The server acts only as an example here, but could be any data source that expects a URL parameter value the QGIS client would send in a URL encoded form)

What this PR will break

This will break projects that are currently using the double URL encoded value workaround in a datasource. In those cases the source configuration must be updated. After updating the double URL encoded URL parameter values to single URL encoded ones, those will start working again. (this must be added to breaking changes)

General thoughts

Using a URL-parameter String is probably not the best way to store/handle the configuration of a data source and maybe in the future it should be considered to switch to something that better fits the requirements (like XML that is already used anyway for project files). This PR makes sure though that we can work with the current URL parameter config in a reliable way.

troopa81 · 2025-11-03T13:47:54Z

-      const QString url = uriComponents[QStringLiteral( "url" )].toString();
+      QUrl decodedUri = QUrl::fromPercentEncoding( uriComponents[QStringLiteral( "url" )].toString().toLocal8Bit() );
+      if ( decodedUri.scheme().isEmpty() && decodedUri.path().startsWith( "http" ) )
+        // url can be encoded twice and needs another % decoding


I don't get why they could be encoded twice ? or even once ? Here we get the non-encoded url, the one store in the layer. We encode only in QgsDataSourceUri::encodedUri(), no?

After discussion, we should display the url as it was entered by the user. If he enters an already encoded URL, we display it encoded.

In fact, as it is saved encoded, it had to be decoded at least once!

troopa81 · 2025-11-03T15:51:47Z

One problem I have encountered. When examining the source properties the URL displayed in the "information"-tab is not URL-decoded and does not work if I click on the link. If I right click->open URL I get this something like this: "Cannot show URL Failed to load URL https%3A%2F%2Fsgx.geodatenzentrum.de%2Fwms_basemapde%3FVersion=1.3.0."

But why would you want to enter an already percent encoded URL. I understand that we can have a MAP or postgres url subpart that could be encoded but IMHO encoding the whole URL makes no sense. In the case of postgres/MAP subpart, we should let it encoded because it would be invalid otherwise.

benoitdm-oslandia · 2025-11-04T15:43:33Z

@troopa81 I have found a bug with the vectortile provider while adjusting the tests. It does not seem a nice fix to me :(

…roperly decoded when retrieved, preventing any potential loss of information.

URl-encoded in URL-parameter values characters will be URL-decoded on retrieval.

…ded in tests/src/providers/testqgswmsprovider.cpp

troopa81

After long discussion and thorough reviewing, I'm OK with the PR in this actual state.

Thank you @benoitdm-oslandia

… QgsDataSourceUri param setter and getter functions to URI serialization QgsDataSourceUri::encodedUri()

…sDataSourceUri strings in QgsDataSourceUri::setEncodedUri

add decoding step before adding the "url" value to QgsDataSourceUri in TestQgsIdentify::identifyVectorTile(), as QgsDataSourceUri now reliably returns the values that were provided. adjust tests to QgsDataSourceUri full value URL-encoding fix test tests/src/core/testqgshttpheaders.cpp fix test tests/src/core/testqgssensorthingsconnection.cpp fix test tests/src/core/testqgstiledsceneconnection.cpp fix test tests/src/core/testqgsvectortileconnection.cpp fix test tests/src/python/test_qgsvectortile.py fix test tests/src/server/wms/test_qgsserver_wms_parameters.cpp fix tests tests/src/core/testqgshttpheaders.cpp fix most tests in tests/src/core/testqgsvectortilelayer.cpp

… of VectorTile Sources in QgsDataSurceUri

…rts with XXX%3A

…Query

ViperMiniQ · 2026-01-22T19:33:29Z

Hi @benoitdm-oslandia,
this broke the loading of SLPK files from local files.
Can you please have a look at #64543?

Also, some of the reported paths in the metadata look weird now and I don't think it was an intended behavior.

Prior to this PR:

After:

benoitdm-oslandia mentioned this pull request Jun 27, 2025

Fix/wms provider url decoding #59144

Closed

benoitdm-oslandia commented Jun 27, 2025

View reviewed changes

Comment thread src/providers/wms/qgswmsprovider.cpp Outdated

benoitdm-oslandia mentioned this pull request Jun 27, 2025

qgsproject: add support for base64 encoded url #62310

Closed

github-actions Bot added this to the 3.46.0 milestone Jun 27, 2025

benoitdm-oslandia self-assigned this Jul 1, 2025

benoitdm-oslandia force-pushed the fix/wms-provider-url-decoding branch from 04ed226 to 739c6ec Compare July 1, 2025 11:17

benoitdm-oslandia requested review from nirvn and troopa81 July 1, 2025 11:19

github-actions Bot added the stale Uh oh! Seems this work is abandoned, and the PR is about to close. label Jul 16, 2025

github-actions Bot closed this Jul 23, 2025

benoitdm-oslandia reopened this Jul 23, 2025

github-actions Bot removed the stale Uh oh! Seems this work is abandoned, and the PR is about to close. label Jul 23, 2025

benoitdm-oslandia force-pushed the fix/wms-provider-url-decoding branch from 739c6ec to 1f2418d Compare July 30, 2025 13:25

3nids modified the milestones: 3.46.0, 4.0.0 Aug 1, 2025

github-actions Bot added the stale Uh oh! Seems this work is abandoned, and the PR is about to close. label Aug 19, 2025

github-actions Bot removed the stale Uh oh! Seems this work is abandoned, and the PR is about to close. label Aug 20, 2025

troopa81 reviewed Nov 3, 2025

View reviewed changes

Comment thread tests/src/core/testqgsvectortilelayer.cpp Outdated

github-actions Bot removed the stale Uh oh! Seems this work is abandoned, and the PR is about to close. label Nov 3, 2025

troopa81 reopened this Nov 3, 2025

benoitdm-oslandia force-pushed the fix/wms-provider-url-decoding branch from 8d6daab to bd10a30 Compare November 4, 2025 15:39

benoitdm-oslandia force-pushed the fix/wms-provider-url-decoding branch from bd10a30 to de40da8 Compare November 17, 2025 15:14

cioddi added 3 commits November 18, 2025 16:58

ensure values in QgsDataSourceUri are URL-encoded when assigned and p…

db4472b

…roperly decoded when retrieved, preventing any potential loss of information.

fix faulty assertion in tests/src/core/testqgsdatasourceuri.cpp

33ccc49

URl-encoded in URL-parameter values characters will be URL-decoded on retrieval.

fix faulty assertion of URI parameter values not being fully URL-enco…

5365536

…ded in tests/src/providers/testqgswmsprovider.cpp

benoitdm-oslandia force-pushed the fix/wms-provider-url-decoding branch from 2dbebd1 to 25ef836 Compare November 19, 2025 08:33

troopa81 approved these changes Nov 19, 2025

View reviewed changes

cioddi and others added 11 commits November 19, 2025 16:45

move QUrl::toPercentEncoding and QUrl::fromPercentEncoding steps from…

5d89e1d

… QgsDataSourceUri param setter and getter functions to URI serialization QgsDataSourceUri::encodedUri()

make sure QgsDataSourceUri values are FullyUrlDecoded when parsing Qg…

d437c30

…sDataSourceUri strings in QgsDataSourceUri::setEncodedUri

fix double URL-encoding issue with curly braces in z,x,y placeholders…

d581ba7

… of VectorTile Sources in QgsDataSurceUri

fix(QgsProjectStorageRegistry): project uri can be urlencoded and sta…

609aa85

…rts with XXX%3A

fix(httpheaders): add missing urlencode to headers when updating QUrl…

2a4fda7

…Query

test(qgsdatasourceuri): add uri in uri test

7c0f8f4

test: add usecases to testqgswmsprovider and url encoding

e7eef34

fix url info when displaying layer properties in UI

8195e4b

fix: QgsMapLayer::generalHtmlMetadata

a0887ae

fix test test_provider_wms_gui to handle encoded uri

0d10f88

benoitdm-oslandia force-pushed the fix/wms-provider-url-decoding branch from 25ef836 to 0d10f88 Compare November 19, 2025 15:45

troopa81 merged commit 1ba5886 into qgis:master Nov 20, 2025
33 checks passed

ViperMiniQ mentioned this pull request Jan 22, 2026

Fix SLPK loading #64543

Merged

nyalldawson mentioned this pull request Feb 19, 2026

[fix] Clip raster by extent/mask algorithms (avoid encoded URL inside the XML description file) #64996

Closed

Withalion mentioned this pull request Mar 18, 2026

Add qgis patch to support qgis 4 projects MerginMaps/mobile#4379

Merged

Uh oh!

Uh oh!

Conversation

benoitdm-oslandia commented Jun 27, 2025

Uh oh!

Uh oh!

github-actions Bot commented Jun 27, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

🍎 MacOS Qt6 builds

🪟 Windows builds

🪟 Windows Qt6 builds

Uh oh!

github-actions Bot commented Jul 16, 2025

Uh oh!

github-actions Bot commented Jul 23, 2025

Uh oh!

benoitdm-oslandia commented Jul 23, 2025

Uh oh!

benoitdm-oslandia commented Jul 30, 2025

Uh oh!

nyalldawson commented Aug 4, 2025

Uh oh!

github-actions Bot commented Aug 19, 2025

Uh oh!

benoitdm-oslandia commented Aug 20, 2025

Uh oh!

kannes commented Aug 21, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

benoitdm-oslandia commented Aug 21, 2025

Uh oh!

nyalldawson commented Aug 22, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

benoitdm-oslandia commented Aug 22, 2025

Uh oh!

cioddi commented Aug 29, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

troopa81 Nov 3, 2025

Choose a reason for hiding this comment

Uh oh!

troopa81 Nov 3, 2025

Choose a reason for hiding this comment

Uh oh!

benoitdm-oslandia Nov 4, 2025

Choose a reason for hiding this comment

Uh oh!

Uh oh!

troopa81 commented Nov 3, 2025

Uh oh!

benoitdm-oslandia commented Nov 4, 2025

Uh oh!

troopa81 left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

ViperMiniQ commented Jan 22, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

7 participants

github-actions Bot commented Jun 27, 2025 •

edited

Loading

kannes commented Aug 21, 2025 •

edited

Loading

nyalldawson commented Aug 22, 2025 •

edited

Loading

cioddi commented Aug 29, 2025 •

edited

Loading