Skip to content

qgsproject: add support for base64 encoded url#62310

Closed
benoitdm-oslandia wants to merge 3 commits into
qgis:masterfrom
benoitdm-oslandia:fix/load_postgres_project
Closed

qgsproject: add support for base64 encoded url#62310
benoitdm-oslandia wants to merge 3 commits into
qgis:masterfrom
benoitdm-oslandia:fix/load_postgres_project

Conversation

@benoitdm-oslandia

@benoitdm-oslandia benoitdm-oslandia commented Jun 17, 2025

Copy link
Copy Markdown
Collaborator

In QgsDataSourceUri, if the 'url' field references a QGIS server then the 'MAP' parameter can contain an url to the project file. When the project is saved in a postgresql db, the connection url will also contains '&' and '=' characters. One solution is to urlencod twice this url and another solution is to use the encode the project file url to base64 and use the 'base64://' scheme.

closes #31192

In QgsDataSourceUri, if the 'url' field references a QGIS server then the 'MAP' parameter can contain an url to the project file.
When the project is saved in a postgresql db, the connection url will also contains '&' and '=' characters.
One solution is to use the encode the original project file url with base64 and use the 'base64://' scheme.

See issue qgis#31192
@benoitdm-oslandia benoitdm-oslandia self-assigned this Jun 17, 2025
@benoitdm-oslandia benoitdm-oslandia added Project Server Related to QGIS server labels Jun 17, 2025
@github-actions github-actions Bot added this to the 3.44.0 milestone Jun 17, 2025
@github-actions

github-actions Bot commented Jun 17, 2025

Copy link
Copy Markdown
Contributor

🪟 Windows builds

Download Windows builds of this PR for testing.
Debug symbols for this build are available here.
(Built from commit ff3247e)

🪟 Windows Qt6 builds

Download Windows Qt6 builds of this PR for testing.
(Built from commit ff3247e)

@benoitdm-oslandia benoitdm-oslandia requested a review from nirvn June 17, 2025 13:23
Comment thread src/core/project/qgsproject.cpp Outdated
@nirvn

nirvn commented Jun 19, 2025

Copy link
Copy Markdown
Contributor

@benoitdm-oslandia , hey, looked at this earlier today. I feel like if we want a base64 encoding for the MAP= request parameter, it might be better to decode it within the relevant src/server classes instead of handling it within the QgsProject class. Namely, we could simply check for and decode base64 in those two locations:

Also, what should really just happen here is for the MAP query value to be URL encoded. It's an alternative you're pointing out in the description, and it has the advantage of avoiding obfuscating the MAP source project a bit.

That said, if you see advantages to base64 that I do not see ATM, I'd suggest we go for the src/server approach I suggested above.

@benoitdm-oslandia

Copy link
Copy Markdown
Collaborator Author

@benoitdm-oslandia , hey, looked at this earlier today. I feel like if we want a base64 encoding for the MAP= request parameter, it might be better to decode it within the relevant src/server classes instead of handling it within the QgsProject class. Namely, we could simply check for and decode base64 in those two locations:

* https://github.com/qgis/QGIS/blob/master/src/server/qgis_mapserver.cpp#L595

* https://github.com/qgis/QGIS/blob/master/src/server/qgsconfigcache.cpp#L139

Also, what should really just happen here is for the MAP query value to be URL encoded. It's an alternative you're pointing out in the description, and it has the advantage of avoiding obfuscating the MAP source project a bit.

That said, if you see advantages to base64 that I do not see ATM, I'd suggest we go for the src/server approach I suggested above.

One disadvantage of the double encoding of the url is: it generate 2 different behaviours between QGIS Desktop calling the QGIS Server and an Apache (for example) calling the same QGIS Server. In the case of QGIS Desktop, you have to do a double encoding to bypass the multiple decoding from Desktop code. And in the case of Apache, you have to do a simple encoding to have it working.

In order to have a coherent behaviour you need to use the base64 encoding. Further more, in the postgresql connection you may have information you would like to hide (even if it is easily decodable).

Also, it looks strange to do a double encoding of a part of the url to bypass a strange behaviour (ie. it is not a bug but a feature) of an application. And, you never know when it would take a triple encoding to have it works or a single one because of a change in the code. IMO, using the base64 fixes the problem once and for all.

What do you think?

@nirvn

nirvn commented Jun 20, 2025

Copy link
Copy Markdown
Contributor

@benoitdm-oslandia , I'm not a QGIS server user myself, sounds like you're trying to address something that's worth fixing. I'd still feel better if the base64 encoding/decoding would be handled within src/server itself. Would that be OK with you?

@benoitdm-oslandia

Copy link
Copy Markdown
Collaborator Author

@nirvn so you would prefer I apply the decoding outside the QgsProject::read function at https://github.com/qgis/QGIS/blob/master/src/server/qgis_mapserver.cpp#L595 and https://github.com/qgis/QGIS/blob/master/src/server/qgsconfigcache.cpp#L139?

@nirvn

nirvn commented Jun 20, 2025

Copy link
Copy Markdown
Contributor

Correct

@benoitdm-oslandia

Copy link
Copy Markdown
Collaborator Author

Fixed! But there are still many places where the QgsProject::read function is called like qgslandingpageutils.cpp or qgsprocess.cpp. Not applying the fix at these places fell like "I fixed my issue but I did not look at the possible side effects".

@nirvn

nirvn commented Jun 23, 2025

Copy link
Copy Markdown
Contributor

@benoitdm-oslandia , ah oui, you should add it in qgslandingpageutils.cpp too -- basically, what I'm thinking here is that it should be a server-specific handling of base64, where the URL context QGIS server is accessed through seems to justify base64 encoding.

But e.g. for QGIS desktop's CLI, or qgis_process CLI, I'm not sure how I'd feel about base64 there :)

Happy to be challenged by others on this though.

@cioddi

cioddi commented Jun 26, 2025

Copy link
Copy Markdown
Contributor

This appears to be related to the problem I had with WMS sources that contained url-parameter values that include & or =. The problem is that all values added to QgsDataSourceUri in a Urlencoded way become urldecoded the next time it is used (e.g. in a http-request).

I believe this must be solved at the QgsDataSourceUri level for the solution to cover all supported source types. That way we can be sure a value added to QgsDataSourceUrl isn't altered on retrieval, which currently is the case.

I had a PR open for this that solves the problem, but it became stale and was closed.

#59144

The decoding is caused by QUrlQuery::queryItems (https://doc.qt.io/qt-6/qurlquery.html#queryItems) that has a default param QUrl::PrettyDecoded (https://doc.qt.io/qt-6/qurl.html#ComponentFormattingOption-enum). The only way I found to reliably get the exact value that was put in was to fully encode anything that goes in and fully decode anything that is retrieved.

@benoitdm-oslandia

Copy link
Copy Markdown
Collaborator Author

Open new PR #62446!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Project Server Related to QGIS server

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Add WMS server fails with QGIS Server with Postgresql project

4 participants