Skip to content

Wikidata mapping queries are not working #1911

@osma

Description

@osma

URL address of the page where you encountered the problem

e.g. https://test.dev.finto.fi/yso/en/page/p14629 (yso: aardvark)

Description of the problem

The Wikidata label for the corresponding concept is not showing (it shows up as wd:Q46212 when it should be "aardvark"):

Image

This also currently causes the Cypress test "contains mappings for hash namespace" to fail. This is an outstanding issue in PR #1904 which attempts to fix breaking or flaky Cypress tests.

Additional information (e.g. screenshots) about the problem

I traced why this happens. Long story short: WDQS (the Wikidata SPARQL endpoint) doesn't like the SPARQL queries performed by EasyRdf. It responds with an error

403, Please respect our robots policy https://wikitech.wikimedia.org/wiki/Robot_policy (1e30f7b)

I understand that it is prudent to restrict traffic by bots, especially in these times of rampant AI bots. But WDQS is a SPARQL API service - it is intended for machine use. And Skosmos tries to follow the policy...

Anyway, by playing with curl I found out that WDQS currently doesn't like HTTP/1.1 requests (--http1.1 option in curl) and will always(?) respond like that when queried over HTTP/1.1. It works fine with HTTP/2.0. But the built-in HTTP client in EasyRdf only supports HTTP/1.1. (Maybe the optional Laminas HTTP client would work better?)

However, WDQS is also slow and unreliable. Recently, QLever has started providing a Wikidata SPARQL endpoint. We could consider switching to that. I'm going to prepare a PR on this soon.

The browser you used when the problem appeared

Firefox

Metadata

Metadata

Assignees

Labels

Type

Projects

No projects

Milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions