Name and Version

Tests are done on latest master (19e92c3).

Operating systems

Linux

Which llama.cpp modules do you know to be affected?

llama-server

Command line

(any command that could start a llama-server instance with Web UI)

Problem description & steps to reproduce

PR #23701 fixes the long-standing issue of unable to provide caching support for embedded static files, and I really appreciate it. However, it's "simplistic" approach on implementing ETag validation is not sufficent for real-world use, and is actually out of spec.

The current logic looks like this. A simple string comparsion and that's it.

However, it did not take "weak" ETags into consideration. I have a Nginx server acting as reverse proxy to llama-server, and gzip compression is enabled on it (since llama-server still does not support that; sad). When request for a static file (say /bundle.js) come through Nginx, it compresses the response, and modifies ETag to be weak (i.e. prepends W/) in the process. This is per spec, as RFC 9110 Section 8.8.3 states:

If an origin server provides an entity tag for a representation and the generation of that entity tag does not satisfy all of the characteristics of a strong validator (Section 8.8.1), then the origin server MUST mark the entity tag as weak by prefixing its opaque value with "W/" (case-sensitive).

Then, on the second request, the user's browser sends a If-None-Match header with this "weakened" ETag. While RFC 9110 Section 13.1.2 states that the "weak" comparsion function must be used here:

A recipient MUST use the weak comparison function when comparing entity tags for If-None-Match (Section 8.8.3.2), since weak entity tags can be used for cache validation even if there have been changes to the representation data.

And RFC 9110 Section 8.8.3.2 defines weak comparison function to be ignoring the "weakened" status of an ETag:

"Weak comparison": two entity tags are equivalent if their opaque-tags match character-by-character, regardless of either or both being tagged as "weak".

Yet the current logic does not work like this, thus fails browser cache validation, forcing the file to be re-downloaded again on each request.

To reproduce, launch an instance of llama-server with Web UI, send a GET request to /bundle.js to get it's ETag, then do the request again with a If-None-Match: W/"xxx" header. Observe that llama-server fails to recognize the ETag and responds with HTTP 200 (instead of HTTP 304).

First Bad Commit

It behaves like this since the ETag feature's introduction, i.e. PR #23701.

Relevant log output

[user@host ~]$ curl -I http://192.168.1.100:9070/bundle.js
HTTP/1.1 200 OK
Accept-Ranges: bytes
Keep-Alive: timeout=5, max=100
Access-Control-Allow-Origin: 
Server: llama.cpp
Content-Length: 5285341
Content-Type: application/javascript; charset=utf-8
ETag: "0xfe3ab1fcfb4ea743"

[user@host ~]$ curl -I http://192.168.1.100:9070/bundle.js -H 'If-None-Match: "0xfe3ab1fcfb4ea743"'
HTTP/1.1 304 Not Modified
Accept-Ranges: bytes
Access-Control-Allow-Origin: 
Server: llama.cpp
Content-Length: 0
ETag: "0xfe3ab1fcfb4ea743"
Keep-Alive: timeout=5, max=100

[user@host ~]$ curl -I http://192.168.1.100:9070/bundle.js -H 'If-None-Match: W/"0xfe3ab1fcfb4ea743"'
HTTP/1.1 200 OK
Accept-Ranges: bytes
Keep-Alive: timeout=5, max=100
Access-Control-Allow-Origin: 
Server: llama.cpp
Content-Length: 5285341
Content-Type: application/javascript; charset=utf-8
ETag: "0xfe3ab1fcfb4ea743"