Skip to content

Multiple issues in components/prism-nginx.js #1718

@ghost

Description

  1. nginx shouldn’t inherit anything from clike. nginx doesn’t have classes, booleans, operators, etc.;
  2. Directive shouldn’t be classified as keyword, because some parameters are keywords too.

Comments

  1. The current regex fails in the following example.
http {# This must be recognized as a comment
}
  1. The current regex doesn’t consider single­‑quoted strings.
location ~ '/example'# This shouldn’t be recognized as a comment
{}

Directives

Problem statement

Directives list is obsolete and inconsistent. AFAIK, the current list is based largely on a third­‑party gist.

Currently Prism supports 339 directives. nginx contains more than 693 directives. Beside that, some directives shown in components/prism-nginx.js are undocumented, removed years ago or not directives at all. See the table below.

Directive Status in nginx
CONTENT_, DOCUMENT_, etc. Unknown
auth Renamed to pop3_auth in 2007 ⁠¹
devpoll_changes An undocumented directive
devpoll_events An undocumented directive
epoll_events An undocumented directive
fastcgi_redirect_errors Renamed to fastcgi_intercept_errors in 2006 ⁠¹
if_not_empty A parameter for fastcgi_param and scgi_param
kqueue_changes An undocumented directive
kqueue_events An undocumented directive
log_format_combined Must be log_format combined. combined is a parameter
more_set_headers A part of ngx_headers_more. It is not distributed with nginx ⁠²
optimize_server_names Replaced by server_name_in_redirect in 2008 ⁠¹, removed in 2015 ⁠³
post_action A dangerous undocumented directive
proxy A parameter ⁠¹
proxy_redirect_errors Renamed to proxy_intercept_errors in 2006 ⁠¹
proxy_upstream_fail_timeout Not supported since 2006 ⁠¹
proxy_upstream_max_fails Not supported since 2006 ⁠¹
rtsig_overflow_events Removed in 2015 ⁠¹
rtsig_overflow_test Removed in 2015 ⁠¹
rtsig_overflow_threshold Removed in 2015 ⁠¹
rtsig_signo Removed in 2015 ⁠¹
satisfy_any Replaced by satisfy in 2008 ⁠¹, removed in 2015 ⁠³
so_keepalive A parameter for listen
worker_rlimit_sigpending Removed in 2015 ⁠⁴
xslt_entities Misspelled xml_entities?

¹ ⁠https://nginx.org/en/CHANGES.
² ⁠https://github.com/openresty/headers-more-nginx-module.
³ ⁠https://hg.nginx.org/nginx/rev/2911b7e5491b.
⁴ ⁠https://hg.nginx.org/nginx/rev/967594ba7571.

There are many more removed directives. Here they are:

  • obsolete aio directives and rtsig_signo (see http://hg.nginx.org/nginx/rev/adba26ff70b5) were removed in 2015;
  • limit_zone is removed in 2014;
  • secure_link_expires is removed in 2010;
  • open_file_cache_retest is removed in 2007;
  • fastcgi_upstream_fail_timeout, fastcgi_upstream_max_fails, fastcgi_x_powered_by, memcached_upstream_fail_timeout, memcached_upstream_max_fails, proxy_pass_server, proxy_pass_x_powered_by, restrict_host_names were removed in 2006;
  • fastcgi_root, fastcgi_set_var, fastcgi_params, default_charset, post_accept_timeout, proxy_add_x_forwarded_for, proxy_pass_unparsed_uri, proxy_preserve_host, proxy_set_x_real_ip, proxy_set_x_url, proxy_x_var, redirect, server_names_hash_threshold, server_names_hash were removed in 2005.

There are many more undocumented directives. These are degradation, degrade, eventport_events, gzip_hash, gzip_no_buffer, gzip_window, http2_pool_size, http2_streams_index_size, postpone_gzipping, ssi_ignore_recycled_buffers, uwsgi_string (see https://forum.nginx.org/read.php?29,277920,277920).

A proposed solution

On the one hand, there is a relatively fast way to update the list:

  1. save directives list from components/prism-nginx.js as old.txt;
  2. replace all occurrences of | in old.txt with \n;
  3. sort lines in old.txt alphabetically;
  4. save directives list from https://nginx.org/en/docs/dirindex.html as new.txt;
  5. remove names of modules from new.txt: \s+\(\w+\);
  6. replace (^[\w]+)(?:\n\1)+$ in new.txt with \1 to remove duplicates;
  7. find entries to remove: diff old.txt new.txt | grep '^<' | sed 's/^< *//';
  8. find entries to add: diff old.txt new.txt | grep '^>' | sed 's/^> *//'.

On the other hand, the resulting list is gonna be really long (I get 631 entries without third­‑party modules).

Maybe it’s better to replace the list with a single regular expression?

Numbers

Is it obligatory to highlight numbers? What about domain2.com, 128k, 192.168.0.1:8000?

Strings

  1. Add EOLs. No escaping required. Moreover, escaping is impossible.
  2. Add escape sequences. Only the following escape sequences are available: \", \', \\, \n, \r, \t. I’ve checked all the other symbols from 0x00 to 0x7E inside both " and '. \xFF, \uFFFF, \u{FFFF} are not supported.
  3. Add interpolation ("$example" and "${example}"). As I previously mentioned, the dollar sign ($) can’t be escaped with \, so nginx interprets \$x as a slash (\) and a subsequent $x variable.

Variables

  1. Variable names can contain digits. Examples are $geoip_city_country_code3, $geoip_country_code3, $http2, $time_iso8601. See https://nginx.org/en/docs/varindex.html.
  2. Names of user­‑defined variables (set $name value;) can only consist of digits, uppercase and lowercase English letters and underscores. Position is irrelevant ($0, $_ are OK). /\$\w+/ is the best choice for a regular variable.
  3. Built­‑in variables with an underscore at the end ($arg_, $cookie_, $http_, $jwt_claim_, $jwt_header_, $sent_http_, $sent_trailer_, $upstream_cookie_, $upstream_http_, $upstream_trailer_) accept a trailing part. This part can contain any character except /[\x00-\x1F\s"';\\{]/. /[;{]/ are only allowed inside strings ("$arg_;", "$arg_{" are OK). Escaping with a slash (\) is useless.

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions