Skip to content

fix(treesitter): remove default match limit#39696

Merged
clason merged 1 commit into
neovim:masterfrom
WillLillis:match_limit_bump
May 9, 2026
Merged

fix(treesitter): remove default match limit#39696
clason merged 1 commit into
neovim:masterfrom
WillLillis:match_limit_bump

Conversation

@WillLillis

@WillLillis WillLillis commented May 9, 2026

Copy link
Copy Markdown
Contributor

Problem

Iterating over query captures previously had catastrophic performance cliffs, leading to unacceptable pauses, lag, etc. in editor.

Solution

In tree-sitter/tree-sitter#5548, capture iteration in tree-sitter was greatly improved. Neovim bumped its tree-sitter dependency to include this change in #39442.

The Numbers

I did some basic testing locally with captures from the tree-sitter-zig grammar, using the reproducer file from the original tree-sitter issue.

VIMRUNTIME=/home/lillis/projects/neovim/runtime \
    timeout 1800 build/bin/nvim --clean --headless \
    -c "luafile bench_zig.lua" -c "qa!"
bench_zig.lua
local FILE = vim.env.NVIM_BENCH_FILE
  or '/home/lillis/projects/grammars/tree-sitter-zig/foo.zig'
local PARSER = vim.env.NVIM_BENCH_PARSER
  or '/home/lillis/projects/grammars/tree-sitter-zig/zig.so'
local QUERY_FILE = vim.env.NVIM_BENCH_QUERY
  or '/home/lillis/projects/grammars/tree-sitter-zig/queries/highlights.scm'
local REPS = tonumber(vim.env.NVIM_BENCH_REPS) or 3

vim.treesitter.language.add('zig', { path = PARSER })

local fd = assert(io.open(FILE, 'r'))
local source = fd:read('*a')
fd:close()
local fd2 = assert(io.open(QUERY_FILE, 'r'))
local query_text = fd2:read('*a')
fd2:close()

local lines = vim.split(source, '\n', { plain = true })
local buf = vim.api.nvim_create_buf(false, true)
vim.api.nvim_buf_set_lines(buf, 0, -1, false, lines)
vim.bo[buf].filetype = 'zig'

local function ns_to_ms(ns) return ns / 1e6 end

local parser = vim.treesitter.get_parser(buf, 'zig')
parser:invalidate(true)
local t0 = vim.uv.hrtime()
parser:parse(true)
local parse_ms = ns_to_ms(vim.uv.hrtime() - t0)
local tree = parser:trees()[1]

local query = vim.treesitter.query.parse('zig', query_text)

local function stats(times)
  table.sort(times)
  local sum = 0
  for _, v in ipairs(times) do sum = sum + v end
  return {
    mean = sum / #times,
    median = times[math.floor(#times / 2) + 1],
    min = times[1],
    max = times[#times],
  }
end

local function bench(label, fn)
  local n = fn()  -- warm
  local times = {}
  for i = 1, REPS do
    local t = vim.uv.hrtime()
    fn()
    times[i] = ns_to_ms(vim.uv.hrtime() - t)
  end
  local s = stats(times)
  io.stdout:write(string.format(
    '%-22s n=%-7d mean=%9.2f ms  median=%9.2f ms  min=%9.2f ms  max=%9.2f ms\n',
    label, n, s.mean, s.median, s.min, s.max))
end

io.stdout:write(string.format('file=%s lines=%d size=%d parse_ms=%.3f reps=%d\n',
  FILE, #lines, #source, parse_ms, REPS))

bench('zig iter_captures', function()
  local n = 0
  for _ in query:iter_captures(tree:root(), buf, 0, -1) do n = n + 1 end
  return n
end)

bench('zig iter_matches', function()
  local n = 0
  for _ in query:iter_matches(tree:root(), buf, 0, -1) do n = n + 1 end
  return n
end)

Without the perf improvement in tree-sitter:

  • iter_captures: 21,242ms (I'm assuming this is the catastrophic slowdown that necessitated such a low match_limit)
  • iter_matches: 44ms

With the perf improvement in tree-sitter:

  • iter_captures: 193ms
  • iter_matches: 50ms

Given this speedup, it seems like raising the match limit is worth a try.

Fixes #26325

AI DISCLAIMER: I used AI to figure out how to conduct this benchmark. Any issues with it are due to my own ignorance of Neovim internals.

Comment thread runtime/lua/vim/treesitter/query.lua Outdated
Recent updates to tree-sitter improved the iteration speed of captures
that previously led to catastrophic performance cliffs. This limit
should (hopefully) no longer be necessary.
@WillLillis

Copy link
Copy Markdown
Contributor Author

Also, this issue looks related: #26325

@clason

clason commented May 9, 2026

Copy link
Copy Markdown
Member

Oh, yeah, meant to link that!

@clason clason changed the title fix(tree-sitter): change default match limit to u32::MAX fix(treesitter): remove default match limit May 9, 2026
@clason clason enabled auto-merge (squash) May 9, 2026 10:09
@clason clason merged commit b44c2bd into neovim:master May 9, 2026
65 of 66 checks passed
@WillLillis WillLillis deleted the match_limit_bump branch May 9, 2026 10:27
@lewis6991

Copy link
Copy Markdown
Member

This PR seems to have exposed that some queries can create a combinatorial explosion of matches. The old default match_limit was masking those cases by truncating query exploration.

The issue I specifically hit was with a dodgy python injection query which I've fixed here, but it's likely the match limit was probably masking similar queries in other languages.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Treesitter: ts_match_limit of 256 is not always enough

3 participants