Skip to content

Expose MuPDF's search function#2279

Merged
Frenzie merged 8 commits into
koreader:masterfrom
Commodore64user:mupdf-search
Feb 19, 2026
Merged

Expose MuPDF's search function#2279
Frenzie merged 8 commits into
koreader:masterfrom
Commodore64user:mupdf-search

Conversation

@Commodore64user

@Commodore64user Commodore64user commented Feb 19, 2026

Copy link
Copy Markdown
Member

what's new

I was originally trying to do this in Lua but it turns out this function was there all along for anyone to C (sorry!)

massively simplifies koreader/koreader#15001

  • Added a C wrapper function mupdf_fz_search_stext_page that exposes the MuPDF fz_search_stext_page API for searching text on a page.
  • Registered the new wrapper function with the FFI cdecl system to make it available to Lua.
  • Declared the new function in the FFI Lua header for use in Lua code.
  • Added a search method to the page_mt metatable in mupdf.lua, which uses the new C function to find all matches for a given string on a page and returns their bounding box coordinates.
my effort ;(
function HtmlBoxWidget:_findMatchesOnPage(page_idx, search_text)
  local page = self.document:openPage(page_idx)
  if not page then return nil end

  local lines = page:getPageText()
  page:close()

  if not lines or type(lines) ~= "table" then return nil end

  -- Build searchable content with position mapping
  local content_parts = {}
  local word_map = {}
  local current_pos = 0

  for _, line in ipairs(lines) do
      if type(line) == "table" then
          for _, word_obj in ipairs(line) do
              if word_obj and word_obj.word then
                  -- Note: We lowercase words BEFORE building the position map because
                  --       case conversion can change UTF-8 byte lengths (e.g., Turkish İ→i: 2 bytes→1 byte).
                  --       If we lowercased after, our byte positions would be misaligned with the search string.
                  -- e.g.,
                  -- print("Original:", test, "Bytes:", #test)    -->    Original:     İİİİİ hello world       Bytes:     22
                  -- print("Lowered:", lower, "Bytes:", #lower)   -->    Lowered:      iiiii hello world       Bytes:     17
                  local word_text = Utf8Proc.lowercase(word_obj.word, false)
                  local start_pos = current_pos + 1

                  table.insert(content_parts, word_text)
                  table.insert(content_parts, " ")
                  current_pos = current_pos + #word_text + 1 -- +1 for space

                  table.insert(word_map, {
                      start_pos = start_pos,
                      end_pos = start_pos + #word_text - 1,
                      box = word_obj,
                      line = line
                  })
              end
          end
      end
  end
  local content = table.concat(content_parts)

  -- Search in the already-lowercased content
  local search_lower = Utf8Proc.lowercase(search_text, false)
  local match_rects = {}
  local search_start = 1
  local word_index = 1     -- Track position in word_map
  local last_word_idx = -1 -- Track last added word to prevent redundant rects

  -- Algorithm is O(n+m), n: number of words, m: matches. It makes a single forward pass
  -- through the word array, collecting rectangles for m matches along the way.
  while true do
      local match_start, match_end = content:find(search_lower, search_start, true)
      if not match_start then break end

      -- Skip words that end before this match starts, word_index only advances forward.
      while word_map[word_index] and word_map[word_index].end_pos < match_start do
          word_index = word_index + 1
      end

      -- Collect all words that overlap with this match
      local i = word_index
      while word_map[i] and word_map[i].start_pos <= match_end do
          local word_mapping = word_map[i]
          local overlap_start = math.max(word_mapping.start_pos, match_start)
          local overlap_end = math.min(word_mapping.end_pos, match_end)

          -- Ensure we only add the rect if there is an overlap and we haven't added this word yet
          if overlap_start <= overlap_end and i > last_word_idx then
              local rect = Geom:new{
                  x = word_mapping.box.x0,
                  y = word_mapping.line.y0,
                  w = word_mapping.box.x1 - word_mapping.box.x0,
                  h = word_mapping.line.y1 - word_mapping.line.y0,
              }
              table.insert(match_rects, rect)
              last_word_idx = i
          end
          i = i + 1
      end -- while word overlap
      -- Note: Moving to match_end + 1 avoids re-processing the same match, but
      --       it will skip overlapping results (e.g., searching "aba" in "ababa"
      --       will only return the first instance).
      search_start = match_end + 1
  end -- while true
  return #match_rects > 0 and match_rects or nil
end

https://github.com/ArtifexSoftware/mupdf/blob/8c39e04a09c74fcc96ebb704bf6d1f59adb9b6fa/source/fitz/stext-search.c#L1254-L1285


This change is Reviewable

@Frenzie

Frenzie commented Feb 19, 2026

Copy link
Copy Markdown
Member

Minor sidenote (though one can't imagine it changing that much):

/**
	Search for occurrence of 'needle' in text page.
	Case insensitive match.

	Return the number of quads and store hit quads in the passed in
	array.

	NOTE: This is an experimental interface and subject to change
	without notice.
*/

https://github.com/ArtifexSoftware/mupdf/blob/53d140db017352190eae56b2c5a71d93494dd226/include/mupdf/fitz/structured-text.h#L636C1-L646

Comment thread ffi-cdecl/wrap-mupdf_cdecl.c Outdated
cdecl_func(mupdf_fz_bound_page)
cdecl_func(fz_drop_page)

cdecl_func(mupdf_fz_search_stext_page);

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's keep it in the structured text block a bit further down.

Comment thread ffi/mupdf.lua Outdated
--[[
Get a list of matches for the given text on the page, with their coordinates.
--]]
function page_mt.__index:search(needle, hit_max)

@Frenzie Frenzie Feb 19, 2026

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd go for something like searchPageText, both to fit in with getPageText right above and to be clear that it's not searching through the entire document.

Comment thread ffi/mupdf_h.lua Outdated
fz_alloc_context *mupdf_get_my_alloc_context();
int mupdf_get_cache_size();
int mupdf_error_code(fz_context *);
int mupdf_fz_search_stext_page(fz_context *ctx, fz_stext_page *text, const char *needle, int *hit_mark, fz_quad *hit_bbox, int hit_max);

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The CI was complaining about this location not matching with the cdefs, but mind like I said I'd like to improve the location in the cdefs. ^_^

Comment thread ffi/mupdf.lua Outdated
Comment on lines +614 to +616
local hit_mark = ffi.new("int[?]", hit_max) -- allocate the hit_mark array

local count = W.mupdf_fz_search_stext_page(ctx, text_page, needle, hit_mark, hits, hit_max)

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You should be able to just pass nil: no point in allocating hit_mark if it's not going to be used.

Comment thread wrap-mupdf.c Outdated
Comment on lines +158 to +160
int mupdf_fz_search_stext_page(fz_context *ctx, fz_stext_page *text, const char *needle, int *hit_mark, fz_quad *hit_bbox, int hit_max) {
return fz_search_stext_page(ctx, text, needle, hit_mark, hit_bbox, hit_max);
}

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The whole point of those wrappers is sandwitching those MuPDF calls between fz_try(ctx) / fz_catch(ctx).

So change the declaration in wrap-mupdf.h to use:

MUPDF_WRAP(mupdf_search_stext_page, int, -1, 
    ret = fz_search_stext_page(ctx, text, needle, hit_mark, hit_bbox, hit_max),
    fz_stext_page *text, const char *needle, int *hit_mark, fz_quad *hit_bbox, int hit_max)

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Which means completely dropping this change.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I see

@Commodore64user

Copy link
Copy Markdown
Member Author

hoping I didn't miss anything, excuse my clumsiness around here... unknown surroundings ;)

Comment thread ffi-cdecl/wrap-mupdf_cdecl.c Outdated
Comment thread ffi/mupdf.lua Outdated
Comment thread ffi/mupdf_h.lua Outdated
Comment thread wrap-mupdf.c Outdated
Comment on lines +158 to +160
int mupdf_fz_search_stext_page(fz_context *ctx, fz_stext_page *text, const char *needle, int *hit_mark, fz_quad *hit_bbox, int hit_max) {
return fz_search_stext_page(ctx, text, needle, hit_mark, hit_bbox, hit_max);
}

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Which means completely dropping this change.

Comment thread wrap-mupdf.h Outdated
Co-authored-by: Benoit Pierre <benoit.pierre@gmail.com>
Comment thread wrap-mupdf.c Outdated
Commodore64user and others added 2 commits February 19, 2026 13:44
Co-authored-by: Benoit Pierre <benoit.pierre@gmail.com>
Comment thread ffi/mupdf_h.lua Outdated
Comment thread wrap-mupdf.c Outdated
@Frenzie Frenzie changed the title expose MuPDF's search function Expose MuPDF's search function Feb 19, 2026
@Frenzie Frenzie merged commit 7396846 into koreader:master Feb 19, 2026
3 checks passed
Commodore64user added a commit to Commodore64user/KOReader_fork that referenced this pull request Feb 19, 2026
@Commodore64user Commodore64user deleted the mupdf-search branch February 19, 2026 17:47
Commodore64user added a commit to Commodore64user/KOReader_fork that referenced this pull request Feb 24, 2026
benoit-pierre pushed a commit to benoit-pierre/koreader that referenced this pull request Feb 26, 2026
benoit-pierre added a commit to benoit-pierre/koreader that referenced this pull request Feb 28, 2026
- zlib: update to 1.3.2 (koreader/koreader-base#2278)
- Expose MuPDF's search function (koreader/koreader-base#2279)
- mupdf: add getMarkupAnnotationBoxesFromPage() (koreader/koreader-base#2280)
- luajit: update to 2.1.1772148810 (koreader/koreader-base#2281)
Frenzie pushed a commit to koreader/koreader that referenced this pull request Feb 28, 2026
- zlib: update to 1.3.2 (koreader/koreader-base#2278)
- Expose MuPDF's search function (koreader/koreader-base#2279)
- mupdf: add getMarkupAnnotationBoxesFromPage() (koreader/koreader-base#2280)
- luajit: update to 2.1.1772148810 (koreader/koreader-base#2281)
0xstillb pushed a commit to 0xstillb/koreader-thai that referenced this pull request May 9, 2026
- zlib: update to 1.3.2 (koreader/koreader-base#2278)
- Expose MuPDF's search function (koreader/koreader-base#2279)
- mupdf: add getMarkupAnnotationBoxesFromPage() (koreader/koreader-base#2280)
- luajit: update to 2.1.1772148810 (koreader/koreader-base#2281)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants