Did you guess the reason for this happening ?
Is it (my guess) happening when we change HTML nodes (in dicts, when getting bits in bold or italic) in that line ?

Did you guess the reason for this happening ?

The lines are added in the fz_add_stext_char_imp function. The function gets the whole span as one string but it splits the string because the gap between the words is larger than the limit. This is this comment there:

Motion is large and unexpected (probably a new table column).

it seems this problem hasn't completely gone away. if there is extreme justification, still happens

@TnS-hun I have been trying to find a wiktionary entry that shows the same behaviour but much to my chagrin I've had no success so, here is a one entry stardict (just connect)

[Dict.zip]

Thanks for the dictionary, I could reproduce the problem with it.

It happens because the way the highlighting works in the HtmlBoxWidget. It works on line by line basis (to be able to draw properly connected rectangles), so when page_mt.__index:getPageText() does not return correct lines highlighting does not work correctly either. HtmlBoxWidget:updateHighlight() tries to fix these incorrect lines but in this case the Y-s do not match, so it cannot.

17 and 18 are seperate lines here but they should be a single one:

The solution could be either using MuPDF for getting the rectangles for highlighting, or getting more info from MuPDF to be able to work on non line by line basis.

Using MuPDF for getting the rectangles sounds the most logical to me (barring something making that difficult).

@poire-z should voice his opinions.

so when page_mt.__index:getPageText() does not return correct lines highlighting does not work correctly either

Isn't that already using MuPDF output, and doing what your suggestion would be doing ?:

The solution could be either using MuPDF for getting the rectangles for highlighting

No real opinion :/
Was about to say : given that MuPDF is unstructured and boxes could be all over the place by design, and it would be a nightmare trying to connect the unconnectable :)

But this is HTML input, so the output should still be linear/connected boxes.
May be it is due to the variations in font size ?:

Dunno if that can be solved by some CSS tweak to force a fixed line-height or font-size on all the bits that make a line.
Or the connection of boxes should be lose (instead of matching y, ensure y0-y1 overlap with the prev boxes y0-y1 ?

Not the difference in font size, other entries work fine. Is something to do with the level of word spacing needed for the justification to happen. After it becomes too large it seems to break the line. Speaking of justification, why does it add space before the first word in a given line? I'll post an example later so is clearer what i mean.

Turns out the clip has an example already, all those are following the same css/html rules

Edit: turning justification off fixes both issues.

Dunno, MuPDF's HTML rendering isn't really its core function, so there may be limitations/deficiencies.
It probably hasn't gotten the amount of time and tweaking I've put into crengine's rendering and typography.

There, it looks like it consideres the leading bullet like a word for justification, and adjust its space like any other one in the line.
With crengine and list bullets, I think they are not considered as part of the line and drawn outside.
And for — or « used in dialogs (and some other leading symbols), that are part of the main text lines, in crengine I added specific code to lock the spacing following it, so it's not considered in justification, ie. fixing this yellow issue:

It probably hasn't gotten the amount of time and tweaking I've put into crengine's rendering and typography.

that's pretty cool, thanks for doing that. Pedants like us can read in peace. ;)

why couldn't the dictionary widget use crengine for html dicts as well? why do we use MuPDF?

It's much simpler to massage it into this kind of little thing.

Yes, MuPDF is much simpler for quick rendering of small HTML snippets to a bitmap, that's why we use it for HtmlBoxWidget.
As for crengine, there is a limitation in that it can only deal with one document at a time (it's supposed to be able to have multiple documents documents opened at the same time, but there are quite a few global rendering options, that do apply to all opened documents - and if you set one different for that HTML snippet rendering, it would cause the book you're reading to get some invalide hash mismatch and cause a re-rendering, etc...)
So, simpler, less work, less risky to use MuPDF.

Isn't that already using MuPDF output, and doing what your suggestion would be doing ?:

MuPDF has more info available about the lines, so it has some advantage.

I have a working version that uses MuPDF to get the rectangles but it does not look that great:

Should I make a pull request?

I have a working version that uses MuPDF to get the rectangles but it does not look that great:

How did it look before ?

Is it our code that is connecting (or not) the word boxes on a line? Or is it MuPDF?

The MuPDF structured text API also has a number of options we don't use.

How did it look before ?

Not too much but somewhat better:

Yes, the old one feels/looks better - and I don't really see how the new one (still having holes, but less, and also boxes of different heights) would help with fixing the issue ?

Yes, the old one feels/looks better - and I don't really see how the new one (still having holes, but less, and also boxes of different heights) would help with fixing the issue ?

I don’t know either but I'd take a properly working one with slightly uglier block selection to a nicer looking one that only sometimes works. Besides, nobody is selecting large chunks of text in the widget, there is no point. It can only do wiki/dict searches.

Using MuPDF helps with the issue because it does not try use lines for drawing, so the rectangles are correct.

I looked a bit into the structured text that MuPDF can provide and it has better Y coordinates, so it might be possible to get rid of the holes too.

[BLOCK START: 2.14748e+09, 2.14748e+09, -2.14748e+09, -2.14748e+09]
  [BLOCK START: 2.14748e+09, 2.14748e+09, -2.14748e+09, -2.14748e+09]
    [BLOCK START: 2.14748e+09, 2.14748e+09, -2.14748e+09, -2.14748e+09]
      [BLOCK START: 2.14748e+09, 2.14748e+09, -2.14748e+09, -2.14748e+09]
        [BLOCK START: 21.6, -2.14199, 491, 349.974]
         [LINE START: 32.4, -2.14199, 491, 22.374]to the other: the bus connects with trains from Union[LINE END]
         [LINE START: 32.4, 21.258, 93.474, 45.774]Station.[LINE END]
         [LINE START: 21.6, 44.658, 491, 69.174]" associate or relate in some respect: employees are[LINE END]
         [LINE START: 32.4, 68.058, 491, 92.574]rewarded with bonuses connected to their firm's[LINE END]
         [LINE START: 32.4, 91.458, 491, 115.974]performance; a variety of physical complaints connected[LINE END]
         [LINE START: 32.4, 114.858, 124.56, 139.374]with stress.[LINE END]
         [LINE START: 21.6, 138.258, 491, 162.774]" think of as being linked or related: I didn't connect the[LINE END]
         [LINE START: 32.4, 161.658, 233.226, 186.174]two incidents at the time.[LINE END]
         [LINE START: 21.6, 185.058, 491, 209.574]" (of a thing) provide or have a link or relationship with[LINE END]
         [LINE START: 32.4, 208.458, 491, 232.974](someone or something): there was no evidence to[LINE END]
         [LINE START: 32.4, 231.858, 248.418, 256.374]connect Jeff with the theft.[LINE END]
         [LINE START: 21.6, 255.258, 491, 279.774]" [intrans.] form a relationship or feel an affinity: I[LINE END]
         [LINE START: 32.4, 278.658, 491, 303.174]taught in a reading program and I connected with kids[LINE END]
         [LINE START: 32.4, 302.058, 128.232, 326.574]individually.[LINE END]
         [LINE START: 21.6, 325.458, 491, 349.974]" [intrans.] INFORMAL (of a blow) hit the intended target:[LINE END]
        [BLOCK END]
        [BLOCK START: 32.4, 348.858, 403.362, 373.374]
         [LINE START: 32.4, 348.858, 403.362, 373.374]the blow connected and he felt a burst of pain.[LINE END]
        [BLOCK END]
      [BLOCK END]
    [BLOCK END]
    [BLOCK START: 0, 372.258, 491, 420.174]
     [LINE START: 0, 372.258, 98.5752, 396.774]<DERIVATIVES> [LINE END]
     [LINE START: 114.853, 372.258, 245.065, 396.774]connectable [LINE END]
     [LINE START: 261.344, 372.258, 295.922, 396.774]adj. [LINE END]
     [LINE START: 312.2, 372.258, 441.782, 396.774]connectedly [LINE END]
     [LINE START: 458.06, 372.258, 491, 396.774]adv.[LINE END]
     [LINE START: 0, 395.658, 169.236, 420.174]connectedness n.[LINE END]
    [BLOCK END]
    [BLOCK START: 8.136, 434.103, 482.864, 453.716]
     [LINE START: 8.136, 434.103, 482.864, 453.716]<ORIGIN> late Middle English (in the sense 'be united physically'; rare[LINE END]
    [BLOCK END]
  [BLOCK END]

Wondering if you were able to figure something out @TnS-hun

Honestly, I did not look into how MuPDF's highlighting could be improved. I think we could just use MuPDF as it is. Multiword selection is rare in the dictionary to the noncontinuous highlighting to bother anyone.

Honestly, I did not look into how MuPDF's highlighting could be improved. I think we could just use MuPDF as it is. Multiword selection is rare in the dictionary to the noncontinuous highlighting to bother anyone.

@poire-z ? (I left that ridiculous space before the question mark just for you)

Thanks for that space, but what is the question ?

We were trying to solve #13276 (comment) old man 👴 ;)

Oh, you were ! Good luck to you all :) (I'm not familiar with mupdf stuff to help)