Skip to content

Fix terminal E315 issue (again)#16552

Closed
h-east wants to merge 1 commit intovim:masterfrom
h-east:fix-terminal-E315-issue
Closed

Fix terminal E315 issue (again)#16552
h-east wants to merge 1 commit intovim:masterfrom
h-east:fix-terminal-E315-issue

Conversation

@h-east
Copy link
Copy Markdown
Member

@h-east h-east commented Jan 31, 2025

Fix: #16024 #16211

@h-east
Copy link
Copy Markdown
Member Author

h-east commented Jan 31, 2025

I wrote a test, but since it doesn't fail in pre-patch Vim, so I didn't include it in the PR.

diff --git a/src/testdir/test_terminal2.vim b/src/testdir/test_terminal2.vim
index 9413905a9..f57cfb91e 100644
--- a/src/testdir/test_terminal2.vim
+++ b/src/testdir/test_terminal2.vim
@@ -289,6 +289,55 @@ func Test_termwinscroll_topline()
   set termwinscroll& mouse&
 endfunc
 
+func Test_termwinscroll_topline2()
+  set termwinscroll=100000 mouse=a
+  let norm_winid = win_getid()
+  terminal
+  call assert_equal(2, winnr('$'))
+  let buf = bufnr()
+  let win = winnr()
+  call WaitFor({-> !empty(term_getline(buf, 1))})
+
+  let num1 = &termwinscroll / 1000 * 999
+  call writefile(range(num1), 'Xtext', 'D')
+  if has('win32')
+    call term_sendkeys(buf, "type Xtext\<CR>")
+  else
+    call term_sendkeys(buf, "cat Xtext\<CR>")
+  endif
+  let rows = term_getsize(buf)[0]
+  " On MS-Windows there is an empty line, check both last line and above it.
+  call WaitForAssert({-> assert_match(string(num1 - 1), term_getline(buf, rows - 1) .. term_getline(buf, rows - 2))})
+  call feedkeys("\<C-W>N", 'xt')
+  call feedkeys("i", 'xt')
+
+  let num2 = &termwinscroll / 1000 * 8
+  call writefile(range(num2), 'Xtext', 'D')
+  if has('win32')
+    call term_sendkeys(buf, "timeout /t 2 && type Xtext\<CR>")
+  else
+    call term_sendkeys(buf, "sleep 2; cat Xtext\<CR>")
+  endif
+  let winrow = get(get(filter(getwininfo(), 'v:val.winid == norm_winid'), 0, {}), 'winrow', -1)
+
+  call test_setmouse(winrow, 1)
+  call feedkeys("\<LeftMouse>", "xt")
+  call WaitForAssert({-> assert_notequal(buf, bufnr())})
+
+  " Change the terminal window row size
+  call win_move_statusline(win,1)
+  " Before the fix, E340 and E315 would occur multiple times at this point.
+  let winrow2 = get(get(filter(getwininfo(), 'v:val.winid == norm_winid'), 0, {}), 'winrow', -1)
+  call assert_equal(winrow + 1, winrow2)
+
+  call test_setmouse(1, 1)
+  call feedkeys("\<LeftMouse>", "xt")
+  call WaitForAssert({-> assert_equal(buf, bufnr())})
+
+  exe buf . 'bwipe!'
+  set termwinscroll& mouse&
+endfunc
+
 " Resizing the terminal window caused an ml_get error.
 " TODO: This does not reproduce the original problem.
 " TODO: This test starts timing out in Github CI Gui test, why????

@h-east h-east changed the title Fix terinal E315 issue (again) Fix terminal E315 issue (again) Jan 31, 2025
@chrisbra
Copy link
Copy Markdown
Member

chrisbra commented Feb 1, 2025

thanks

@chrisbra
Copy link
Copy Markdown
Member

chrisbra commented Feb 1, 2025

I include the test nevertheless

@chrisbra chrisbra closed this in 3219da5 Feb 1, 2025
@h-east h-east deleted the fix-terminal-E315-issue branch February 1, 2025 14:21
@ychin
Copy link
Copy Markdown
Contributor

ychin commented Feb 8, 2025

Seems like the test is failing in CI for macOS (and for local machines too). I saw that @chrisbra added some fixes to try to fix macOS failures in 44a2135 (v9.1.1067) but it's still failing in CI and my local machine. Can we revert or turn the test off for now? Example where it's failing: https://github.com/vim/vim/actions/runs/13207075134. If we have ongoing test failures it's quite difficult to get good signal on whether new changes are introducing regressions.

Taking a brief look into the test, it seems like the term_wait call is not doing the job. It stalls the whole process rather than letting the terminal print the messages (the longer you set the timeout on it, the worse it gets because it's not actually letting the prints through). I'm not sure if it's a misuse of the API or a bug in it. I personally never really understood when it's a good time to use term_wait when I write terminal tests. But from local testing that seems to be the problematic function.

Just to be clear, this is not a CI flakiness issue (since it fails on local machine with 100% certainty). It's a bug in the test I think (unless it's some weird oddity in the terminal implementation).

@ychin
Copy link
Copy Markdown
Contributor

ychin commented Feb 8, 2025

Looking at this a little more, I think the term_wait() call is not doing what it's supposed to be doing. The timeout there is a wait that will stall Vim completely, and also stall the terminal as a result. It's really just a hammer that we don't really need to use here.

The reason why the tests is failing is that the call WaitForAssert({-> assert_match(string(num1 - 1), term_getline(buf, rows - 1))}) call seems to be significantly affecting the performance of the terminal (not sure if this would be considered a bug or just the way it works). In this case all we want is to be notified when the job is done printing a giant file. We could do something like printf '\033]51;["call", "terminal_print_complete"]\007' from within the terminal (see :h terminal-api) to have a proper way to be notified when it finishes printing instead. Alternatively, I think it would be nice if we have a better API that notifies us that the terminal has finished a job (you can check that by seeing if the terminal has a child process that's running or whether it's just the shell). I think this API is useful outside of testing as well.

@chrisbra
Copy link
Copy Markdown
Member

chrisbra commented Feb 8, 2025

Yes Thanks that is a good idea

chrisbra added a commit to chrisbra/vim that referenced this pull request Feb 8, 2025
call a terminal callback function when finished printing, instead of
using term_wait(). This causes some timeouts on CI with the macos
runners

related: vim#16552

Signed-off-by: Christian Brabandt <cb@256bit.org>
@chrisbra
Copy link
Copy Markdown
Member

chrisbra commented Feb 8, 2025

To add a proper API would also be nice, but I don't currently know how to do it. So let me make the terminal-api related adjustements.

chrisbra added a commit to chrisbra/vim that referenced this pull request Feb 8, 2025
call a terminal callback function when finished printing, instead of
using term_wait(). This causes some timeouts on CI with the macos
runners

related: vim#16552

Signed-off-by: Christian Brabandt <cb@256bit.org>
chrisbra added a commit to chrisbra/vim that referenced this pull request Feb 8, 2025
call a terminal callback function when finished printing, instead of
using term_wait(). This causes some timeouts on CI with the macos
runners

related: vim#16552

Signed-off-by: Christian Brabandt <cb@256bit.org>
@ychin
Copy link
Copy Markdown
Contributor

ychin commented Feb 8, 2025

Right. I tested out your WIP commit @chrisbra and it does fix the issue. However, the test takes a while to finish on my machine (10s).

I think if you replace the sleep 1 with call term_wait(buf) (note that there is no timeout here) it will make the test go a lot faster (2.8s on my machine). If you look at the implementation of do_sleep, it does call parse_queued_messages() that will pump the loop, but it's annoying here because Vim will wait a little bit before it pumps it again. What we want here is a spin loop that will continually pump the loop without delay, which is what term_wait(buf) will do.

This still works slower than just a normal Vim running instance running a terminal printing the file out though. If I just do cat Xtext in the terminal it takes <1 s to do so. I'm not sure where the inefficiency here. We don't really have a way to write asynchronous tests in Vim so we have to find a way to pump the event loop and maybe term_wait(buf) is still not as efficient as the normal main loop, but at least it gets us closer to a reasonable performance for what should be a simple test.

chrisbra added a commit to chrisbra/vim that referenced this pull request Feb 9, 2025
call a terminal callback function when finished printing, instead of
using term_wait(). This causes some timeouts on CI with the macos
runners

related: vim#16552

Signed-off-by: Christian Brabandt <cb@256bit.org>
chrisbra added a commit to chrisbra/vim that referenced this pull request Feb 9, 2025
call a terminal callback function when finished printing, instead of
using term_wait(). This causes some timeouts on CI with the macos
runners

related: vim#16552

Signed-off-by: Christian Brabandt <cb@256bit.org>

wait longer

Signed-off-by: Christian Brabandt <cb@256bit.org>

don't call terminal api on windows
chrisbra added a commit to chrisbra/vim that referenced this pull request Feb 9, 2025
call a terminal callback function when finished printing, instead of
using term_wait(). This causes some timeouts on CI with the macos
runners

related: vim#16552

Signed-off-by: Christian Brabandt <cb@256bit.org>

wait longer

Signed-off-by: Christian Brabandt <cb@256bit.org>

don't call terminal api on windows
chrisbra added a commit that referenced this pull request Feb 9, 2025
Problem:  tests: test_terminwscroll_topline2 unreliable
          (Yee Cheng Chin)
Solution: instead of using term_wait() with a specific time, use
          terminal-api and to wait until the terminal is finished

call a terminal callback function when finished printing, instead of
using term_wait(), with a defined time, which caused timeouts on CI
with the macos runners

Unfortunately I couldn't figure out how to call the terminal-api on Windows,
so skip the test on Windows. cmd.com echo didn's seem to work and
neither did trying to use python, but perhaps it was just me fighting
with the terminal quoting rules 🤷

related: #16599
related: #16552

Signed-off-by: Christian Brabandt <cb@256bit.org>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

E315 ml_get:invalid_lnum: [number] in terminal buffer [again]

3 participants