Skip to content

Fixes linux wheel backspace by upgrading ncurses-6.2 to 6.4#2484

Merged
pramodk merged 5 commits into
masterfrom
hines/backspace
Sep 1, 2023
Merged

Fixes linux wheel backspace by upgrading ncurses-6.2 to 6.4#2484
pramodk merged 5 commits into
masterfrom
hines/backspace

Conversation

@nrnhines

Copy link
Copy Markdown
Member

Closes #2399

When new docker image is published, 8.2.2 linux wheel needs to be recreated and become what users get with pip install neuron

@nrnhines nrnhines requested review from pramodk and ramcdougal August 31, 2023 17:25
Comment thread packaging/python/Dockerfile Outdated
@azure-pipelines

Copy link
Copy Markdown

✔️ 99ea46c -> Azure artifacts URL

@bbpbuildbot

This comment has been minimized.

@nrnhines nrnhines changed the title Fixes linux wheel backspace by regressing to ncurses-5.9 Fixes linux wheel backspace by upgrading ncurses-6.2 to 6.4 Aug 31, 2023
@azure-pipelines

Copy link
Copy Markdown

✔️ 7b542f7 -> Azure artifacts URL

@codecov

codecov Bot commented Aug 31, 2023

Copy link
Copy Markdown

Codecov Report

Merging #2484 (7b542f7) into master (d3b7e7b) will decrease coverage by 0.08%.
The diff coverage is n/a.

❗ Current head 7b542f7 differs from pull request most recent head ae688ab. Consider uploading reports for the commit ae688ab to get more accurate results

@@            Coverage Diff             @@
##           master    #2484      +/-   ##
==========================================
- Coverage   61.16%   61.08%   -0.08%     
==========================================
  Files         630      627       -3     
  Lines      121443   121064     -379     
==========================================
- Hits        74276    73950     -326     
+ Misses      47167    47114      -53     

see 13 files with indirect coverage changes

📣 We’re building smart automated test selection to slash your CI/CD build times. Learn more

@bbpbuildbot

This comment has been minimized.

@ramcdougal

Copy link
Copy Markdown
Member

To fix #2399, we also need to make it work on mac. (Not sure if it works on x86_64, but it definitely doesn't work on M2).

@nrnhines

nrnhines commented Sep 1, 2023

Copy link
Copy Markdown
Member Author

To fix #2399, we also need to make it work on mac. (Not sure if it works on x86_64, but it definitely doesn't work on M2).

What does not work on M2? The use of ncurses-6.4? Or the existing wheel for 8.2.2 with, presumably, ncurses-6.2. I'm trying to build a wheel on my apple M1 but have not succeeded yet.

@nrnhines

nrnhines commented Sep 1, 2023

Copy link
Copy Markdown
Member Author

The reason I though it was unique to linux wheels is that I don't see the backspace problem on my M1 with 8.2.2.

@ramcdougal

Copy link
Copy Markdown
Member

The wheel for 8.2.2 with ncurses 6.2 does not work on a Mac.

If everything works with ncurses 6.4 that seems better than reverting to an old version.

@ramcdougal

Copy link
Copy Markdown
Member

The reason I though it was unique to linux wheels is that I don't see the backspace problem on my M1 with 8.2.2.

That's fascinating, because I have the issue on my M2, and Pramod's #2399 shows the issue on an x86_64 mac.

@nrnhines

nrnhines commented Sep 1, 2023

Copy link
Copy Markdown
Member Author

Now I see the issue on my M1. Perhaps I had the wrong environment.

@nrnhines

nrnhines commented Sep 1, 2023

Copy link
Copy Markdown
Member Author

I'm still having trouble building a wheel on the M1, though now I'm getting almost to the end...

 => LINKING shared library ./libnrnmech.dylib
 => LINKING executable ./special LDFLAGS are:       
Successfully created arm64/special
dyld[29515]: symbol not found in flat namespace '_BC'
/Users/hines/neuron/backspace/build/cmake_install/bin/neurondemo: line 34: 29515 Abort trap: 6           ${NRNBIN}nrniv -dll "${NRNDEMO}release/${MODSUBDIR}/.libs/libnrnmech.so" "${NRNDEMO}demo.hoc" "$@" -

I'm trying with

bash packaging/python/build_wheels.bash osx 3.11

@pramodk

pramodk commented Sep 1, 2023

Copy link
Copy Markdown
Member

@nrnhines : Sorry, it was my ignorance! I see that we also use static readline + ncurses 6.2 for Mac OS: https://github.com/neuronsimulator/nrn/blob/master/packaging/python/build_static_readline_osx.bash.

Will get that updated soon!

@nrnhines

nrnhines commented Sep 1, 2023

Copy link
Copy Markdown
Member Author

I figured out that _BC is undefined in libreadline :)

@nrnhines

nrnhines commented Sep 1, 2023

Copy link
Copy Markdown
Member Author

@pramodk Great! With your hint, I successfully built an osx wheel for my M1 using ncurses-6.4 and it does work.

Question: I presume this PR fixes the issue on osx for the next neuron-nightly. But the linux issue does not get fixed until a new Docker is published. Then our normal CI is used to build the linux neuron-nightly. I guess I'm a bit shaky on how to get all this put back into 8.2.2 (or 8.2.3?)

Comment thread azure-pipelines.yml
@pramodk

pramodk commented Sep 1, 2023

Copy link
Copy Markdown
Member

But the linux issue does not get fixed until a new Docker is published.

I have pushed the updated image to DockerHub. So that should be also good!

I successfully built an osx wheel for my M1 using ncurses-6.4 and it does work.

Tested the same on x86 Mac and it works for me as well.

I guess I'm a bit shaky on how to get all this put back into 8.2.2 (or 8.2.3?)

I guess we should be able to rebuild older releases as you hinted. Need to think a bit about what's the "easier" way to do this. But I guess it's not urgent and can happen in the coming week(s)?

@pramodk pramodk left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM - once we test the wheels generated from this PR build then we should be good to merge this.

@nrnhines

nrnhines commented Sep 1, 2023

Copy link
Copy Markdown
Member Author

Ted wrote

Strange stuff also happens under Win 10.

Start bash, launch nrngui and start executing statements, and soon the cursor disappears, and cursor images randomly "stick" to lines that have already been executed. Backspace seems to work properly.

Instead, just open an MSWin command prompt, execute nrngui, and start executing statements. Similar cursor problems appear, and backspace doesn't work correctly.

I notice in ci/win_install_deps.cmd the line

mingw-w64-x86_64-ncurses ^

I wonder if 8.2.2 was getting ncurses-6.2? (And how could we know?)

@pramodk

pramodk commented Sep 1, 2023

Copy link
Copy Markdown
Member

I wonder if 8.2.2 was getting ncurses-6.2? (And how could we know?)

Do we have this "weird" behaviour only with 8.2 on Windows and older version is working fine?

@ramcdougal

ramcdougal commented Sep 1, 2023

Copy link
Copy Markdown
Member

I guess I'm a bit shaky on how to get all this put back into 8.2.2 (or 8.2.3?)

Needs to be 8.2.3. The installation has changed. Features are different (e.g. backspace exists). Bugfixes are what minor versions are for. Silent updates don't get fixed on centrally managed systems and cause confusion. (e.g., "This works on my machine but not yours, and we're all using exactly the same software.")

But I guess it's not urgent and can happen in the coming week(s)?

HOC users would disagree. I have one specific person in mind.

@azure-pipelines

Copy link
Copy Markdown

✔️ ae688ab -> Azure artifacts URL

@pramodk

pramodk commented Sep 1, 2023

Copy link
Copy Markdown
Member
330/358 Test #126: coreneuron_modtests::spikes_mpi_file_mode_py_gpu .........................***Failed  125.20 sec
srun: error: Unable to confirm allocation for job 1699382: Socket timed out on send/recv operation
srun: Check SLURM_JOB_ID environment variable. Expired or invalid job 1699382

one specific test that failed for GPU is unrelated, cluster error. So I am going to ignore it.

I downloaded the wheels produced from CI and they worked on Mac OS x86_64 and Linux. So I will merge this PR.

@pramodk

pramodk commented Sep 1, 2023

Copy link
Copy Markdown
Member

I wonder if 8.2.2 was getting ncurses-6.2? (And how could we know?)

@nrnhines : for windows, looking at the GitHub build log here, following versions of readline & ncurses are used today:

2023-09-01T15:25:29.7678358Z  mingw-w64-x86_64-readline-8.2.001-6-any downloading...
2023-09-01T15:25:29.7670430Z  mingw-w64-x86_64-ncurses-6.4.20230708-1-any downloading...

So at least ncurses/readline versions are now newer.

At the time of 8.2, difficult to know now. But considering pacman packages are relatively newer (?), probably there were no old versions.

Does the nightly windows installer work fine?

Like Mac/Linux, we might need to debug the windows issues separately. So I will go ahead and merge this.

@pramodk pramodk merged commit 138282b into master Sep 1, 2023
@pramodk pramodk deleted the hines/backspace branch September 1, 2023 21:09
nrnhines added a commit that referenced this pull request Sep 2, 2023
* Change MacOS wheel building with ncurses 6.4 as well
* Secure file on AzureCI is updated
* Update the documentation for better clarity

Co-authored-by: Pramod S Kumbhar <pramod.s.kumbhar@gmail.com>
nrnhines added a commit that referenced this pull request Sep 6, 2023
* Change MacOS wheel building with ncurses 6.4 as well
* Secure file on AzureCI is updated
* Update the documentation for better clarity

Co-authored-by: Pramod S Kumbhar <pramod.s.kumbhar@gmail.com>
nrnhines added a commit that referenced this pull request Sep 13, 2023
* Fix MacOS linux wheel backspace by upgrading ncurses-6.2 to 6.4 (#2484)

* Fix Windows and Mac crash on multiline HOC statements input from terminal. (#2258)

* Windows 11 fix for hoc icon. (#2279)

* test using nrniv -isatty    Mac and Windows have trouble with the test.

* Update the documentation for better clarity

* Pin cython to < 3. Workaround for #2430 (#2433)
Cherry picked from 116a541.
Eliminate non cython<3 changes from cherrypick

* Bump VERSION to 8.2.3 in CMakeLists.txt

* deal with .inputrc if missing on windows

* Backport changes from github docs workflow

* Disable shallow clone with the  option

* use  docs/domains/hocdomain.py from master for latest sphinx compatibility

* downgrade black version (30 python files have trivial format differences)

* Picked the "final:" fragment out of the master version.
Fixes "Final CI Expected ---  Waiting for status to be reported"

---------

Co-authored-by: Pramod S Kumbhar <pramod.s.kumbhar@gmail.com>
Co-authored-by: Alexandru Săvulescu <alexandru.savulescu@epfl.ch>
@nrnhines nrnhines mentioned this pull request Jun 25, 2024
27 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Wheels don't support backspace/delete keys in interactive usage with version >= 8.1.0

4 participants