Skip to content

Bring NMODL transpiler under NEURON#3333

Merged
JCGoran merged 860 commits into
masterfrom
jelic/clean-nmodl-merge
Mar 12, 2025
Merged

Bring NMODL transpiler under NEURON#3333
JCGoran merged 860 commits into
masterfrom
jelic/clean-nmodl-merge

Conversation

@JCGoran

@JCGoran JCGoran commented Feb 13, 2025

Copy link
Copy Markdown
Collaborator

This is a cleaned-up version of #3272, where all of the CMake changes were squashed into one commit (this one 3894e81), and it has been rebased on latest master.

Closes #3308.

Content below the line is taken from #3265 and updated to reflect the necessary changes.


  • Script to trim and clean NMODL repository
  • Test NMODL repository
  • Merge nmodl trimed repo into NEURON - A Dry Run!
  • "Marriage" of CMake - merging/linking NEURON and NMODL CMake
  • Cleanup and remove unnecessary/duplicate files

Steps To Follow

Here are 5 major steps that we need to follow:

Clone NRN and NMODL Repos

#!/bin/bash

BASE_DIR=$(pwd)/nrn-nmodl-merge
mkdir -p ${BASE_DIR} && cd ${BASE_DIR}

git clone -b master --single-branch git@github.com:BlueBrain/nmodl.git
git clone -b master --single-branch git@github.com:neuronsimulator/nrn.git

Install git-filter-repo

pip install git-filter-repo

# TODO: change if necessary, `git-filter-repo` should be in $PATH
export PATH=$HOME/.local/bin/:$PATH

Prepare NMODL Repo

Use git filter-repo to do all hard work 👷

#!/bin/bash


cd ${BASE_DIR}/nmodl

# create a commit hook file that will be used by `git filter-repo`
cat <<EOF >> commit_hook.py

commit_message = commit.message.decode()
original_id = commit.original_id.decode()

# lets first remove trailing new lines
commit_message = commit_message.rstrip("\n")

# append now the original SHA in the commit message
commit_message = commit_message + "\n\nNMODL Repo SHA: BlueBrain/nmodl@" + original_id

# make links to PRs
matches = re.findall('#[0-9]+', commit_message, re.DOTALL)
if matches:
    for match in set(matches):
        # rename the PR IDs as they could be misleading / point to nrn PRs
        match.replace('#', '#nmodl-')
        new_prefix = "BlueBrain/nmodl" + match
        commit_message = commit_message.replace(match, new_prefix)

commit.message = commit_message.encode()
EOF

# first change commit messages and PR references
git filter-repo --force --preserve-commit-hashes --commit-callback commit_hook.py
rm commit_hook.py

# rename tags
git filter-repo --tag-rename '':'nmodl-'

# remove files that we don't need after merging in nrn
git filter-repo \
    --path .bbp-project.yaml \
    --path .clang-format.changes \
    --path .clang-tidy \
    --path .clang-tidy \
    --path .gitignore \
    --path .gitlab-ci.yml \
    --path .github \
    --path .gitmodules \
    --path .sanitizers \
    --path .cmake-format.changes.yaml \
    --path AUTHORS.txt \
    --path MANIFEST.in \
    --path azure-pipelines.yml \
    --path ci \
    --path ext \
    --path packaging \
    --path pyproject.toml \
    --path requirements.txt \
    --path setup.cfg \
    --path sonar-project.properties \
    --invert-paths

# now start renaming and remapping of directories and files 
git filter-repo --path-rename cmake/:cmake/nmodl/
git filter-repo --path-rename CMakeLists.txt:cmake/nmodl/CMakeLists.txt
git filter-repo --path-rename CONTRIBUTING.rst:docs/nmodl/transpiler/CONTRIBUTING.rst
git filter-repo --path-rename docs/:docs/nmodl/transpiler/
git filter-repo --path-rename INSTALL.rst:docs/nmodl/transpiler/INSTALL.rst
git filter-repo --path-rename LICENSE:src/nmodl/LICENSE
git filter-repo --path-rename python/:src/nmodl/python/
git filter-repo --path-rename README.rst:src/nmodl/README.rst
git filter-repo --path-rename share/nrnunits.lib:share/nmodl/nrnunits.lib
git filter-repo --path-rename src/:src/nmodl/
git filter-repo --path-rename test/:test/nmodl/transpiler/

We now have a "cleaned" version of NMODL repository. See if generated structure is what you want:

$ tree -L 3
.
├── cmake
│   └── nmodl
│       ├── CMakeLists.txt
│       ├── ClangTidyHelper.cmake
│       ├── CompilerHelper.cmake
│       ├── Config.cmake.in
│       ├── ExternalProjectHelper.cmake
│       ├── FlexHelper.cmake
│       ├── GitRevision.cmake
│       ├── PythonLinkHelper.cmake
│       ├── RpathHelper.cmake
│       └── hpc-coding-conventions
├── docs
│   └── nmodl
│       └── transpiler
├── share
│   └── nmodl
│       └── nrnunits.lib
├── src
│   └── nmodl
│       ├── CMakeLists.txt
│       ├── ast
│       ├── codegen
│       ├── config
│       ├── language
│       ├── lexer
│       ├── main.cpp
│       ├── nmodl
│       ├── parser
│       ├── printer
│       ├── pybind
│       ├── solver
│       ├── symtab
│       ├── units
│       ├── utils
│       └── visitors
└── test
    └── nmodl
        └── transpiler

We can now push this trimmed-down version of the repo into neuronsimulator org to verify if the commits and PRs are properly linked to original NMODL repo:

git remote add  neuronsimulator  git@github.com:neuronsimulator/nmodl-test.git
git push -f --all neuronsimulator

See the result at https://github.com/neuronsimulator/nmodl-test.

Merge NMODL Repo into NEURON

Now comes the part of merging the two repositories. Add "cleaned" nmodl repo as remote and merge the two repos as:

cd ${BASE_DIR}/nrn
git remote add nmodl file://${BASE_DIR}/nmodl
git fetch  nmodl --tags
git merge --allow-unrelated-histories nmodl/master

This should merge the repos cleanly. We can push this to nrn branch:

git push -f origin HEAD:pramodk/nmodl-merge

"Marriage" of CMake and Build System

We have now nmodl and nrn repo merged. But there are various things that needs to be happened:

  • Add any submodules needed for NMODL
  • Update CMakeLists.txt and connect nmodl's CMakeLists.txt
  • Check requirements.txt and other build dependencies
  • Update Spack recipe (?) due to BBP closure, any Spack recipes will need to be updated by third parties
  • Update CIs
  • Remove unnecessary files from docs, cmake, tests, etc

JCGoran and others added 30 commits March 11, 2024 08:56
The previous behaviour for

    KINETIC states {
         ~ x + y << 42.0
    }

was to print a warning and continue as if the line didn't exist. The new
behaviour is to throw an error, because it's unclear why ignoring the
line could ever lead to a correct translation of the MOD file.

NMODL Repo SHA: BlueBrain/nmodl@9292c18
* Format Python files with `black`.
* Enable `black` formatter.

NMODL Repo SHA: BlueBrain/nmodl@40aa0bf
When encountering an indexed `COMPARTMENT` block:

    COMPARTMENT i, volume_expr { species }

All state variables are searched if they match they match the pattern
`"{species}[%d]"`. For each matching state variables, we obtain the
value of the array index and substitute the name of the array index with
its value in `volume_expr`. This is then stored in the array of
compartment factors.

Check that the resulting derivatives are divided by the volume
of the compartment.

NMODL Repo SHA: BlueBrain/nmodl@219a3ed
Co-authored-by: Nicolas Cornu <me@alkino.fr>

NMODL Repo SHA: BlueBrain/nmodl@c818f16
…1234)

This can be reproduced by:

    cmake -B build && cmake --build build --target=generate_references

i.e. call `--target=generate_references` without building it first. It
would complain about missing `bin/nmodl`.

NMODL Repo SHA: BlueBrain/nmodl@735b11d
Store the forward and reverse rates in local variables, e.g.:
```
rate = 42.0
~ a <-> b (1.0*rate, 2.0*rate)
```
is converted to:
```
LOCAL kf0_, kb0_
rate = 42.0
kf0_ = 1.0*rate
kb0_ = 2.0*rate
~ a <-> b (kf0_, kb0_)
```

This solves a bug that assigning to `rate` between two reaction equation statements would mean the first line sees the value meant for the second line.

NMODL Repo SHA: BlueBrain/nmodl@868ce13
In NRN we depend on the target `nmodl` to ensure that when we run `nrnivmodl -coreneuron` the binary `nmodl` exists. We need to also ensure that everything `nmodl` needs to works is present.

This fixes a dependency bug on certain copied files by making the target for copying the files a dependency of `nmodl`.

NMODL Repo SHA: BlueBrain/nmodl@b78c3f3
* Fix array variables.

These are the code generation changes required for:

    #2779

The solution is to fill `nrn_prop_param_size` with the number of
doubles, not number of variables.

NMODL Repo SHA: BlueBrain/nmodl@f6821ce
…dl#1244)

- If `-DNMODL_ENABLE_PYTHON_BINDINGS=OFF` then there is no need
  to generate AST and other wrapper classes for Pybind11.
- Using sympy-based solvers does not require Python bindings to
  be enabled. Avoid confusing warning when using nmodl binary.
- When someone tries to use the nmodl module via Python, the
  warning is still preserved.

```console
$ nmodl mod_examples/sparse.mod
[NMODL] [warning] :: Python bindings are not available with this installation
..

$ nmodl mod_examples/sparse.mod
$ python3.11 -c "import nmodl"
[NMODL] [warning] :: Python bindings are not available with this installation
```

NMODL Repo SHA: BlueBrain/nmodl@8eb88e4
- when neuron is built with nmodl then nmodl related files
  should be installed in <prefix>/ and not in <prefix>/nmodl/.data
- <prefix>/nmodl/.data is used when we build standalone wheels
- add one mod file using sparse solver - nmodl will automatically
  use sympy solver

NMODL Repo SHA: BlueBrain/nmodl@2fb037e
Instead of setting the conductance as:

  g = g + g0 * exp(-(t - t0)/tau)

we set it to
  g += g0

where `g0` is the argument passed to NET_RECEIVE, i.e. the weight.

This allows for an analytic solution, that can be tested.

NMODL Repo SHA: BlueBrain/nmodl@ada65df
Random collection of fixups to make hh.mod work.

---------

Co-authored-by: Luc Grosheintz <luc.grosheintz@gmail.com>

NMODL Repo SHA: BlueBrain/nmodl@b451d39
The idea is to generate the references, but not check that there are no differences. Doing so allows us to push the changes automatically to `nmodl-references`.

NMODL Repo SHA: BlueBrain/nmodl@0e69948
@JCGoran JCGoran marked this pull request as ready for review February 13, 2025 12:32
@JCGoran JCGoran changed the title [WIP] Bring NMODL transpiler under NEURON Bring NMODL transpiler under NEURON Feb 13, 2025
@JCGoran JCGoran requested a review from pramodk February 13, 2025 12:32
@azure-pipelines

Copy link
Copy Markdown

✔️ 3894e81 -> Azure artifacts URL

@JCGoran JCGoran added the nmodl label Feb 13, 2025
@codecov

codecov Bot commented Feb 13, 2025

Copy link
Copy Markdown

Codecov Report

Attention: Patch coverage is 18.66667% with 305 lines in your changes missing coverage. Please review.

Project coverage is 68.31%. Comparing base (23a4a0f) to head (709ab85).
Report is 937 commits behind head on master.

Files with missing lines Patch % Lines
share/lib/python/neuron/nmodl/ode.py 0.00% 244 Missing ⚠️
share/lib/python/neuron/nmodl/ast.py 0.00% 23 Missing ⚠️
share/lib/python/neuron/nmodl/__init__.py 0.00% 22 Missing ⚠️
share/lib/python/neuron/nmodl/dsl.py 0.00% 13 Missing ⚠️
share/lib/python/neuron/__init__.py 0.00% 1 Missing ⚠️
share/lib/python/neuron/nmodl/symtab.py 0.00% 1 Missing ⚠️
share/lib/python/neuron/nmodl/visitor.py 0.00% 1 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##           master    #3333      +/-   ##
==========================================
+ Coverage   67.05%   68.31%   +1.25%     
==========================================
  Files         571      681     +110     
  Lines      111076   116420    +5344     
==========================================
+ Hits        74480    79527    +5047     
- Misses      36596    36893     +297     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@JCGoran

JCGoran commented Feb 16, 2025

Copy link
Copy Markdown
Collaborator Author

There appear to be two issues which do not appear in the CI, but I'm not sure whether are reproducible (don't have another Apple machine to compare):

  1. when building with homebrew'd gcc 14 on MacOS, GCC does not play along nicely with the linker (I think) and we end up with NMODL not working:
dyld[98181]: symbol not found in flat namespace '_PyBaseObject_Type'
[1]    98181 abort      ./bin/nmodl

CMake build flags:

-DNRN_ENABLE_TESTS=OFF -DNRN_ENABLE_PERFORMANCE_TESTS=OFF -DNRN_ENABLE_PYTHON=ON -DNRN_ENABLE_PYTHON_DYNAMIC=ON -DNRN_ENABLE_CORENEURON=ON -DCMAKE_INSTALL_PREFIX=./install -B build -DNRN_ENABLE_INTERVIEWS=OFF -DNRN_ENABLE_RX3D=OFF -DCMAKE_C_FLAGS="-O1" -DCMAKE_CXX_FLAGS="-O1" -DCMAKE_C_COMPILER=/opt/homebrew/opt/gcc@14/bin/gcc-14 -DCMAKE_CXX_COMPILER=/opt/homebrew/opt/gcc@14/bin/g++-14
  1. Similar to bottom half of Merge NMODL CMakeLists.txt #3272 (comment), when trying out the NMODL Python module when one uses the -O1 flag to compile it, there is an assertion error (same build flags as above, minus the CMAKE_{TYPE}_COMPILER, it happens both with Apple Clang as well as the homebrew'd one, which is new compared to the linked comment where it works fine with homebrew'd Clang + ASAN):
./bin/nrniv -python -c 'from neuron import nmodl;nmodl.NmodlDriver().parse_string("NEURON {SUFFIX asdf}")'
NEURON -- VERSION 9.0a-1313-g517dd1261+ jelic/clean-nmodl-merge (517dd1261+) 2025-02-13
Duke, Yale, and the BlueBrain Project -- Copyright 1984-2022
See http://neuron.yale.edu/neuron/credits

Assertion failed: (*yytypeid_ == typeid (T)), function as, file nmodl_parser.hpp, line 307.
[1]    7542 abort      ./bin/nrniv -python -c

The first one is not a deal-breaker since people usually use the default Clang on MacOS, though the second one is a bit sketchy.

@nrnhines

Copy link
Copy Markdown
Member

I thought I'd look into

dyld[98181]: symbol not found in flat namespace '_PyBaseObject_Type'

as that is something I've had a bit of experience with.
I'm using an Apple M1 Python3.13 and started with

pip install -r nrn_requirements.txt
cmake .. -G Ninja -DCMAKE_INSTALL_PREFIX=install -DNRN_ENABLE_NMODL=ON

to check that the basic build was working.
However I was diverted by the fatal build error towared the end of ninja at step [929/1204] Generating ../nrnoc/hh.cpp

NMODL_PYLIB environment variable must be set to load embedded python

Trying again after

export NMODL_PYLIB="/Library/Frameworks/Python.framework/Versions/3.13/lib/libpython3.13.dylib"

I get

NMODLHOME environment variable must be set to load embedded python

It is a bit ambiguous what is needed for that, but in analogy to the output of nrnpyenv.sh I tried

export NMODLHOME=/Library/Frameworks/Python.framework/Versions/3.13

and get

NMODLHOME doesn't contain libpywrapper.dylib library

After a bit of searching and trial and error I see it needs to be (for the build)

export NMODLHOME=/Users/hines/neuron/nrnnmodl/build

I suppose some of this is a consequence of an assumption that python needs to be dynamically determined at nmodl runtime (NMODL_PYLIB). However, that is not the case for this build (Python is specified at cmake time). And if it does need to be dynamically determined, NEURON does this with the nrnpyenv.sh script. Seems like default NMODLHOME under most circumstances is determinable from the user install location and only rarely would be needed to be set by the user.

Now back to looking into _PyBaseObject_Type ...

@JCGoran JCGoran force-pushed the jelic/clean-nmodl-merge branch from 3894e81 to 4f17bd9 Compare February 16, 2025 18:22
@JCGoran

JCGoran commented Feb 16, 2025

Copy link
Copy Markdown
Collaborator Author

@nrnhines good catch, I didn't try that configuration out at all, I pushed a change so now it should work :) Note that NRN_ENABLE_NMODL=ON enables NMODL codegen for NEURON, which is currently lacking some features. The "correct" way (for now at least) to enable NMODL is via NRN_ENABLE_CORENEURON=ON, since NMODL codegen for coreNEURON is more or less complete. Note that the _PyBaseObject_Type issue seems to require setting the default compiler to GCC (I tried out version 14, but not others), as on Clang (either the Apple one or the homebrew one) this issue doesn't occur.

@azure-pipelines

Copy link
Copy Markdown

✔️ 4f17bd9 -> Azure artifacts URL

@nrnhines

Copy link
Copy Markdown
Member

pushed a change so now it should work

Thanks! Just to clarify the present situation.

cmake .. -G Ninja -DCMAKE_INSTALL_PREFIX=install -DNRN_ENABLE_NMODL=ON

ends with a nice summary of what I desired (if I'm interpreting it correctly):

-- CoreNEURON    | OFF
-- NMODL codegen   | ON

but near the end of the build, one is told that

ninja install
...
make: *** [camchan.cpp] Error 134
[NMODL] [critical] :: NMODL_PYLIB environment variable must be set to load embedded python
[FATAL] NMODL encountered an unhandled exception.

Leaving out the install and just using ninja is successful. And one can use nrniv, although executing neurondemo (or nrnivmodl still requires specifying NMODL_PYLIB and NMODLHOME
As expected neurondemo used nmodl to translate the mod files. nrnivmodl also, by default, uses nmodl without requiring the -nmodl /where/to/find/the/nmodl/executable args.

Per your suggestion, I also tried

cmake .. -G Ninja -DCMAKE_INSTALL_PREFIX=install -DNRN_ENABLE_CORENEURON=ON
...
-- CoreNEURON    | ON
--   PATH        | /Users/hines/neuron/nrnnmodl/src/coreneuron
--   LINK FLAGS  |  -lcorenrnmech  -Wl,-rpath,/opt/homebrew/Cellar/open-mpi/4.1.5/lib /opt/homebrew/Cellar/open-mpi/4.1.5/lib/libmpi.dylib
-- NMODL codegen   | OFF

I wonder about the precise meaning of -- NMODL codegen | OFF and hypothesize that nrnivmodl will by default use nocmodl but nmodl for NEURON can be forced with the -nmodl arg. We'll see.

ninja install succeeds (neurondemo used nocmodl for the NEURON part and did not create a CoreNEURON special or library. That's fine.)

At the moment I see issues with the neurondemo mod files with respect to coreneuron. They are rather trivial and there is no need to deal with them now.

[NMODL] [error] :: Code incompatibility detected
[NMODL] [error] :: Cannot translate mod file to .cpp file
[NMODL] [error] :: Fix the following errors and try again
[NMODL] [error] :: Code Incompatibility :: "inf" variable found at [9.9-11] should be defined as a RANGE variable instead of GLOBAL to enable backend transformations
[NMODL] [error] :: Code Incompatibility :: "tau" variable found at [9.14-16] should be defined as a RANGE variable instead of GLOBAL to enable backend transformations
make: *** [arm64/corenrn/mod2c/cachan1.cpp] Error 1
make: *** Waiting for unfinished jobs....
[NMODL] [error] :: Code incompatibility detected
[NMODL] [error] :: Cannot translate mod file to .cpp file
[NMODL] [error] :: Fix the following errors and try again
[NMODL] [error] :: Code Incompatibility :: "vol" variable found at [15.9-11] should be defined as a RANGE variable instead of GLOBAL to enable backend transformations

nrnivmodl -nmodl `which nmodl` needs NMODL_PYLIB and NMODLHOME and with those is successful.

@nrnhines

nrnhines commented Feb 17, 2025

Copy link
Copy Markdown
Member

Here is a progress note about gcc-14 and _PyBaseObject_Type on Apple M1 with Python 3.13

export NMODL_PYLIB="/Library/Frameworks/Python.framework/Versions/3.13/lib/libpython3.13.dylib"export NMODLHOME=/Users/hines/neuron/nrnnmodl/build
CC=gcc-14 CXX=g++-14 cmake .. -G Ninja -DCMAKE_INSTALL_PREFIX=install -DNRN_ENABLE_NMODL=ON
...
-- C COMPILER    | /opt/homebrew/bin/gcc-14
-- CXX COMPILER  | /opt/homebrew/bin/g++-14
-- COMPILE FLAGS | -g  -O2    -fopenmp-simd
-- CoreNEURON    | OFF
-- NMODL codegen   | ON
ninja
...
[836/1203] Linking CXX executable bin/nmodl_lexer
FAILED: bin/nmodl_lexer 
: && /opt/homebrew/bin/g++-14 -fopenmp-simd -g  -O2   -arch arm64 -isysroot /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX15.2.sdk -Wl,-search_paths_first -Wl,-headerpad_max_install_names -Wl,--unresolved-symbols=ignore-all src/nmodl/lexer/CMakeFiles/nmodl_lexer.dir/main_nmodl.cpp.o -o bin/nmodl_lexer  lib/liblexer.a  lib/libutil.a  lib/libspdlog.a  lib/libfmt.a && :
ld: unknown options: --unresolved-symbols=ignore-all 

I guess to get further one has to figure out how to force an alternative to the cmake determined -Wl,--unresolved-symbols=ignore-all

My cmake version is 3.26.3

@azure-pipelines

Copy link
Copy Markdown

✔️ 3bc292a -> Azure artifacts URL

@JCGoran

JCGoran commented Feb 24, 2025

Copy link
Copy Markdown
Collaborator Author

@nrnhines thanks for checking, the linker issue should be fixed now.
Note that I merged the NMODL flags with NEURON ones, and their defaults were different (for instance, NMODL's LINK_AGAINST_PYTHON was by default ON, while NEURON uses NRN_LINK_AGAINST_PYTHON=OFF everywhere except on Windows), which could be the source of some of the issues you are experiencing.
The only NMODL flag that wasn't merged are NMODL_ENABLE_PYTHON_BINDINGS and NMODL_EXTRA_CXX_FLAGS, while the rest (see https://github.com/BlueBrain/nmodl/blob/06132d23125bf6d65b0cc7b7136754ed378969d9/CMakeLists.txt#L23-L37 for a list) were merged into their corresponding NEURON ones (or removed if they weren't independent).

@nrnhines

Copy link
Copy Markdown
Member

the linker issue should be fixed now.

It does get a lot further now (using my above exports, cmake, ninja, on my apple M1). Near the end of the build,
nmodl fails when translating mod files. E.g.

[927/1204] Generating ../nrnoc/hh.cpp
FAILED: src/nrnoc/hh.cpp /Users/hines/neuron/nrnnmodl/bldg14/src/nrnoc/hh.cpp 
cd /Users/hines/neuron/nrnnmodl/bldg14/src/nrniv && /opt/homebrew/Cellar/cmake/3.26.3/bin/cmake -E env MODLUNIT=/Users/hines/neuron/nrnnmodl/bldg14/share/nrn/lib/nrnunits.lib NMODL_PYLIB=/Library/Frameworks/Python.framework/Versions/3.13/lib/libpython3.13.dylib NMODLHOME=/Users/hines/neuron/nrnnmodl/bldg14 /Users/hines/neuron/nrnnmodl/bldg14/bin/nmodl /Users/hines/neuron/nrnnmodl/bldg14/src/nrnoc/hh.mod --neuron -o /Users/hines/neuron/nrnnmodl/bldg14/src/nrnoc && sed 's/_reg()/_reg_()/' /Users/hines/neuron/nrnnmodl/bldg14/src/nrnoc/hh.cpp > /Users/hines/neuron/nrnnmodl/bldg14/src/nrnoc/hh.cpp.tmp && /opt/homebrew/Cellar/cmake/3.26.3/bin/cmake -E copy /Users/hines/neuron/nrnnmodl/bldg14/src/nrnoc/hh.cpp.tmp /Users/hines/neuron/nrnnmodl/bldg14/src/nrnoc/hh.cpp && /opt/homebrew/Cellar/cmake/3.26.3/bin/cmake -E remove /Users/hines/neuron/nrnnmodl/bldg14/src/nrnoc/hh.cpp.tmp
dyld[71768]: symbol not found in flat namespace '_PyBaseObject_Type'

One can focus on the details with

hines@Michaels-MacBook-Pro-2 bldg14 % rm bin/nmodl
hines@Michaels-MacBook-Pro-2 bldg14 % ninja -v nmodl
[0/2] /opt/homebrew/Cellar/cmake/3.26.3/bin/cmake -P /Users/hines/neuron/nrnnmodl/bldg14/CMakeFiles/VerifyGlobs.cmake
[1/1] : && /opt/homebrew/bin/g++-14 -fopenmp-simd -g  -O2   -arch arm64 -isysroot /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX15.2.sdk -Wl,-search_paths_first -Wl,-headerpad_max_install_names -Wl,-undefined,dynamic_lookup src/nmodl/printer/CMakeFiles/printer.dir/code_printer.cpp.o src/nmodl/printer/CMakeFiles/printer.dir/json_printer.cpp.o src/nmodl/printer/CMakeFiles/printer.dir/nmodl_printer.cpp.o src/nmodl/CMakeFiles/nmodl.dir/main.cpp.o -o bin/nmodl  lib/libcodegen.a  lib/libvisitor.a  lib/libsymtab.a  lib/libutil.a  lib/liblexer.a  lib/libpyembed.a  -ldl  lib/libutil.a  lib/libspdlog.a  lib/libfmt.a && :
ld: warning: ignoring duplicate libraries: 'lib/libutil.a'
hines@Michaels-MacBook-Pro-2 bldg14 % bin/nmodl
dyld[71870]: symbol not found in flat namespace '_PyBaseObject_Type'
zsh: abort      bin/nmodl

Note:

libnanobind.a
                 U _PyBaseObject_Type
libpyembed.a
                 U _PyBaseObject_Type
libpywrapper.dylib
                 U _PyBaseObject_Type

I'm not very familiar with nmodl. If this is a consequence of runtime dynamic linking of python, it is a puzzle to me why clang has no problem (it used -Wl,-undefined,dynamic_lookup) but gcc-14 does have a problem with -Wl,-undefined,dynamic_lookup. Note that the clang link step seems identical to gcc-14. ie.

hines@Michaels-MacBook-Pro-2 build % ninja -v nmodl
[0/2] /opt/homebrew/Cellar/cmake/3.26.3/bin/cmake -P /Users/hines/neuron/nrnnmodl/build/CMakeFiles/VerifyGlobs.cmake
[1/1] : && /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/c++ -openmp-simd -g  -O2   -arch arm64 -isysroot /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX15.2.sdk -Wl,-search_paths_first -Wl,-headerpad_max_install_names -Wl,-undefined,dynamic_lookup src/nmodl/printer/CMakeFiles/printer.dir/code_printer.cpp.o src/nmodl/printer/CMakeFiles/printer.dir/json_printer.cpp.o src/nmodl/printer/CMakeFiles/printer.dir/nmodl_printer.cpp.o src/nmodl/CMakeFiles/nmodl.dir/main.cpp.o -o bin/nmodl  lib/libcodegen.a  lib/libvisitor.a  lib/libsymtab.a  lib/libutil.a  lib/liblexer.a  lib/libpyembed.a  -ldl  lib/libutil.a  lib/libspdlog.a  lib/libfmt.a && :

@JCGoran JCGoran force-pushed the jelic/clean-nmodl-merge branch 2 times, most recently from 6801782 to 14e27c4 Compare March 3, 2025 15:28
@azure-pipelines

Copy link
Copy Markdown

✔️ 14e27c4 -> Azure artifacts URL

@nrnhines

nrnhines commented Mar 11, 2025

Copy link
Copy Markdown
Member

Just to finish up the gcc-14 comments. The following works on my Apple M1

CC=gcc-14 CXX=g++-14 cmake .. -G Ninja -DCMAKE_INSTALL_PREFIX=install -DNRN_ENABLE_NMODL=ON
ninja install

No manual exports needed.

edit:
I realize I botched this test by using a fresh clone of the master instead of jelic/clean-nmodl-merge. With the latter, the use of nrnnmodl/bldg14/bin/nmodl still gives

hines@Michaels-MacBook-Pro-2 bldg14 % bin/nmodl -h
dyld[38113]: symbol not found in flat namespace '_PyBaseObject_Type'

Nevertheless, this issue should not prevent merging this PR to master. It can be fixed in subsequent PR.

@nrnhines nrnhines left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great job! You mentioned that this needs to be merged with a rebase, not with a squash so we preserve the commit history from NMODL. Please go ahead and do that.

@JCGoran JCGoran force-pushed the jelic/clean-nmodl-merge branch from 14e27c4 to 709ab85 Compare March 12, 2025 10:52
@sonarqubecloud

Copy link
Copy Markdown

@azure-pipelines

Copy link
Copy Markdown

✔️ 709ab85 -> Azure artifacts URL

@JCGoran JCGoran merged commit 9c36354 into master Mar 12, 2025
@JCGoran JCGoran deleted the jelic/clean-nmodl-merge branch March 12, 2025 13:57
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

NMODL integration

7 participants