Use SOA format for mechanism data by olupton · Pull Request #2027 · neuronsimulator/nrn

olupton · 2022-10-17T11:24:55Z

Introduce new SOA data structures for NEURON model data. So far:

Nodes: area, voltages and matrix equation data (d, a, b, rhs).
Mechanisms: floating point data (i.e. RANGE variables)

are migrated to use the new neuron::container::soa<...> format.

Status:

The CI on this PR is normally passing
Internal BBP CI is normally passing
ModelDB CI is normally passing without suspicious diffs

Dependencies:

Merge Compatibility with SOA NEURON data structures BlueBrain/nmodl#989 and update submodule.
Merge std::function<double()> as alternative to double* BlueBrain/libsonatareport#37
Merge https://bbpgitlab.epfl.ch/hpc/sim/neurodamus-py/-/merge_requests/166
Merge updates to ModelDB models
- 7399, 9889, 12631, 52034, 113732: need fixes due to taking the address of RANGE variables, see Use SOA format for mechanism data #2027 (comment), Use SOA format for mechanism data #2027 (comment), Use SOA format for mechanism data #2027 (comment) and Use SOA format for mechanism data #2027 (comment)
  - Do not store pointers to RANGE variable values ModelDBRepository/7399#3
  - Do not store pointers to RANGE variable values ModelDBRepository/12631#4
  - Do not store pointers to RANGE variable values ModelDBRepository/52034#2
  - 9889 and 113732 do not use the addresses they store; no patches needed
  - 113732 also needed a RandomPlay fix
- 106891: compilation failure due to explicit use of param, Compatibility with new data structures. ModelDBRepository/106891#4
- 113446: mis-declared array variable dimension, Make the allocated size of ca match the usage ModelDBRepository/113446#1
- 116838 136095 138379 140881 141505 144538 146949 150245: these had expressions like _p=X in VERBATIM blocks that needed to be fixed, plus assumptions about the layout of Datum
- Some models assume that full definitions of structs like Node, Prop and Section are visible:
Merge Fixes for SoA NEURON data structures nrn-modeldb-ci#67 after confirming that it is still sufficient to make ModelDB CI pass with this PR.
Document common VERBATIM patterns: @olupton: adding this as todo here so that we don't forget to discuss: considering ModelDB PR updates, are there any patterns that are worth documenting explicitly like we did in Adapting MOD files for C++ ?

This builds on #1929 (which was not merged) and incorporates ideas from https://github.com/neuronsimulator/data-structure-design.

* Bump submodule past iv#45. * PyObject_HEAD contains an ; * drop nrn_isdouble * drop unused memory pool code

codecov-commenter · 2022-11-02T13:00:51Z

Codecov Report

Merging #2027 (2b44298) into master (b4ade55) will increase coverage by 1.38%.
The diff coverage is 92.02%.

❗ Current head 2b44298 differs from pull request most recent head e23d8d8. Consider uploading reports for the commit e23d8d8 to get more accurate results

@@            Coverage Diff             @@
##           master    #2027      +/-   ##
==========================================
+ Coverage   58.77%   60.15%   +1.38%     
==========================================
  Files         623      642      +19     
  Lines      119597   121134    +1537     
==========================================
+ Hits        70288    72872    +2584     
+ Misses      49309    48262    -1047

Impacted Files	Coverage Δ
...re/lib/python/neuron/tests/utils/capture_stdout.py	`0.00% <0.00%> (ø)`
src/ivoc/oc2iv.h	`50.00% <ø> (ø)`
src/ivoc/symdir.cpp	`0.00% <0.00%> (ø)`
src/ivoc/xmenu.h	`68.00% <ø> (+2.61%)`	⬆️
src/nmodl/netrec_discon.cpp	`62.31% <ø> (ø)`
src/nrncvode/netcvode.h	`100.00% <ø> (ø)`
src/nrncvode/vrecitem.h	`65.51% <ø> (ø)`
src/nrniv/arraypool.h	`96.10% <ø> (-3.90%)`	⬇️
src/nrniv/bbsavestate.h	`50.00% <ø> (ø)`
src/nrniv/glinerec.h	`0.00% <ø> (-35.72%)`	⬇️
... and 118 more

... and 9 files with indirect coverage changes

📣 We’re building smart automated test selection to slash your CI/CD build times. Learn more

… results with names not indices

azure-pipelines · 2023-06-05T12:00:30Z

✔️ 2b44298 -> Azure artifacts URL

github-actions · 2023-06-05T12:01:10Z

NEURON ModelDB CI: launching for 2b44298 via its drop url

alexsavulescu

🚀

github-actions · 2023-06-05T14:53:45Z

NEURON ModelDB CI: 2b44298 -> download reports from here

azure-pipelines · 2023-06-05T15:41:18Z

✔️ e23d8d8 -> Azure artifacts URL

bbpbuildbot · 2023-06-05T16:30:56Z

olupton changed the base branch from olupton/data-structures to master October 27, 2022 08:32

olupton commented Nov 1, 2022

View reviewed changes

Comment thread src/nrniv/kschan.cpp Outdated

olupton commented Nov 1, 2022

View reviewed changes

Comment thread src/nrniv/singlech.cpp Outdated

olupton force-pushed the olupton/data-structures-including-mechanisms branch 2 times, most recently from 3f121e0 to b51f37f Compare November 1, 2022 12:14

olupton mentioned this pull request Nov 1, 2022

Minor improvements from #2027 #2051

Merged

olupton added a commit that referenced this pull request Nov 1, 2022

Minor improvements from #2027 (#2051)

53f87a1

* Bump submodule past iv#45. * PyObject_HEAD contains an ; * drop nrn_isdouble * drop unused memory pool code

olupton force-pushed the olupton/data-structures-including-mechanisms branch from ac0450f to 88cf7b6 Compare November 1, 2022 21:07

This was referenced Nov 2, 2022

Find or implement an apply_reverse_permutation algorithm for runtime-variable-width zips of ranges #2054

Closed

Modernise underlying data structure for node voltages #1929

Closed

olupton commented Nov 8, 2022

View reviewed changes

Comment thread src/nrniv/shape.cpp Outdated

alexsavulescu force-pushed the olupton/data-structures-including-mechanisms branch from d41678e to 63e20bc Compare November 8, 2022 16:14

olupton force-pushed the olupton/data-structures-including-mechanisms branch 3 times, most recently from a4e9fac to ce1bade Compare November 9, 2022 18:09

This comment was marked as outdated.

Sign in to view

nrnhines mentioned this pull request Nov 10, 2022

tests intended for helping with some datahandle code fragments. #2080

Merged

olupton force-pushed the olupton/data-structures-including-mechanisms branch from 73e3340 to cd03677 Compare November 11, 2022 21:35

This comment was marked as outdated.

Sign in to view

pramodk requested review from iomaganaris and kotsaloscv November 14, 2022 13:12

This comment was marked as outdated.

Sign in to view

olupton force-pushed the olupton/data-structures-including-mechanisms branch from 7ded4ab to 4ac4e2b Compare November 21, 2022 16:15

This comment was marked as outdated.

Sign in to view

olupton commented Nov 22, 2022

View reviewed changes

Comment thread share/examples/nrniv/nmodl/cagkftab.py Outdated

olupton added 11 commits June 5, 2023 13:08

drop some more methods

86b7665

pickup patch in nrntest

9e07a81

test_kschan.py: scale voltage tolerances with *time*, index reference…

418c299

… results with names not indices

tweaks for nvhpc

658787f

fix NODEA and NODEB usage in .mod files

ea79d55

first chunk of data structures doc page

e4de4f4

drop some assert(cml->ml.size() == 1), triggered by new model 267666

0bbf3bd

Expand documentation

12291d9

add a test for non-owning handles

4285b1e

tweak docs, test another case

1c90673

slightly expand docs

2b44298

alexsavulescu approved these changes Jun 5, 2023

View reviewed changes

This comment has been minimized.

Sign in to view

nmodl submodule (and a typo fix)

e23d8d8

This was referenced Jun 6, 2023

Fix compilation on Debian stable #2372

Merged

test_wheels.sh fails on macOS #2374

Open

ramcdougal mentioned this pull request Jun 13, 2023

plotshape issues with Python on mac #2386

Closed

This was referenced Jun 14, 2023

Use modern SoA helpers for fast_imem data. #2381

Merged

neuron: add 9.0.a9 including SoA data BlueBrain/spack#2051

Merged

olupton mentioned this pull request Jun 26, 2023

Curate skipped models neuronsimulator/nrn-modeldb-ci#33

Open

ramcdougal mentioned this pull request Jul 1, 2023

Rxd sourcevar #1549

Draft

This was referenced Aug 21, 2023

Another problem with datahandle #2458

Closed

Fix leak in write_nrnthread. #2456

Merged

ramcdougal mentioned this pull request Mar 27, 2024

Higher simulation time in NEURON 9 compared to NEURON 8.2.4 #2787

Closed

JCGoran mentioned this pull request Oct 14, 2025

Segfault when running tests with PyNN #3638

Closed

Uh oh!

Conversation

olupton commented Oct 17, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

Uh oh!

codecov-commenter commented Nov 2, 2022 • edited by codecov Bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

Uh oh!

This comment was marked as outdated.

This comment was marked as outdated.

This comment was marked as outdated.

This comment was marked as outdated.

This comment was marked as outdated.

This comment was marked as outdated.

This comment was marked as outdated.

This comment was marked as outdated.

Uh oh!

azure-pipelines Bot commented Jun 5, 2023

Uh oh!

github-actions Bot commented Jun 5, 2023

Uh oh!

alexsavulescu left a comment

Choose a reason for hiding this comment

Uh oh!

This comment has been minimized.

github-actions Bot commented Jun 5, 2023

Uh oh!

azure-pipelines Bot commented Jun 5, 2023

Uh oh!

bbpbuildbot commented Jun 5, 2023

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

9 participants

olupton commented Oct 17, 2022 •

edited

Loading

codecov-commenter commented Nov 2, 2022 •

edited by codecov Bot

Loading