Summary

Functions defined in the python interpreter's main module are not found by lammps.
E.g. python end_of_step_callback input 1 SELF format p exists always fails if the function is defined in __main__.
Minimal reproducible example below.

LAMMPS Version and Platform

LAMMPS (23 Jun 2022), compiled from source
OS: OSX 13.0
Architecture: arm64
Python : 3.10.6
Python distribution: conda-forge

Expected Behavior

When called from python via the python library, lammps allows calling and using in fixes python functions, by first assigning them a name and specifying their argument format; this functions can be defined with the same name in the main module.
Documentation here https://docs.lammps.org/python.html .

Actual Behavior

Exception thrown here

Steps to Reproduce

Download and compile from source the same lammps version (I used conda-forge compilers and tools).
Run the script below:

import lammps

script = """
units		lj
atom_style	atomic

#setup test data
lattice		sc 1
region		box block 0 3 0 2 0 2
create_box	3 box # (number of atom types)
create_atoms	1 box
mass		* 1.0

python end_of_step_callback input 1 SELF format p exists

"""


def end_of_step_callback(ptr):
    lmp = lammps.lammps(ptr=ptr)

def main():
    lmp = lammps.lammps(cmdargs=['-log','none','-nocite','-echo','log'])
    
    lmp.commands_string(script)

if __name__=='__main__':
    main()

Output:

#output
"""
LAMMPS (23 Jun 2022)
Lattice spacing in x,y,z = 1 1 1
Created orthogonal box = (0 0 0) to (3 2 2)
  1 by 1 by 1 MPI processor grid
Created 12 atoms
  using lattice units in orthogonal box = (0 0 0) to (3 2 2)
  create_atoms CPU = 0.000 seconds
AttributeError: module '__main__' has no attribute 'end_of_step_callback'
ERROR: Could not find Python function end_of_step_callback (src/PYTHON/python_impl.cpp:246)
Last command: python end_of_step_callback input 1 SELF format p here ""
Traceback (most recent call last):
  File ".../lammps_python_invoke_test.py", line 34, in <module>
    main()
  File ".../lammps_python_invoke_test.py", line 31, in main
    lmp.commands_string(script)
  File "$PREFIX/lib/python3.10/site-packages/lammps/core.py", line 617, in commands_string
    with ExceptionCheck(self):
  File "$PREFIX/lib/python3.10/site-packages/lammps/core.py", line 49, in __exit__
    raise self.lmp._lammps_exception
Exception: ERROR: Could not find Python function end_of_step_callback (src/PYTHON/python_impl.cpp:246)
Total wall time: 0:00:00

"""

Further Information, Files, and Links

Either I am forgetting something in the parameters for the python command, in which case I am really sorry for everyone's time, or something weird is going on with __main__.

I looked at #3204 ; the python I compiled against has a shared library in $CONDA_PREFIX/lib/libpython3.10.dylib.
However the python executable itself python3.10 does not reference this library, while liblammps.dylib does.
from distutils import sysconfig; print(sysconfig.get_config_var('Py_ENABLE_SHARED')) prints 0.

>otool -L $(which python)
$CONDA_PREFIX/bin/python:
	/usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1292.0.0)
>otool -L $PREFIX/lib/liblammps.dylib
liblammps.dylib:
	@rpath/liblammps.0.dylib (compatibility version 0.0.0, current version 0.0.0)
	@rpath/libjpeg.9.dylib (compatibility version 15.0.0, current version 15.0.0)
	@rpath/libpng16.16.dylib (compatibility version 55.0.0, current version 55.0.0)
	@rpath/libpython3.10.dylib (compatibility version 3.10.0, current version 3.10.0)
	/usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1319.0.0)
	@rpath/libc++.1.dylib (compatibility version 1.0.0, current version 1.0.0)

Build script:

#!/bin/bash
SRC_DIR=$(pwd)

BUILD_DIR=${SRC_DIR}/build
mkdir "${BUILD_DIR}"
cd "${BUILD_DIR}" || (
	echo "can't move to build directory"
	exit 1
)

cmake \
	-D BUILD_MPI=no \
	-D BUILD_OMP=no \
	-D PKG_PYTHON=on \
	-D BUILD_SHARED_LIBS=on \
	-D LAMMPS_EXCEPTIONS=yes \
	-D PKG_OPENMP=yes \
	-D CMAKE_INSTALL_PREFIX="${PREFIX}" \
	"$SRC_DIR"/cmake

make -j$(nproc)

make install