The logic for handling Python files is mostly encompassed in python/CMakeLists.txt. To ensure that the new src directory is handled correctly, we need:

• Copy the directory in the build folder.
• Add the files in src as a cleanup target so that when you run ninja clean, those files are removed from the build directory.
• Ensure that the files in src are installed in the installation prefix path (if needed).

Something like this should work:

diff --git a/python/CMakeLists.txt b/python/CMakeLists.txt
index 46187b89c..f555c197b 100644
--- a/python/CMakeLists.txt
+++ b/python/CMakeLists.txt
@@ -44,11 +44,13 @@ add_custom_command(
     TARGET CopyPythonFiles
     COMMAND ${CMAKE_COMMAND} -E copy_directory
         "${CMAKE_CURRENT_SOURCE_DIR}/lib" "${PROJECT_BINARY_DIR}/lib"
+    COMMAND ${CMAKE_COMMAND} -E copy_directory
+        "${CMAKE_CURRENT_SOURCE_DIR}/src" "${PROJECT_BINARY_DIR}/src"
 )
 
 set_target_properties(CopyPythonFiles 
     PROPERTIES ADDITIONAL_CLEAN_FILES
-        "${PYTHON_BUILD_BIN_FILES};${PROJECT_BINARY_DIR}/lib"
+        "${PYTHON_BUILD_BIN_FILES};${PROJECT_BINARY_DIR}/lib;${PROJECT_BINARY_DIR}/src"
 )
 
 install(DIRECTORY ${CMAKE_CURRENT_SOURCE_DIR}/bin/
@@ -58,7 +60,9 @@ install(DIRECTORY ${CMAKE_CURRENT_SOURCE_DIR}/bin/
     PATTERN ".pyc" EXCLUDE
 )
 
-install(DIRECTORY ${CMAKE_CURRENT_SOURCE_DIR}/lib/
+install(
+    DIRECTORY 
+        ${CMAKE_CURRENT_SOURCE_DIR}/lib/ ${CMAKE_CURRENT_SOURCE_DIR}/src/
     DESTINATION ${CMAKE_INSTALL_LIBDIR}
     USE_SOURCE_PERMISSIONS
     PATTERN "__pycache__" EXCLUDE

One thing I would like to comment on is that, in my opinion, setting the PYTHONPATH should not be treated as a backup option, but rather should be the principal way through which a module is located. This variable is a standard across the entire Python ecosystem, and if we have the means to make our workflow work by relying on this, we should embrace it fully and assume it to be our default method. In fact, to deal with the build folder structure, we could even configure CMake to write to the Python files and modify the PATH from within a script (by adding the corresponding directory which contains the Python module).

The issue we've had with PYTHONPATH in the past is specific to developers rather than end users. Many of us have multiple parallel MRtrix3 installations using different code branches, and some of these can have modifications to Python scripts and/or API. Therefore, if a Python script is executed, it's vital that the API from the corresponding code branch be utilised.

Updating to a cmake universe:

Perhaps if someone has gone through the cmake installation step, then it would be reasonable to have that added to PYTHONPATH. For developers more likely to be running from the build directory, I don't see how PYTHONPATH could be set for the user without leading to version conflicts.

Though I think you're later suggesting something else. If I'm following that correctly, it's not utilisation of PYTHONPATH but rather hard-coding absolute path for module import at build / installation time, then my current intuition is that I'd prefer the #2741 solution. Because that involves doing such hard-coding within a standalone executable and not modifying the source files, there's the prospect of it working with the use of softlinks at build time, which would allow editing of source code and testing without invoking cmake. So that argument might be better made in contrast to #2741. Hopefully I can have a decent shot at updating cmake to realise that particular vision of structure, and then we can better assess pros and cons.

Just to add a couple of comments on top of @Lestropie's:

One thing I would like to comment on is that, in my opinion, setting the PYTHONPATH should not be treated as a backup option, but rather should be the principal way through which a module is located.

We've discussed this quite a few times in the past, and always came to the conclusion that we shouldn't rely on it. We're far from alone in that particular conclusion by the way - see these blogs on the issues others have with PYTHONPATH, which I think are relevant for us too:

• https://orbifold.xyz/pythonpath.html
• https://levelup.gitconnected.com/import-your-own-python-code-without-pythonpath-tricks-9068495c1bba

Personally, I think it should be avoided like the plague, for very similar reasons to why LD_LIBRARY_PATH is strongly discouraged (see e.g. here or here)...

Superseded by #2920.