-
-
Notifications
You must be signed in to change notification settings - Fork 12.2k
Description
Describe the issue:
When trying to build a Python module for a legacy Fortran 90 code base, I think I came across a small bug in f2py.
When f2py is used with the only: directive to only selectively map routines, then private/public statements for subroutines in f90 modules will be mistaken as variables (the subroutines themselves will be filtered out at an earlier step). This leads to an exception in f2py because no type for those 'variables' can be inferred. In the example below, the module mod2 has two public subroutines, which will be filtered by a corresponding only: option for f2py. yet, the definitions
module mod2
implicit none
PUBLIC :: hey_do
PUBLIC :: do_hey
contains
will remain, leading to the problems described here.
My interpretation of this is: f2py will remove the body of subroutines that are not listed in the onlyfuncsvariable (in crackfortran.py), which then leads to the remaining artifacts in the declaration of the module being mistaken as variables (f90mod_rules.py).
I was able to work around this issue by adding a check into the corresponding function, which I believe filters out those remaining public/private declarations (while I'm not deeply familiar with Fortran 90 syntax, I believe that anything in the form PUBLIC :: [NAME] or PRIVATE :: [NAME]: must always refer to subroutines. Variables should always require a type?).
if len(var) == 1 and 'attrspec' in var and var['attrspec'][0] in ('public', 'private'):
This, however, lead to another issue: It is now possible that a module has no attributes or subroutines to export (either because everything is private, or filtered out due to only:). This leads to the generation of boilerplate C code for the empty module that crashes Python when the generated module is loaded. To solve this, I added another check that skips a given module if neither variables or anything else is to be exported.
The second issue can be prevented by either USEin the module, or by adding public dummy variables.
The work-around patch for current numpy HEAD (abeca76):
diff --git a/numpy/f2py/f90mod_rules.py b/numpy/f2py/f90mod_rules.py
index db53beaf61..5b4dcf2ff6 100644
--- a/numpy/f2py/f90mod_rules.py
+++ b/numpy/f2py/f90mod_rules.py
@@ -110,11 +110,20 @@ def dadd(line, s=doc):
notvars.append(b['name'])
for n in m['vars'].keys():
var = m['vars'][n]
- if (n not in notvars) and (not l_or(isintent_hide, isprivate)(var)):
+ if len(var) == 1 and 'attrspec' in var and var['attrspec'][0] in ('public', 'private'):
+ is_not_var = True
+ else:
+ is_not_var = False
+
+ if (n not in notvars and not is_not_var) and (not l_or(isintent_hide, isprivate)(var)):
onlyvars.append(n)
mfargs.append(n)
outmess('\t\tConstructing F90 module support for "%s"...\n' %
(m['name']))
+ if len(onlyvars) == 0 and len(notvars) == 1 and m['name'] in notvars:
+ outmess(f"\t\t\tSkipping {m['name']} since there are not vars/func...\n")
+ continue
+
if m['name'] in usenames and not contains_functions_or_subroutines:
outmess(f"\t\t\tSkipping {m['name']} since it is in 'use'...\n")
continue
Reproduce the code example:
$ cat tmod.f90
module mod2
implicit none
PUBLIC :: hey_do
PUBLIC :: do_hey
contains
subroutine hey_do()
implicit none
end subroutine hey_do
subroutine do_hey()
implicit none
end subroutine do_hey
end module mod2
module mod1
implicit none
private
PUBLIC :: func1
contains
subroutine func1()
end subroutine func1
end module mod1
$ f2py -c --backend=meson --build-dir output tmod.f90 only: func1
The Fortran code above compiles cleanly with gfortran:
(numpy) mweigand@c5c53cfebebc:~/t1$ cat main.f90
program main
use tmod
end program main
(numpy) mweigand@c5c53cfebebc:~/t1$ gfortran -Wall tmod.f90 main.f90Error message:
(numpy) mweigand@c5c53cfebebc:~/t1$ f2py -c --backend=meson --build-dir output tmod.f90 only: func1
Using meson backend
Will pass --lower to f2py
See https://numpy.org/doc/stable/f2py/buildtools/meson.html
Reading fortran codes...
Reading file 'tmod.f90' (format:free)
Post-processing...
Block: untitled
Block: mod2
Block: mod1
Block: func1
Applying post-processing hooks...
character_backward_compatibility_hook
Post-processing (stage 2)...
Block: untitled
Block: unknown_interface
Block: mod2
Block: mod1
Block: func1
Building modules...
Building module "untitled"...
Constructing F90 module support for "mod2"...
Variables: hey_do do_hey
getctype: No C-type found in "{'attrspec': ['public']}", assuming void.
Traceback (most recent call last):
File "/home/mweigand/.virtualenvs/numpy/bin/f2py", line 8, in <module>
sys.exit(main())
^^^^^^
File "/home/mweigand/.virtualenvs/numpy/lib/python3.12/site-packages/numpy/f2py/f2py2e.py", line 765, in main
run_compile()
File "/home/mweigand/.virtualenvs/numpy/lib/python3.12/site-packages/numpy/f2py/f2py2e.py", line 711, in run_compile
run_main(f" {' '.join(f2py_flags)} -m {modulename} {' '.join(sources)}".split())
File "/home/mweigand/.virtualenvs/numpy/lib/python3.12/site-packages/numpy/f2py/f2py2e.py", line 496, in run_main
ret = buildmodules(postlist)
^^^^^^^^^^^^^^^^^^^^^^
File "/home/mweigand/.virtualenvs/numpy/lib/python3.12/site-packages/numpy/f2py/f2py2e.py", line 401, in buildmodules
dict_append(ret[name], rules.buildmodule(module, um))
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/mweigand/.virtualenvs/numpy/lib/python3.12/site-packages/numpy/f2py/rules.py", line 1312, in buildmodule
mr, wrap = f90mod_rules.buildhooks(m)
^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/mweigand/.virtualenvs/numpy/lib/python3.12/site-packages/numpy/f2py/f90mod_rules.py", line 146, in buildhooks
at = capi_maps.c2capi_map[ct]
~~~~~~~~~~~~~~~~~~~~^^^^
KeyError: 'void'Python and NumPy Versions:
python -c "import sys, numpy; print(numpy.__version__); print(sys.version)"
2.0.0
3.12.4 (main, Jun 12 2024, 19:06:53) [GCC 13.2.0]
Runtime Environment:
import numpy; numpy.show_runtime()
[{'numpy_version': '2.0.0',
'python': '3.12.4 (main, Jun 12 2024, 19:06:53) [GCC 13.2.0]',
'uname': uname_result(system='Linux', node='c5c53cfebebc', release='6.1.0-22-amd64', version='#1 SMP PREEMPT_DYNAMIC Debian 6.1.94-1 (2024-06-21)', machine='x86_64')},
{'simd_extensions': {'baseline': ['SSE', 'SSE2', 'SSE3'],
'found': ['SSSE3',
'SSE41',
'POPCNT',
'SSE42',
'AVX',
'F16C',
'FMA3',
'AVX2'],
'not_found': ['AVX512F',
'AVX512CD',
'AVX512_KNL',
'AVX512_KNM',
'AVX512_SKX',
'AVX512_CLX',
'AVX512_CNL',
'AVX512_ICL']}},
{'architecture': 'Haswell',
'filepath': '/home/mweigand/.virtualenvs/numpy/lib/python3.12/site-packages/numpy.libs/libscipy_openblas64_-99b71e71.so',
'internal_api': 'openblas',
'num_threads': 12,
'prefix': 'libscipy_openblas',
'threading_layer': 'pthreads',
'user_api': 'blas',
'version': '0.3.27'}]
Context for the issue:
For legacy code bases, sometimes only a small subset of functionality needs to be exported to python. Potentially, the described behavior prevents the successful generation of Python interfaces for those packages.