# F2PY compilation study

- Started by reading [name=Ralf]'s article on [moving SciPy to Meson](https://labs.quansight.org/blog/2021/07/moving-scipy-to-meson/). It lead me to SciPy's GitHub Issue [RFC:switch to Meson as a build system](https://github.com/scipy/scipy/issues/13615)
- This article specifies in great detail features needed from a build system and how meson fulfils them.

**Doubt**
- I was going through the `run_compile()` method, and it doesn't use `crackfortran()`. It just takes input source files and sends it to `distutils` to compile and produce the extension module. So is the C wrapper created again inside `distutils`?
- `-m` flag specifies module name, which is assumed `untitled` if not given (although I saw some [wierd code](https://github.com/HaoZeke/numpy/blob/main/numpy/f2py/f2py2e.py#L606-L613) tying to find module name in source file). If `f2py -c source.f` produces untitled shared library, shouldn't `f2py source.f` produce `untitledmodule.c` by assuming module name 'untitled'.
- Why the parser handling `-h` flag in `f2pyarg.py` named wrapper_generation

### Numpy distutils study
- Normal `distutils` cannot compile fortran sources.
- `numpy.distutils` uses `f2py` to compile the fortran source files. How does it do that?
1. I thought `distutils` is a dependency of `f2py` but actually, is it the opposite? Because `distutils` requires `f2py` to generate C/API module so that it can compile it to produce shared library?
2. How does compiling both - the fortran source file and C/API module works. I thought `compilers` are language specific. A compiler reads one program to one machine code file. How is it that two different programs are being compiled to produce a single `.so` file?
- [name=Rohit] So that's the job of the linker. We set two languages which means it looks for two compilers. Then if you look in the build directory you see two object (o) files. These are then linked into a single so. You can inspect the object files to see that only the compiled symbols from one language are present using name mangler (nm). [Link to blog](https://rgoswami.me/posts/iso-c-type-bound-fortran/)

### F2PY meson documentation study
**[Documentation](https://numpy.org/doc/stable/f2py/buildtools/meson.html) requires a few changes**
- `python -n numpy.f2py...` ---> `python -m numpy.f2py...` at multiple places.
- The first `meson.build` file in demonstration is incorrect. Correct build code will be:

project('f2py_examples', 'c',
  version : '0.1',
  default_options : ['warning_level=2'])

add_languages('fortran')

py_mod = import('python')
py3 = py_mod.find_installation('python3')
py3_dep = py3.dependency()
message(py3.path())
message(py3.get_install_dir())

incdir_numpy = run_command(py3,
  ['-c', 'import os; os.chdir(".."); import numpy; print(numpy.get_include())'],
  check : true
).stdout().strip()

incdir_f2py = run_command(py3,
    ['-c', 'import os; os.chdir(".."); import numpy.f2py; print(numpy.f2py.get_include())'],
    check : true
).stdout().strip()

inc_np = include_directories(incdir_numpy, incdir_f2py)

py3.extension_module('fibby',
           'fib1.f',
           'fibbymodule.c',
           incdir_f2py+'/fortranobject.c',
           include_directories: inc_np,
           dependencies : py3_dep,
           install : true)

This code will only need the the previously generated C wrapper file `fibbymodule.c`, and will generate `ImportError: /home/namami2011/dev/fortry/builddir/fibby.cpython-310-x86_64-linux-gnu.so: undefined symbol: FIB_` error upon importing the shared library explainind the need for `--lower` flag.
- [name=Rohit] The documentation is structured in a narrative manner and discusses the symbol issue.
- Change `import fib2` to `import fibby` at multiple places in doc.

## F2PY compilation flag equivalents

- **Problems**
- Meson accepts only a single compiler for 'Fortran' and does not distinguish between f77 or f90 files.

Can we deprecate `--f77exec`, `--f90exec`, `--f77flags`, `--f90flags`. Because if a growing modern build system doesn't provide functionality like this, why should we?
- [name=Rohit] Because of legacy code and existing systems. Even if growing systems don't care we need to have backwards compatibility.
In any case, fortran 77 and fortran 90 sources will be treated the same way, and will be compiled by a single compiler. Should we use `custom_target` in meson if we want to preserve this feature?
- [name=Rohit] This is actually not true. Due to parsing differences, many compilers always have different F77 and F90 variants. However, the `custom_target` is a good idea.

I am not sure if compiler path can be passed to `FC` env variable.
- [This issue](https://github.com/mesonbuild/meson/issues/1534) mentions that meson uses C linker when compiling a mix of Fortran and C files, but fortran linker should be used instead. (This entire issue maybe related to our case)
- [name=Rohit] No that's not exactly correct. The linker and resolution of symbols is language agnostic. However, things which need `-lgfortran` for example, need the fortran linker. I'm not sure how or where this issue fits with the current project.

- `--compiler` has no [documentation](https://numpy.org/doc/stable/f2py/usage.html)

| Flag | Alternative |
| -------- | -------- |
|`--help-fcompiler`, `--help-compiler`| Discussed below |
|`--fcompiler`| [Set FC Flag](https://mesonbuild.com/Reference-tables.html#compiler-and-linker-selection-variables)
|`--f77exec`, `--f90exec`| [Set FC Flag](https://mesonbuild.com/Reference-tables.html#compiler-and-linker-selection-variables) for now. Will use [custom_targets](https://mesonbuild.com/Custom-build-targets.html) after prototype is finished. `--fcompiler` takes precedence.
|`--compiler`| Set CC Flag |
|`--include-paths`, `-I`| `include_directories` in `py3.extension_module`|
|`--f77flags`, `--f90flags` | Pass [global args](https://mesonbuild.com/Adding-arguments.html#global-arguments) to fortran compiler |
|`--link-<resource>` | Add [dependency](https://mesonbuild.com/Dependencies.html#dependencies) in build file |
| Debug and optimization **defaults**| By default pass `-Ddebug=false` and `-Doptimization=3` to meson [buildtype](https://mesonbuild.com/Builtin-options.html#core-options)|
| `--debug` | Pass `-Ddebug=true` to meson |
| `--opt=`, `--arch=` | Pass [global args](https://mesonbuild.com/Adding-arguments.html#global-arguments) to C and Fortran compilers|
| `--noopt` | `-Doptimization=0` |
| `--noarch` | `-Doptimization=2` |
| `-L<dir> -l<libname>` | pass to linker argument `py3.extenion_module(..., link_args=['-L<dir>', '-l<libname>'`|
| `-D<macro>` | Pass to `c_args` |
| `-Uvar=value` | Pass to `c_args` as `-Dvar=value` |

- `--help-fcompiler, --help-compiler`
Regarding compiler detection using meson, I have been trying [this method](https://github.com/mesonbuild/meson/blob/b49b9f52b29896cce58a1e3dbeb1b6cf54420d45/mesonbuild/compilers/detect.py#L205)
**Try method 1:** Only get a single compiler

    from mesonbuild.compilers.detect import compiler_from_language
    from mesonbuild.environment import Environment
    import argparse
    opts = argparse.Namespace()
    opts.native_file = []
    opts.cross_file = None
    opts.wrap_mode = None
    opts.prefix = '/'
    opts.cmd_line_options = {}
    env = Environment(None, None, opts)
    c = compiler_from_language(env=env, lang='fortran', for_machine=0)
    >>> c
    <GnuFortranCompiler: v9.4.0 `/home/namami2011/mambaforge/envs/scipy-meson/bin/x86_64-conda-linux-gnu-gfortran`>
    # This c guy has a lot of info including linkers
    >>> c.full_version, c.exelist,
    ('GNU Fortran (GCC) 9.4.0', ['/home/namami2011/mambaforge/envs/scipy-meson/bin/x86_64-conda-linux-gnu-gfortran'], 0)
    >>> c.linker.exelist, c.linker.id, c.linker.prefix_arg
    (['/home/namami2011/mambaforge/envs/scipy-meson/bin/x86_64-conda-linux-gnu-gfortran'], 'ld.bfd', '-Wl)

With this other method I only got a single compiler. However I have `cc`, `gcc`, `clang` installed. Not able to detect all of them.

I thought maybe, `compiler_from_language` gives only the default compiler. Tried to check its parent function `_get_compilers`, thats when I spotted this comment [L242: # Return value has to be a list of compiler 'choices'](https://github.com/mesonbuild/meson/blob/b49b9f52b29896cce58a1e3dbeb1b6cf54420d45/mesonbuild/compilers/detect.py#L242)

**Try method 2:** Emulate `_get_compilers` --> Still getting single compiler:

    from mesonbuild.environment import Environment
    import argparse
    opts = argparse.Namespace()
    opts.native_file = []
    opts.cross_file = None
    opts.wrap_mode = None
    opts.prefix = '/'
    opts.cmd_line_options = {}
    env = Environment(None, None, opts)
    value = env.lookup_binary_entry(0, 'c')
    >>> value
    ['/home/namami2011/mambaforge/envs/scipy-meson/bin/x86_64-conda-linux-gnu-cc']

Still got a single compiler for 'C'. But I was supposed to get a list of all available compilers for C. Every function in meson for detecting compilers for any languages uses `_get_compilers`.

**Final Try**: Modified [_detect_c_or_cpp_compiler](https://github.com/mesonbuild/meson/blob/b49b9f52b29896cce58a1e3dbeb1b6cf54420d45/mesonbuild/compilers/detect.py#L359) to return an array of all `cls` objects detected i.e. the compilers it detects using `_get_compilers` method.
**Result**: Still getting only one compiler.

Should I talk to the maintainers regarding this?
- [name=Rohit] Not before checking the results after setting `FC` / `CC` variables

- **[Native file ref](https://mesonbuild.com/Machine-files.html#properties)**
Can be used to specify compiler path and flags. We will not need if `FC` and `CC` are able to handle compiler paths. Else we will provide compiler paths in native file using `fortran = path/to/fortran/compiler/`,`c = path/to/C/compiler/` and `fortran_args=['list', '--of', 'flags']` and `c_args` similarly.

**Doubts**
Are `--opt` and `--arch` flags passed to the fortran compiler directly?