Try   HackMD

NumPy2 lighning talk: Change np.int_ to np.int64 on all platforms

Today np.int_ reflects the C long type, which is 32-bits for MSVC (windows) and 64-bits on all other 64-bit platforms, and 32-bits on 32-bit platforms. The relevant issues discussing this: 9464, 12332, 17531 (which points out that the canonical name for np.int32 on win64 is np.intc), 17640 18540

Implications for Array API compatibility

Under Dtypes, the Array API spec says:

The default integer data type should be the same across platforms, but the default may vary depending on whether Python is 32-bit or 64-bit.

So this change would bring NumPy into compliance with the Array API

Who is in favor?

The difference confuses some users, who expect np.array(2**31) to have the same dtype as np.array(2**30) on all platforms. There were comments from the CuPy developers on one of the linked issues that they needed to adjusts tests for windows because of this.

What will break?

Cython imports npy_long and uses it. We would have to check the implications of making npy_long 64-bits everywhere, or if there would be problems with inconsistencies between npy_long and np.int_.

Why not make the change?

It would be disruptive. In the best case, SciPy users (to take a concrete example) would silently get different results on windows when updating NumPy to 2.0. In a worse case, their programs might crash if this affects Cython.

We could develop a strategy to migitate this with a legacy code-path when import_array() is called in Scipy, which would cause all SciPy code to use the older dtype and prevent crashes, but that is really complicated and might not work.

Is it worth it?

<Discussion></Discussion>

Decision

<Discussion></Discussion>

Last changed by 
 
Matti Picus
2
452

Read more

NEP 53 — Deciding on adoption Google Highway to underpin NumPy's SIMD functionality

switching the C++ wrappers from universal intrinsics to Highway, but in any case the work to move to C++ is not completemeson compiler switches or attributes (with a subtopic of MSVC). As far as I could tell there were concerns about CPU feature detection as well but I am not sure that is an issue since we need some kind of meson feature detection anywayspecifying features for runtime dispatch or more to the point to prevent runtime dispatch on certain featuresrewriting existing kernels using Highway intrinsics, but in any case we need to refactor to use C++dispatching inner loops on selected features: a desire to setup dispatching only once and not on every kernel callrewriting the _simd testing module to use C++ and to Highway

Jul 3, 2024
NumPy2 lighning talk: Change `NPY_MAX_DIMS` to 64 (from 32)

Originally proposed in issue 5744, and on the mailing list. The motivation seem a bit unclear. There is a clear use case for this in simulating quantum systems. While 64 is better than 32, it is not clear that will be the end of this discussion. In the issue there is a suggestion to make the limit dynamic. In PyArrayObject the value is not used, but it is used heavily in the structs behind PyArrayIterObject. PyArrayMultiIterObject, PyArrayMapIterObject, PyArrayNeighborhoodIterObject. It is also used as a shortcut in code like npy_intp dimensions[NPY_MAXDIMS]; for (i=0; i&lt;PyArray_NDIMS(arr); i++) { dimensions[i] = PyArrayDIMS(arr, i); } /* Some code that broadcasts or otherwise manipulates &quot;dimensions&quot; */ ret = new_array(..., dimensions, ...); SciPy has about 30 such uses.

Apr 3, 2023
NumPy2 lighning talk: Refactor `np.MaskedArray` to use `__array_function__` instead of inheriting from `np.ndarray`

Even with all the checks added for MaskedArrays inside NumPy code, there are still issues around properly propogating masks. In PR 22914, @greglucas proposed adding MaskedArray.__array_ufunc__ building on work done in other PRs that were subsequently reverted. On the mailing list, Allan Haldane announced a third-party implementation of MaskedArray using __array_function__ https://github.com/ahaldane/ndarray_ducktypes. The post mentions two open problems: what to do about the mask of arr.imag = MaskedArray([1,X,1]) how to ducktype scalars when using __array_function__ ? Do we always need an associated &quot;MaskedScalar&quot; type? In the PR, @greglucas says: My desires from a new class:

Apr 3, 2023
Work package 1 "Faster emulation of the RISC-V Sail model: Build Tooling, Documentation, CI Integration"

Using the Sail RISC-V model for emulation and testing currently has significant downsides compared to other existing emulators for RISC-V, such as Spike or QEMU. Specifically, the Sail-based emulators are 1000x-5000x slower, which makes them unworkable for many applications. This is one of the main pain-points of using the Sail model in RISC-V development. Pydrofoil (PyPy&apos;s Sail backend) [1] significantly increases the speed of the RISC-V Sail model. However, Pydrofoil is not currently easily usable by the RISC-V community: it is not documented and hard to use by anyone except the authors. In this work package we propose to provide build tooling,

Nov 11, 2022
Read more from Matti Picus
Published on HackMD