changed 2 years ago  
Published Linked with GitHub

C-API

Actual changes

Delete all old macro names.

Intp/UIntp
Image Not Showing Possible Reasons
  • The image file may be corrupted
  • The server hosting the image is unavailable
  • The image path is incorrect
  • The image format is not supported
Learn More →

(True ABI break, but on almost no platforms, only OpenVMS?)

  • Redefine npy_uintp to size_t and npy_intp as Py_ssize_t (same size as size_t, C has no standard name). Yes, it is a misnomer.
  • Map characters 'P' and 'p' to current version for compat.
  • New character 'n' and 'N' (compatible with struct module and Python; yes I checked).
  • Notes:
    • On theoretical segmented platforms, this is not ABI compatible
      • For these platforms it will not be possible to distribute wheels targeting both old and new NumPy versions.
      • I am willing to bet it fixes more things than it breaks (i.e. these platforms don't work without patches anyway).
    • Casts of pointers to npy_intp will be technically wrong. Quick grep: NumPy/SciPy don't do this, it should be rare).

Opaquify (most of) PyArray_Descr struct

Assume Cython 3 availability.

  • descr->elsize: Force them to update to macro. (cython 3 not a problem)
    • SciPy/pandas haver very few uses
  • Legacy custom dtypes implementors:
    • Update code to PyArray_DescrProto struct (same layout)
    • Fetch actual instance from scalar type/type number
    • (May or may not have to change e.g. ufunc registration use)

MapIter
Image Not Showing Possible Reasons
  • The image file may be corrupted
  • The server hosting the image is unavailable
  • The image path is incorrect
  • The image format is not supported
Learn More →

All MapIter API should be private, its too messy. No known users exist and to iterate fast you need to dig deep into the structs. It makes cleaning up impossible:

  • PyArray_MapIterArray
  • PyArray_MapIterNext
  • PyArray_MapIterArrayCopyIfOverlap
  • PyArray_MapIterSwapAxes
  • PyArrayMapIterObject

Replacement: None Only user I found is Thaeno which is EOL, its replacement doesn't use it. Even in thaeno it is really just used to implement a slow version of np.add.at and arr[...] = vals.

Neighborhood Iterator

This would be nice to remove. It is quite messy and not really tested or used outside of SciPy, it also unlikely to generalize over all dtypes to begin with.

  • SciPy only uses NPY_NEIGHBORHOOD_ITER_ZERO_PADDING
  • Won't generalize to user dtypes easily
  • Almost untested
  • There was a single PR/issue fixed once: https://github.com/numpy/numpy/pull/18987 (which broke things since it is almost untested).

Replacement: Vedor into SciPy and hope SciPy eventually replaces it. The ndimage version is much better (but no object support and maybe subtly different).

Other iterators

Yes, I am sure it would be nice to slim out some there too, but I am less confident about that.

General functions/macros removal (mostly small)

(Deleted items that are already dealt with)

Symbol Replacement Note
NPY_CHAR Also all Python API
NPY_NUMUSERTYPES old API and seems useless
NPY_DEFAULT_ASSIGN_CASTING unnecessary info
NPY_NUM_FLOATTYPE unnecessary info
PyArray_FILLWBYTE normal fill/memset
PyTypeNum_ISPYTHON weird concept?
PyBigArray_Type Garbage
PyArray_SetStringFunction can readd if needed
PyArray_Cast PyArray_CopyInto no doc
PyArray_CastAnyTo PyArray_CopyInto no doc
PyArray_CopyAnyInto PyArray_CopyInto doesn't ravel, no doc
PyArray_MoveInto PyArray_CopyInto
PyArray_ScalarAsCtype buffer or PyArray_Pack too specific to NumPy scalars
PyArray_CastScalarToCtype PyArray_Pack used in swig
PyArray_CastScalarDirect PyArray_Pack
PyArray_ConvertToCommonType
PyArray_Zero/PyArray_One New API SciPy uses it, so maybe not, but bad API
PyArray_GetCastFunc New API (no priority) nonsensically used in pandas
PyArray_Reshape PyArray_Newshape (or call Python) NewShape doesn't use tuple
PyArray_DescrFromTypeObject DescrCoverter undocumented
PyArray_CompareString vendor it undocumented+unused
PyArray_CompareUCS4 vendor it undocumented+unused
PyArray_ScalarFromObject deprecated+undocumented
PyArray_GetField remove also arr.getfield (use indexing)
PyArray_SetField remove also arr.setfield (use indexing)
PyArray_UpdateFlags Maybe not, but shouldn't really be used
PyArray_FillObjectArray PyArray_FillWithScalar or empty/ones/zeros …
PyArray_FromInterface FromAny Just doubt its used/useful (takes object not interface itself)
PyArray_HasArrayInterface FromAny
PyArray_HasArrayInterfaceType FromAny
PyArray_FromStructInterface FromAny Just doubt its used/useful
PyArray_FromArrayAttr FromAny Has "context" that is ignored.
PyArray_CheckStrides Can keep but seems unnecessary unless you allow user input for strides.
PyArray_ScalarKind Outdated concept
PyArray_MinScalarType ResultType (hopefully) Outdated concept
PyArray_CanCoerceScalar Outdated concept
PyArray_CanCastScalar Not documented; Outdated concept and no error return
PyArray_NewFlagsObject undocumented, Why public?
PyArray_FieldNames Looks wrong, dict-order is important not offset order.
PyArray_ElementStrides (easy to roll own) not useless but also not document and simple, used once, we don't even reuse interally…
PyArray_SortkindConverter, PyArray_ClipmodeConverter, PyArray_SearchsideConverter, PyArray_CastingConverter, PyArray_ConvertClipmodeSequence These parese Python input to C-flags and thus are very NumPy specific (not all are!)
PyArray_CheckAxis Manual (not sure, but seems a bit niche)
PyArray_TypestrConvert use np.dtype(...) C-api
PyArray_TypeNumFromName not even used in NumPy
PyArray_NeighborhoodIterNew Very limited old cruft for SciPy can we move it?
PyUFunc_f_f_As_d_d, PyUFunc_Om_On, … not even scipy uses these, they are outdated.
PyUFunc_ReplaceLoopBySignature pnumpy only, would be nice to remove. Yes, we need legacy fallback for scipy in ufuncs, but this could be one less thing to worry about and needs better API anyway…
PyUFunc_DefaultTypeResolver public?!

Limited API (may only work with builtin types for example)

Functions that use type numbers could be hidden behind a warning. But, I don't think this is a priority for a 2.0. It can be done at any point.

PyArray_INCREF PyArray_XDECREF PyArray_Item_INCREF PyArray_Item_XDECREF

PyArray_DescrFromType

PyArray_CanCastSafely

PyArray_DescrFromScalar

PyArray_FromScalar

PyArray_ValidType

PyArray_EquivTypenums

Opaquify?

If we can rely on Cython 3, and agree on compile against "NumPy 2" we can opaquify structs quite liberally. (Note: I am happy with not making the array struct opaque, I feel that changes to the "typical" fields probably mean more interesting behavior changes. And I suspect that will mean a new type)

  • PyArrayFlags_Type
  • PyArrayIter_Type
  • PyArrayMultiIter_Type
  • …

ABI/API changes that I can think of

  • Return value updated, e.g.:
    • Bug magnet
    • NpyIter API often uses boolean false rather than -1 for error signalling.

Affects: Very rather few, the new iterator API never really took off.

How: Breaks things hard, impacted code will report errors where there are none (a bit hard to track down). Fix itself is easy.

New API

We have a bit of new API, which we could (with small limitations) "backport" header only:

  • PyArray_ClearBuffer (decref+memset)
  • PyArray_Pack (probably just SETITEM, that is wrong, but what everyone uses anyway)
  • … others?
Select a repo