(These are rough personal notes; don't be fooled by PEP-like wording)

Accessing the fields

[add some background info here]

Sam's plan was to use dynamic symbol lookup or weak symbols. I propose to not go with a platform-specific solution.

Some of the functions have Python API, e.g. Py_TYPE(o) is type(o) in Python. I'm not proposing that: users can redefine type from Python, breaking type safety.

Instead, I propose we add a capsule, sys._abi_compat. The capsule will contain a bit of version info and the addresses of new functions.

(Note that users can break this from Python, most easily by del sys._abi_compat. I'd say a Py_FatalError is fair in that case, though. To get wrong behaviour rather than an aborted process, you'd need to create a fake capsule.)

typedef {
    uint32_t version;  // set to Py_Version
    PyObject * (*func_Py_TYPE)(PyObject *o)
    Py_ssize_t (*func_Py_REFCNT)(PyObject *o)
    Py_ssize_t (*func_Py_SET_REFCNT)(PyObject *o)
    Py_ssize_t (*Py_SIZE)(PyObject *o)
    void (*Py_SET_SIZE)(PyObject *o, Py_ssize_t size)
} struct _Py_abi_compat_capsule;

• Get the capsule using PySys_GetObject & PyCapsule_GetPointer. Failure is a fatal error.
• If the capsule exists, get the result from it. (None of the fields may be NULL.)

Since none of these macros are expected to report exceptions, any failure means Py_FatalError.

static inline int
_Py_get_abi_compat_capsule(_Py_abi_compat_capsule **p_result)
{
    int result = -1;
    static _Py_abi_compat_capsule *result = NULL;

    static PyMutex mutex;  // (*if* this gets added to stable ABI)
    PyMutex_Lock(mutex);

    if (result) {
        result = 1;
        p_result = result;
        goto finally;
    }

    // Note that `sys` is special; we don't use `PyCapsule_Import`
    PyObject *capsule = PySys_GetObject("_abicompat")
    if (capsule) {
        result = (_Py_abi_compat_capsule*) PyCapsule_GetPointer(capsule, "sys._abicompat");
        if (!result) {
            Py_FatalError("sys._abicompat unavailable");
        }
        goto finally;
    }

    old_impl:
    PyObject *hexversion_obj = PySys_GetObject("hexversion");
    if (!hexversion_obj) {
        PyMutex_Unlock(mutex);
        Py_FatalError("sys.hexversion unavailable");
        goto finally;
    }
    long version = PyLong_AsLong(hexversion_obj);
    if (version < Py_LIMITED_API) {
        PyMutex_Unlock(mutex);
        if (!PyErr_Occurred) {
            Py_FatalError("sys._abicompat version mismatch");
        }
        goto finally;
    }
    if (version > Py_PACK_VERSION(3, 14)) {
        PyMutex_Unlock(mutex);
        Py_FatalError("sys._abicompat version mismatch");
        goto finally;
    }
    // result left NULL;
    result = 0;
finally:
    PyMutex_Unlock(mutex);
    return result;
}

Field-specific notes

Best way to get/set a PyObject field (assuming the capsule is added), by lowest limited API one needs to support:

• ob_refcnt
  • Increment/Decrement
    • 3.6+:
      • Py_IncRef/Py_DecRef have been functions since before 3.0
      • Refcounting macros (Py_[X]{INC|DEC|New}REF) will call Py_IncRef/Py_DecRef
    • 3.10+:
      • Py_[X]NewRef are exported functions; called directly
    • 3.12+:
      • Macros that don't take NULL (Py_{INC|DEC|New}REF) call _Py_IncRef/_Py_DecRef instead
    • Py_CLEAR and Py_[X]SETREF remain macros (C-only). (Add their expansion to the documentation, for benefit of non-C languages.)
  • Get (Py_REFCNT):
    • 3.6+: Use capsule
    • 3.14+: exported function; called directly
  • Set (Py_SET_REFCNT):
    • 3.6+: Use capsule
    • 3.14+: exported function; called directly.
• ob_type
  • Get (Py_TYPE)
    • 3.6+: Use capsule
    • 3.14+: exported function; called directly.
  • Set (Py_SET_TYPE):
    • Users should setattr __class__ instead; this includes checks.
    • 3.6+: Use capsule
    • new: exported function; called directly.
• ob_base - cast to PyObject*
• ob_size
  • Get (Py_SIZE)
    • 3.6+: Use capsule
    • new: exported function; called directly.
    • Note that many types make this available via PyObject_Size (len(o) in Python), or specialized functions (PyTuple_Size). These should be preferred; how an object uses ob_size for size information should generally be treated as its implementation detail.
  • Set (Py_SET_SIZE)
    • 3.6+: Use capsule
    • new: exported function; called directly.

Existing API:

• ob_refcnt
  • Increment/decrement:
    • 3.6+: Py_IncRef/Py_DecRef
    • 3.10+: Py_NewRef & Py_XNewRef
    • 3.12+: Py_INCREF/Py_DECREF macros (→ _Py_IncRef/_Py_DecRef in 3.10+; Py_IncRef/Py_DecRef)
    • macro: Py_XINCREF/Py_XDECREF (→ Py_IncRef/Py_DecRef)
  • Get:
    • 3.6+: sys.getrefcount(o) in Python
    • 3.14+: Py_REFCNT
  • Set:
    • NEW: Py_SetRefcnt
    • 3.13: Py_SET_REFCNT macro (→ _Py_SetRefcnt)
• ob_type
  • Get
    • type(o) in Python
    • 3.14+: Py_TYPE
  • Set
    • setattr __class__
    • 3.14+: Py_SET_TYPE`
• ob_base - cast to PyObject*
• ob_size
  • Get
    • NEW: Py_GetSize
    • Many types make this available via len(o)
    • Py_SIZE to call Py_GetSize, on failure clear the exception & return -1
  • Set
    • NEW: Py_SetSize
    • Py_GET_SIZE to call Py_SetSize, on failure clear the exception