owned this note   changed 8 days ago  
Published Linked with GitHub

Video call link: https://meet.google.com/jrp-qgcf-nxh
If not working: https://colgate.zoom.us/j/276854695

Monday May 12, 2025

Attendees: No meeting

Release schedule for SciPy 1.16:

  • Timetable: maint branch May 17.
  • Should we add milestone labels for the nD PRs? Others? (Dan'll add them – can remove them if needed)

PRs Ready for Review:

Think about

  • Right now eq and ne return True/False (not an array at all!) if shapes do not match!? Shouldn't we raise? Can we deprecate/remove this for sparray?
  • has_canonical_format:
    • in test suites for SciPy, networkx, scikit-image, scikit-learn, and cvxpy every call to _binopt has matching canonical structure and has_canonical_format property. We could probably use the property in place of the check. :rocket:
    • The tests added in the above PR show how it might occur that they not agree. Basically, someone has to set the property incorrectly, or mess with the inner arrays after 1st accessing the property. Once the cache of the flag is made, further changes to inner arrays are not protected. Most people dont access the property in their code, so the caching doesnt cause a problem.
  • Many astype w/o copy kwarg could be copy=False. I changed a couple. More in a later PR.

Monday April 28, 2025

Attendees: Dan, CJ, Stéfan

PRs Ready for Review:

  • PR 22873: nD with unary functions
    • sum/mean/min/max/argmin/argmax
    • refactor nD->2D conversion to a helper func
    • refactor dot/tensordot/matmul to use helper
    • Other changes:
      • switched int axis to tuple in validateaxis
      • Added more tests for min/max(explicit=True)cuz different codepath
      • When axis=None, use sputils._todata().sum() instead of self @ ones
      • assign to out instead of res.sum(axis=(), out=out). (Feels closer to intent of out…??)
  • PR 21743 test_base.py split.
    • Removed list of tests that dont use get_index_dtype and tests that the list was correct
  • PR 22897 nD binopt users:
    • eq/ne/gt/ge/lt/le
    • multiply/_divide
    • maximum/minimum
    • refactor _convert_to_2d tools
    • refactor dot/tensordot/matmul
    • other changes
      • Pass around op in comparison/bool funcs. Use op.name when calling _binopt. Use op(0, other) for whether to warn.
      • switched long names to argminmax, arminmax_axis

Think about:

  • has_canonical_format: in test suite _binopt is called ### times with ### attribute and check both True amd ### both False. :rocket:

  • Right now eq and ne return True/False if shapes do not match!?!? Shouldn't we raise? Can we deprecate/remove this for sparray?

  • Many astype w/o copy kwarg could be copy=False. I changed a couple. More in a later PR.

  • 21925 broadcasting of _binop at C++ level

    • row broadcasting is easy. col requires a few differences
    • do we put col broadcasting into the existing functions, or as a separate func?
    • Idea: create small C++ structure to hold shape and/or stride info. Perhaps like:
int ndim; 
int nnz;
int shape[];   // length ndim 
int strides[]; // length ndim     (or compute these)
I indptr[];    // length shape[0] + 1
I indices[];   // length nnz
T data[];      // length nnz
  • 19984 binop general/canonical discussion (Nothing implemented since last mtg)
    • Does this capture last mtg discussion?
      • Would be nice to capture the egregious cases and boost doc warnings. Egregious cases (based on code) when the vector created in General is long and sparse.

Monday April 14, 2025

Attendees: Dan and CJ

PRs Ready for Review:

  • Ready for review/merge (approved already):

    • 22647 validateaxis :tada: MERGED
    • 22527 add returns sections to construct.py function :tada: MERGED
    • 22169 refactor CSC to use CSR sparsetools :tada: MERGED

  • Think about:

    • Next nD Steps:
      • PR has sum/mean, min/max, binopt, multiply, add, sub, minimum/maximum
      • ?split out sum/mean/min/max from bigger PR?
      • add other nD features:
        • CSR support (or new format name csx to do csr in nD)
        • indexing?
        • construction functions?
        • array_api nD functions like expanddims
    • 21925 broadcasting of _binop at C++ level
      • row broadcasting is easy. col requires a few differences
      • do we put col broadcasting into the existing functions, or as a separate func?
      • if we split col broadcasting, do we support both general and canonical for this case?
    • 19984 binop general/canonical discussion
      - simple heuristic to pick general or sort+canonical
      - or docs to push users to choose
      - Let's aim for this: Would be nice to capture the egregious cases and boost doc warnings. Egregious cases (based on code) when the vector created in General is long and sparse.
    • 21743 test_base.py split.  2 issues:
      • right now Test64bit sparray tests are run when "slow" tests are run. spmatrix tests always.
      • Test64bit is hard to separate from the rest of test_base.py when testing
      • Q1: should we turn on sparray beyond "slow" tests? (Ans: Yes) Should we change some spmatrix to be slow? (Ans: No – we will remove them with spmatrix so won't be long. And we'll see if people complain.)
      • Q2: is it worth splitting out Test64Bit  to a separate module to allow partial testing? (Ans: Yes)

Monday March 17, 2025

Attendees: CJ, Stefan

PRs Ready for Review:

  • Ready for review/merge (*approved already):

    • 22647 validateaxis
    • 22535 * add returns sections to construct.py function
    • 22169 refactor CSC to use CSR sparsetools

  • Read through (easy?)

    • 22535 update roadmap sparse

  • Think about:

    • 21925 broadcasting of _binop at C++ level
      • row broadcasting is easy. col requires a few differences
      • do we put col broadcasting into the existing functions, or as a separate func?
      • if we split col broadcasting, do we support both general and canonical for this case?
    • 21743 test_base.py split.  2 issues:
      • right now Test64bit sparray tests are run when "slow" tests are run. spmatrix tests always.
      • Test64bit is hard to separate from the rest of test_base.py when testing
      • Q1: should we turn on sparray beyond "slow" tests? should we change spmatrix at all?
      • Q2: is it worth splitting out Test64Bit  to a separate module to allow partial testing?
    • 19984 binop general/canonical discussion
      - simple heuristic to pick general or sort+canonical
      - or docs to push users to choose

Monday March 3, 2025

Attendees: Dan, CJ

Active PRs and Issues

Monday February 17, 2025

Attendees: CJ, Dan

News:

PR discussions, Topics/Questions:

  • nD features:
    • PR 22540: nD binop split from broadcasting
      • coo copies python code for binopt & minmax from _compressed.py and _data.py
      • minmax calls mixin methods without subclassing. works, but why not subclass?
    • PR 21613: broadcasting with copies
    • Currently nD-add/sub creates duplicates without calling sum_duplicates
    • Q: should CSR support nD with an axis arg? Or call it something other than CSR? (1D currently uses the name CSR)
  • Q: How to reliably time binop sparsetools?
    • Issue 19984: predictably slow "add"
      • Considering general vs sort & canonical, general is predictably slower for some parameter regions. (most of the time it is faster)
      • Posts from commit 618c4ea and #1385 (comment) suggest that nbell put general back in due to an incorrect timing reported on the issue.
      • Need to check whether sorting followed by canonical is faster than general. My timing comment shows that the general method is usually faster for unsorted indices. But some conditions make it slower than sort & canonical. The conditions seem to be large M with small nnz. But large M suggests should switch to CSC format.
    • PR 21925: broadcasting without copy will need timing for this
      • Currently general & canonical
      • broadcast indptr can be easily included
      • broadcast indices could be separate func or separated in an if clause.
      • C++ changes: PR 21925 csr.h
    • PR 22169: CSC binop via CSR and _swap Timing? (no difference? but how to check that)

Follow-up items for 1.16

  • dont copy when broadcasting
  • support int32 index arrays for nD
  • improve coo_todense_nd performance?
  • avoid assigning to self? Maybe assign self/other -> A/B
  • choose priority between CSR nD and broadcasting w/o copy

Monday December 9, 2024

Attendees: Dan, CJ, Stéfan

News:

Follow-up items for 1.16

  • dont copy when broadcasting
  • support int32 index arrays for nD
  • improve csr_todense_nd performance??
  • avoid assigning to self? Maybe assign self/other -> A/B

PR Discussion:

  • Try Broadcasting in sparsetools.
  • nD elementwise mixed with broadcasting
    • PR 21613
    • lots here. should we split? or try to fix/merge/follow-up
    • broadcasting works, but copies data
    • coo copies binopt & minmax from _compressed
    • minmax calls mixin methods without subclass (which might be ok?)

sparray migration PR Tracking:

Conversion of SciPy subpackages to sparray

Monday November 25, 2024

Attendees: Dan, CJ

News:

Topics Discussion:

  • For v1.15
  • Broadcasting in sparsetools.
    • two PRs. not trying to do views. Just changing loop control with B/C in mind.
  • Other nD topics:
    • 32-bit indexing for COO nd

PR Review Discussions:

sparray migration PR Tracking:

Conversion of SciPy subpackages to sparray

More minutes in the 2024 hackmd file

More minutes in the 2023 hackmd file

Select a repo