owned this note   changed 10 hours ago  
Published Linked with GitHub

Video call link: https://meet.google.com/jrp-qgcf-nxh
If not working: https://colgate.zoom.us/j/276854695

Monday April 28, 2025

Attendees:

PRs Ready for Review:

  • PR 22873: nD with unary functions
    • sum/mean/min/max/argmin/argmax
    • refactor nD->2D conversion to a helper func
    • refactor dot/tensordot/matmul to use helper
  • PR 21743 test_base.py split.
    • Removed list of tests that dont use get_index_dtype and tests that the list was correct

Think about:

  • Next nD PR will involve (in)equality, multiply, divide, maximum/minimum
    • Right now eq and ne return True/False if shapes do not match!?!? Can we deprecate/remove this for sparray?
    • Anushka's pr had copied much of inequality code to _coo.py. How to structure?
      • Currently:
        • base.__gt__ converts tocsr()
        • csr.__gt__ makes bad_scalar_msg and IDs the operator. (why separate from base.__gt__)
        • _inequality(), __eq__ and __ne__ duplicate dispatch code for cases isdense, isscalarlike and issparse
        • _scalar_binopt and binopt help. (no changes needed)
      • New COO nD code:
        • single dispatcher for isdense isscalarlike, issparse.
        • Converts to CSR and calls CSR.__gt__() (which re-does disptach logic)
        • Supports list and array-like input (unlike current code)
      • Ideas:
        • Pass around op in comparison/bool funcs. Use op.name when calling _binopt.
        • move bad_scalar_msg code to _spbase.__gt__. remove __gt__ from _cs_matrix
        • COO code only overrides __gt__ making conversion to CSR or 2D & CSR, and postprocessing results.
        • maybe add base._comparison() to do universal checking and e.g. isdense processing from _cs_matrix (which is in eq, ne, inequality now)
      • Could instead aim for more minimal refactor with coo overriding which is what Anushka's approach takes.
  • 21925 broadcasting of _binop at C++ level
    • row broadcasting is easy. col requires a few differences
    • do we put col broadcasting into the existing functions, or as a separate func?
    • Idea: create small C++ structure to hold shape and/or stride info. Perhaps like:
int ndim; 
int nnz;
int shape[];   // length ndim 
int strides[]; // length ndim     (or compute these)
I indptr[];    // length shape[0] + 1
I indices[];   // length nnz
T data[];      // length nnz
  • 19984 binop general/canonical discussion (Nothing implemented since last mtg)
    • Does this capture last mtg discussion?
      • Would be nice to capture the egregious cases and boost doc warnings. Egregious cases (based on code) when the vector created in General is long and sparse.

Monday April 14, 2025

Attendees: Dan and CJ

PRs Ready for Review:

  • Ready for review/merge (approved already):

    • 22647 validateaxis
      Image Not Showing Possible Reasons
      • The image file may be corrupted
      • The server hosting the image is unavailable
      • The image path is incorrect
      • The image format is not supported
      Learn More →
      MERGED
    • 22527 add returns sections to construct.py function
      Image Not Showing Possible Reasons
      • The image file may be corrupted
      • The server hosting the image is unavailable
      • The image path is incorrect
      • The image format is not supported
      Learn More →
      MERGED
    • 22169 refactor CSC to use CSR sparsetools
      Image Not Showing Possible Reasons
      • The image file may be corrupted
      • The server hosting the image is unavailable
      • The image path is incorrect
      • The image format is not supported
      Learn More →
      MERGED

  • Think about:

    • Next nD Steps:
      • PR has sum/mean, min/max, binopt, multiply, add, sub, minimum/maximum
      • ?split out sum/mean/min/max from bigger PR?
      • add other nD features:
        • CSR support (or new format name csx to do csr in nD)
        • indexing?
        • construction functions?
        • array_api nD functions like expanddims
    • 21925 broadcasting of _binop at C++ level
      • row broadcasting is easy. col requires a few differences
      • do we put col broadcasting into the existing functions, or as a separate func?
      • if we split col broadcasting, do we support both general and canonical for this case?
    • 19984 binop general/canonical discussion
      - simple heuristic to pick general or sort+canonical
      - or docs to push users to choose
      - Let's aim for this: Would be nice to capture the egregious cases and boost doc warnings. Egregious cases (based on code) when the vector created in General is long and sparse.
    • 21743 test_base.py split.  2 issues:
      • right now Test64bit sparray tests are run when "slow" tests are run. spmatrix tests always.
      • Test64bit is hard to separate from the rest of test_base.py when testing
      • Q1: should we turn on sparray beyond "slow" tests? (Ans: Yes) Should we change some spmatrix to be slow? (Ans: No – we will remove them with spmatrix so won't be long. And we'll see if people complain.)
      • Q2: is it worth splitting out Test64Bit  to a separate module to allow partial testing? (Ans: Yes)

Monday March 17, 2025

Attendees: CJ, Stefan

PRs Ready for Review:

  • Ready for review/merge (*approved already):

    • 22647 validateaxis
    • 22535 * add returns sections to construct.py function
    • 22169 refactor CSC to use CSR sparsetools

  • Read through (easy?)

    • 22535 update roadmap sparse

  • Think about:

    • 21925 broadcasting of _binop at C++ level
      • row broadcasting is easy. col requires a few differences
      • do we put col broadcasting into the existing functions, or as a separate func?
      • if we split col broadcasting, do we support both general and canonical for this case?
    • 21743 test_base.py split.  2 issues:
      • right now Test64bit sparray tests are run when "slow" tests are run. spmatrix tests always.
      • Test64bit is hard to separate from the rest of test_base.py when testing
      • Q1: should we turn on sparray beyond "slow" tests? should we change spmatrix at all?
      • Q2: is it worth splitting out Test64Bit  to a separate module to allow partial testing?
    • 19984 binop general/canonical discussion
      - simple heuristic to pick general or sort+canonical
      - or docs to push users to choose

Monday March 3, 2025

Attendees: Dan, CJ

Active PRs and Issues

Monday February 17, 2025

Attendees: CJ, Dan

News:

PR discussions, Topics/Questions:

  • nD features:
    • PR 22540: nD binop split from broadcasting
      • coo copies python code for binopt & minmax from _compressed.py and _data.py
      • minmax calls mixin methods without subclassing. works, but why not subclass?
    • PR 21613: broadcasting with copies
    • Currently nD-add/sub creates duplicates without calling sum_duplicates
    • Q: should CSR support nD with an axis arg? Or call it something other than CSR? (1D currently uses the name CSR)
  • Q: How to reliably time binop sparsetools?
    • Issue 19984: predictably slow "add"
      • Considering general vs sort & canonical, general is predictably slower for some parameter regions. (most of the time it is faster)
      • Posts from commit 618c4ea and #1385 (comment) suggest that nbell put general back in due to an incorrect timing reported on the issue.
      • Need to check whether sorting followed by canonical is faster than general. My timing comment shows that the general method is usually faster for unsorted indices. But some conditions make it slower than sort & canonical. The conditions seem to be large M with small nnz. But large M suggests should switch to CSC format.
    • PR 21925: broadcasting without copy will need timing for this
      • Currently general & canonical
      • broadcast indptr can be easily included
      • broadcast indices could be separate func or separated in an if clause.
      • C++ changes: PR 21925 csr.h
    • PR 22169: CSC binop via CSR and _swap Timing? (no difference? but how to check that)

Follow-up items for 1.16

  • dont copy when broadcasting
  • support int32 index arrays for nD
  • improve coo_todense_nd performance?
  • avoid assigning to self? Maybe assign self/other -> A/B
  • choose priority between CSR nD and broadcasting w/o copy

Monday December 9, 2024

Attendees: Dan, CJ, Stéfan

News:

Follow-up items for 1.16

  • dont copy when broadcasting
  • support int32 index arrays for nD
  • improve csr_todense_nd performance??
  • avoid assigning to self? Maybe assign self/other -> A/B

PR Discussion:

  • Try Broadcasting in sparsetools.
  • nD elementwise mixed with broadcasting
    • PR 21613
    • lots here. should we split? or try to fix/merge/follow-up
    • broadcasting works, but copies data
    • coo copies binopt & minmax from _compressed
    • minmax calls mixin methods without subclass (which might be ok?)

sparray migration PR Tracking:

Conversion of SciPy subpackages to sparray

  • sparray migration: csgraph docs: PR 21873 merged
    Image Not Showing Possible Reasons
    • The image file may be corrupted
    • The server hosting the image is unavailable
    • The image path is incorrect
    • The image format is not supported
    Learn More →
  • sparray migration: csgraph code: PR 21779 merged
    Image Not Showing Possible Reasons
    • The image file may be corrupted
    • The server hosting the image is unavailable
    • The image path is incorrect
    • The image format is not supported
    Learn More →
  • sparray migration: linalg: PR 21710 merged
    Image Not Showing Possible Reasons
    • The image file may be corrupted
    • The server hosting the image is unavailable
    • The image path is incorrect
    • The image format is not supported
    Learn More →
  • sparray migration: optimize: 1st pass PR 21785 merged
    Image Not Showing Possible Reasons
    • The image file may be corrupted
    • The server hosting the image is unavailable
    • The image path is incorrect
    • The image format is not supported
    Learn More →
  • sparray migration: io: PR 21905
  • scikit-image
  • sparray migration optimize pass 2 – in progress.
  • pyamg
  • scikit-sparse
  • cvxpy
  • sklearn

Monday November 25, 2024

Attendees: Dan, CJ

News:

Topics Discussion:

  • For v1.15
  • Broadcasting in sparsetools.
    • two PRs. not trying to do views. Just changing loop control with B/C in mind.
  • Other nD topics:
    • 32-bit indexing for COO nd

PR Review Discussions:

sparray migration PR Tracking:

Conversion of SciPy subpackages to sparray

  • sparray migration: csgraph docs: PR 21873 merged
    Image Not Showing Possible Reasons
    • The image file may be corrupted
    • The server hosting the image is unavailable
    • The image path is incorrect
    • The image format is not supported
    Learn More →
  • sparray migration: csgraph code: PR 21779 merged
    Image Not Showing Possible Reasons
    • The image file may be corrupted
    • The server hosting the image is unavailable
    • The image path is incorrect
    • The image format is not supported
    Learn More →
  • sparray migration: linalg: PR 21710 merged
    Image Not Showing Possible Reasons
    • The image file may be corrupted
    • The server hosting the image is unavailable
    • The image path is incorrect
    • The image format is not supported
    Learn More →
  • sparray migration: optimize: 1st pass PR 21785 merged
    Image Not Showing Possible Reasons
    • The image file may be corrupted
    • The server hosting the image is unavailable
    • The image path is incorrect
    • The image format is not supported
    Learn More →
  • sparray migration: io: PR Requires new keyword arg to indicate sparray/spmatrix. Can we go over the deprecation plan to see if it looks OK?
  • sparray migration optimize pass 2 – in progress. waiting for HIGHS library to be setup. That's in. linprog still not removed.
  • PR 21799: is_sptriangular function merged
    Image Not Showing Possible Reasons
    • The image file may be corrupted
    • The server hosting the image is unavailable
    • The image path is incorrect
    • The image format is not supported
    Learn More →

More minutes in the 2024 hackmd file

More minutes in the 2023 hackmd file

Select a repo