# Notes on float precision dicussion

### Might be good to have a brief summary:

* [name=rkern] Quite strongly opposed to sacrificing accuracy at least on sin/cos
* [name=mattip] Voiced opinion for reversion
* [name=Dan Allen (brookhaven national lab)] Tends towards it being too inaccurate; some scipy lecture note examples give surprising results for new-comers. (you can see that as a teaching opportunity, but...)
* [name=Jerome Kieffer] Is opposed (was already opposed to any adoption of "SVML" the last time around)
* [name=rgommers] Is open to reverting the sin/cos changes since they seem to be significantly more disruptive than other SIMD changes over the past years. But also wants to see an improved general accuracy-precision tradeoff methodology or framework, because the comments about always needing maximum accuracy (in general, not specifically for sin/cos) are too simplistic - it's always a tradeoff. Suggests a compile time flag (or inclusion in the `disable-svml` one), probably defaulting to off.
* [name=seberg] I agree with this! But, I am a bit disappointed in the amount of methodology we are given to work with by those who like the less precise versions. (I agree that full precision is a pipe-dream, though)
* [name=Matthew Brett] doesn't like less precise defaults, either.
* [name=tcaswell] prefers not to revert because compared to SVML the results are mainly different rather than worse. (suggesting the burden of proof should be reverted). Matplotlib dealt with the changes relatively easily.
* [name=charris] Aligned with a high accuracy default.
* [name=Benjamin Root] Special values behavior is the most concerning part (note seberg: SVML seems to be a tiny bit better here, but not much)
* ambivalent about reversion though, because GIS workloads should actually benefit from the speedup.
* [name=seberg] I lean towards reversion due to special values. (more discussion is warrented, but maybe not the place here)

### Random thoughts from seberg:

* **Issues related to SVML adoption:**
* https://github.com/numpy/numpy/issues/20988
* https://github.com/numpy/numpy/issues/21209
* https://github.com/numpy/numpy/issues/22028
* https://github.com/numpy/numpy/issues/22904
* https://github.com/numpy/numpy/issues/23289
(We adopted it in NumPy 1.22, released in the beginning of 2022; most of the lower accuracy of these paths are seen only on )

Links to individual PRs for accelerated functions with an accuracy/speed tradeoff (to check benchmarks and see if there's any issues linking to them):
- https://github.com/numpy/numpy/pull/20991
- https://github.com/numpy/numpy/pull/22240
- https://github.com/numpy/numpy/pull/22164

* **What are the defaults?**
* C/C++ math has higher accurate defaults I think normally.
* Intel seems to default to high accuracy ones (but suggest to use the LA)?
* `libmvec` defaults to 4ULP versions, but has `--disable-mathvec`. You only semi explicitly use it with `#pragma omp simd` *and* using `-ffast-math` (which I would expect to do this type of thing): sourceware.org/glibc/wiki/libmvec

* Sloppily said, I think the "fast compute" crowd pushed low accuracy versions and dumped the responsibility of deciding on the trade-off decision on us. They have given us next to nothing to go on to make that decision. Yes, just ULP isn't all there is to it, but there is a trade-off.
* For `sin` the tradeoff between great accuracy and "low accuracy" for SVML means that 0.5 ULP vs. ~2.5 ULP makes a bit more than a 2x difference apparently. The high accuracy version has very good accuracy, better than what we need to worry about probably.
**WARNING:** the accuracy reported there is better than what the version that NumPy uses supposedly has!
**Reference:** [performance](https://www.intel.com/content/www/us/en/docs/onemkl/developer-reference-vector-math-performance-accuracy-data/2021-1/performance-of-all-real-vm-functions.html), [accuracy](https://www.intel.com/content/www/us/en/docs/onemkl/developer-reference-vector-math-performance-accuracy-data/2021-1/measured-accuracy-of-all-real-vm-functions.html); [accuracy in NumPy](https://github.com/numpy/numpy/pull/19485#issuecomment-887995864)
* **My point here:** We brought up if we should use the high accuracy version. I don't think that was even possible so the speed trade-offs were never really discussed (as far as I remember). But: Even if that would have failed on licensensing or other issues, I am not convinced that this should influence our decision on what the default behavior should be!

* Snarky comment: "I use CuPy because its accuracy is better than NumPy!". SIMD is great, accuracy trade-offs are more than just introducing SIMD though. At this point CuPy has *clearly* better accuracy than NumPy. To some degree that is frankly ridiculous, we may "accelerators" to have sacrifice precision. NumPy is **not** an accelerator.
* (Sure, CUDA doesn't support FPEs, I guess)

* NVIDIA internal chat: Some involved in the math stuff were seriously surprised by the terrible precision of the new sin/cos around special values (which probably even translates to the SVML version).

* There are paths forward to allow opting in to lower precision. Yes, they are a lot of work, but if we allow free-reign for those who want the fast version, we won't get it either.