# 2020-12-02 conda-forge core meeting

****

[Zoom link](https://flatiron.zoom.us/j/93242638216?pwd=bjRCWmVJRW1oTGJhN09VUmxtTTJOUT09)
[What time is the meeting in my time zone](https://arewemeetingyet.com/UTC/2020-08-26/17:00/w/Conda-forge%20dev%20meeting#eyJ1cmwiOiJodHRwczovL2hhY2ttZC5pby9wUk15dFVKV1FmU3NJM2xvMGlqQzJRP2VkaXQifQ==)
[last weeks meeting](https://hackmd.io/r5eeo5cGQ7iHG1IgpB6axQ)

## Attendees

* Cheng H. Lee (CHL)
* Crystal Soja (CAS)
* Filipe (FF)
* John Kirkham
* Keith Kraus
* Matthew Becker
* Isuru Fernando
* Sylvain Corlay
* Eric Dill
* Markus Gerstel
* Chris Burr
* CJ Wright
* Paul Ivanov
* Connor Martin
* Stephanie Guo
* Marcel Bargull

## Agenda

### Standing items

* [X] intros for new folks on the call

* [x] (CJ) budget
* current approvals?
* Whenever updated numbers land, please screenshare and show the budget.
* Link is in Keybase (numfocus_spreadsheets.txt)
* (CJ) We're all up to date and Oct P&L is zero

* [ ] open votes
* Markus for staged-recipes

* [ ] (MRB/ED/SC) Roadmap / Funding
* **WILL RESUME NEXT WEEK WITH A SPECIAL MEETING**
* goal is to spend 15 minutes each core meeting for ~3-4 meetings to discuss this
* Save last 15 minutes for this.
* https://hackmd.io/0zGSUS71SbOdBsdLtDmGjg
* notes will get added to hackmd above
* MRB will collate into a document of sorts
* some resources
1. Some numbers:
* https://github.com/conda-forge/by-the-numbers/blob/master/conda-forge-timelines.ipynb
* conda-forge has added about 3k feedstocks per year in 2019 and will in 2020
* the growth in the amount of data we store appears to be accelerating
2. risk measurements
* CJ deserves all of the credit for this idea
* https://docs.google.com/spreadsheets/d/1ADNNauwVZWUsEdlh5aEg0OLjyDWvCX7PLoo-K34EqcM/edit#gid=0
* going to skip today due to my own constraints
* TODO:
* Everyone take a look at the pypa roadmap:
* fill out the risk measurement spread sheet:
https://github.com/psf/fundable-packaging-improvements/blob/master/FUNDABLES.md

#### From previous meeting(s)

* [x] (MRB/IF) pybind11 packaging
* issue: https://github.com/conda-forge/conda-forge-pinning-feedstock/pull/849#issuecomment-727207060
* we agreed to a pybind11-abi metapackage that
* is versioned with the pybind11 internal abi
* has a run export on itself
* pybind11 will have a run_constrained on its version
* can be optionally added to host envs by users to enforce ABI compat as needed
* IF: this has the side effect of enforcing one global pybind11 ABI

### Your __new__() agenda items

* [X] (Filipe) Kaleido PR is still pending: https://github.com/conda-forge/staged-recipes/pull/12747
* ED: can we merge and then fix it later?
* FF: Let's do it!

* [x] (Filipe) We need a new conda-build release that fixes the prefix issue on Windows or we need to use a really old version there.
* effects pyqt and sip
* IF: we should backport
* FF: if soon, then no need
* CHL: should release in next two or three weeks

* [x] (CJ) NumFOCUS is having a legal Q&A, do we have concrete questions for them?

* [x] (CJ) Depfinder audit results
* Various improvements have gone into the depfinder based dependency inspection system.
* This jupyter notebook shows some of the results
* https://github.com/conda-forge/by-the-numbers/blob/master/audit_accuracy.ipynb
* There are some important subtle points around depfinder
* A feedstock is "accurate" (from depfinders perspective) if all the conda run requirements are either found as required or questionable imports. Questionable imports are imports obscured so that they might not be run (inside a function, behind a try except, etc.)
* The audit is run on the source code itself, not the resulting `site-packages` so files we wouldn't otherwise ship (tests, examples, etc.) may be drawn into the audit.
* The audit doesn't have much visability to optional files so we assume that all files (and their associated imports) are required. This can cause depfinder to think conda-forge is underspecified.
* If a feedstock requires a pkg that clobbers other pkgs then we may loose requirements since those imports are formally supplied by the clobbering pkg
* These audits could form the basis of efforts to
* fix our depenencies where we are missing required dependencies and transitive dependencies
* fix upstream requirements specifications and determine how reliable upstream specs are at the pkg requirement level
* help maintainers make informed decisions around dependency updates
* Audit source code: https://github.com/regro/cf-scripts/blob/master/conda_forge_tick/audit.py#L39
* Import maps: https://github.com/regro/libcfgraph/tree/master/import_maps

* [x] (MRB) packaging ray
* we have a working recipe: https://github.com/conda-forge/staged-recipes/pull/11160
* are we happy with it?
* KK: will push out to ppl I know who care about this

* [x] (MRB) off-label github actions usage
* we have at least two feedstocks that are using github actions in the conda-forge org for their custom CI scripts
* we cannot support every feedstock doing this
* I sent them a note: https://github.com/conda-forge/pangeo-notebook-feedstock/issues/49
* We need a policy.
* I put a draft policy here: https://github.com/conda-forge/conda-forge.github.io/pull/1199
* TODO
* MRB: put up for a 50% vote

* [x] (MRB) artifact-validation and clobbering in the prefix
* see here: https://github.com/conda-forge/artifact-validation
* this works as follows
1. when a package copy request is sent to the heroku service, it sends the artifact to be validated via a GHA repo dispatch event
2. this event runs a validation CI job on github actions (https://github.com/conda-forge/artifact-validation/actions?query=workflow%3Avalidate-artifact)
3. the validation job downloads the artifact, double checks the MD5 checksum, and then inspects its files for paths not
allowed using a set of glob filters
* the glob filters are listed in yaml files which indicate which paths are protected and which packages are allowed to write to those paths
* we use a combination of hand specified paths and generated paths
* by hand ones are here: https://github.com/conda-forge/artifact-validation/tree/master/validate_yamls
* generated ones are here: https://github.com/conda-forge/artifact-validation/tree/master/generated_validate_yamls
* we use the list of files in libcfgraph to generate the protected paths
* we also continuously scan artifacts using libcfgraph and downloads
* https://github.com/conda-forge/artifact-validation/actions?query=workflow%3Ascan
* we also update the filters as new packages are added
* https://github.com/conda-forge/artifact-validation/actions?query=workflow%3Aupdate-filters
* next steps
* mark invalid artifacts as broken: https://github.com/conda-forge/artifact-validation/blob/master/scan_data/invalid_packages.yaml
* do uploads to anaconda.org from the GHA validation jobs and don't upload invalid artifacts
* expand the set of filters
* FF: send some data to PyPA

* [x] (CHL) conda & conda-forge being used in IoT, embedded, etc.
* Anaconda curious if anyone is using, plans to, or wants to use conda and its package ecosystem in such environments; if so, what needs to be done to (better) support it.
* Answers:
* Robotics
* NVIDIA Jetsons & RAPIDS signal processing lib

### Pushed to next meeting

### Active votes

### Subteam updates

#### Bot

#### ARM

#### POWER

#### CUDA

#### Docs

#### staged-recipes

#### website

#### security+systems

### CI infrastructure

#### Compiler upgrade

### CFEP updates

#### Open PRs

* [cfep-04](https://github.com/conda-forge/conda-forge-enhancement-proposals/pull/7) X11 and CDT policy
* INACTIVE - Merge in with some inactive-esque status?
* Needs new champion. Thanks for your work on this pkgw! Has unaddressed comments from pkgw as from Jan 10, 2020

* [cfep-06](https://github.com/conda-forge/conda-forge-enhancement-proposals/pull/9) Staged-recipes review lifecycle
* INACTIVE - Merge in with some inactive-esque status?
* Lingering comment from @saraedum. @jakirkham, can you reply? Has unadressed comment from @saraedum from Jan 8, 2020
* (MRB) The stalebot has solved the worst of the issues here. I think we could defer this one permanently.

* [cfep-10](https://github.com/conda-forge/conda-forge-enhancement-proposals/pull/15) Feedstock statuses, unmaintained
* INACTIVE - Merge in with some inactive-esque status?
* Needs another review. Has unaddressed updates from pkgw as of Jan 11, 2020

* [cfep-12](https://github.com/conda-forge/cfep/pull/23) Removing packages that violate the terms of the source package
* Stalled since May 26, 2020
* Active debate about moving to "broken" vs deleting from conda-forge channel
* Active vote, ends on 2020-03-11
* What were the results of the vote?
* Did we hear back from NumFOCUS?

* [cfep-17](https://github.com/conda-forge/cfep/pull/32) Handling pin backports and dependency rebuilds
* Stalled debate about implementation details between Isuru, CJ and Matt
* **UPDATE 2020-07-22**: We in principle have agreement to render the extra pinnings needed directly in the feedstock
on a temporary basis (i.e., until the migration has ended).

* [cfep-20](https://github.com/conda-forge/cfep/pull/39)

## Discussion

## Check in on previous action items
Copy previous action items from last meeting agenda.

### This meeting
2020-11-24

### Last meeting
2020-11-18
* [ ] (IF/MRB/MV) intel oneAPI
* todo
* [ ] (Nikolay) licensing for opencl_rt
* [ ] (Nikolay) intelmpi ABI compat w/ mpich
* [ ] (MRB/IF) figure out how exactly to package C/C++ compilers
* [ ] (MRB/IF) think about fortran ABI
* [ ] (MRB) make conda-forge compilers room (add people including keith)
* [ ] (MB) asking core members to move to "emeritus" status
* [ ] TODO: Eric to set up quarterly check-in for all core members to see if they're interested in remaining "active" or if they want to move to emeritus
* Remove emeritus folks from having access to various credentials (api tokens, twitter password, etc.)? This would require a change to the governance doc.

### 2 meetings ago
2020-11-11
* TODO: Think about bringing in JOSS to provide context around how we might best write papers

### Move to Issue Tracker
2020-11-03
* [x] (MRB) proposed policy on when core pushes to the feedstocks they don't maintain
* [x] (MRB) put in docs PR
* [ ] (MRB) make PR on bot to mention the policy
* TODO: Check on Forrest Watters permissions for core
* [x] (FF) Outreachy would cost 6500 USD.
* Next steps: write abstract and vote on spending of funds.

2020-10-28
2020-10-21
* [ ] (Marius?) Python 2.7 migration
* ( ) [ ] make a hint
* ( ) [ ] make an announcement
* ( ) [ ] make the hint a lint

2020-10-07
* [ ] Make sure to add the NVBug info to the cudatoolkit package that conda-forge makes (if we make one)

2020-09-30

2020-09-23
* [x] (MRB)
* do libgfortran name change
* add target platform to hashes
* do gfortran migration with bot
* bump pinnings

2020-09-16
* [x] Get a call set up with Jon Mease about the kaleido staged recipes PR
* Emailed on 2020-09-16
* [x] (FF) Open up a PR on the python feedstock for python 3.9 and see what fails

2020-09-09
* [ ] (ED) Update governance docs with similar voting model as what got put into conda-tools (+3 with no -1 is a pass)
* [ ] (SC) Write jinja template to turn institutional partners yaml into a website https://github.com/conda-forge/conda-forge.github.io/blob/master/src/inst_partners.yaml
* [ ] (SC) Document what needs to be done to create an OVH account and get access

2020-08-26
**Docker hub**
* [ ] (JK) Check in on Azure build workers to see if they have the docker hub limitation.
* [ ] (JK) work with dockerhub to see if we can get OSS status
* [ ] Check in again at some point. We haven't heard back as of 2020-09-23
* [x] (MRB) start pushing images to quay (https://github.com/conda-forge/docker-images/pull/152)

**OVH**
* [ ] (???) build webpage to credit them (and others)
* [ ] If we're adding a logo, will want to make sure that we have permission to use it.
* [ ] Shout-out on twitter at some point. "Thanks forOVHCloud for providing a VM", etc. (maybe after we ship qt on windows with it?)

* [ ] Figure out how to communicate breaking changes to users. Likely should open up an issue immediately for futher discussion. Ping @kkraus, plus capture notes from further up in these meeting notes

* [ ] John K. will update the cuda toolkit feedstock on the git repo to note the NVBug link to the internal NVIDIA issue tracker
* [ ] Jonathan will update docs to note that some non-exhaustive list of packages (like cuda-toolkit, MKL, etc.)
* [ ] Jonathan will review this [PR](https://github.com/AnacondaRecipes/cudatoolkit-feedstock/pull/7)

* [ ] (Kale) schedule conda working group
* [ ] cfep-10 next steps: CJ to call a vote for feedback
* [ ] cfep-06 next steps: Ask staged recipes team to champion this CFEP and move it forward
* [ ] jakirkham & CJ-wright to sync on adding CUDA to the migration bot
* [ ] (Eric) Scheduling Anaconda <-> conda-forge sync on anaconda.org requirements gathering
* Will try and get this scheduled in the next month.
* [ ] (Anthony) Reach out to NumFocus to figure out legal ramifications of not including licenses in files.
* [ ] (Eric) check internally for funding levels for hotels & flying folks from the community in?
* [ ] (Eric) Figure out finances of conda-forge to support themselves?
* [ ] (jjhelmus) Open up CFEP for which python's we're going to support
* [ ] (jakirkham) write a blog post on CUDA stuff we discussed today
* [ ] (jakirkham) update docs on how to add CUDA support to feedstocks
* [ ] (jakirkham) will open an issue on conda-smithy to investigate Drone issues. (ping the aarch team)
* https://github.com/conda-forge/conda-forge.github.io/issues/954
* [ ] (ED) Who we are page? Some combination of a FAQ and a who is everyone. FAQ things like:
* who's the POC for CF <> Anaconda, CF <> NumFocus, CF <> Azure
* who's the POC for the various subteams?
* Informal information: roles, day jobs, bios, the whole nine yards, why you're here, etc.
* Public or internal? I don't really care either way. Anyone feel strongly one way or the other?
* opt-in to public bios
* software carpentry has a large number of instructors and has https://carpentries.org/instructors
* some concern about "yet another place to keep stuff up to date"
* [ ] (ED) document strategies for reproducible environments using conda-forge
* [ ] (UK) Static libraries stuff
* [ ] Add linting hints to builds to find them
* [x] Recommend how to package them -> CFEP-18
* [x] We should write docs saying we don't provide support and this is a bad idea. -> CFEP-18