HackMD
Dev Summit meeting notes
Information
- When: 2021-06-14
- Participants, session 1, 0900UTC:
- Matt Hall (
welly,bruges) - Dieter Werthmüller (
emsig,simpeg,subsurface) - Jørgen Kvalsvik (
segyio,dlisio) - Irene Wallis (
fractoolbox) - Rob Leckenby (
wellpathpy) - Tony Hallam (
segysak) - Leo Uidea (
fatiando) - Wesley Banfield
- Matt Hall (
- Participants, session 2, 1700UTC:
- Matt Hall
- Martin Bentley
- Agustina Pesce (
fatiando) - DC Slagel (
lasio) - Miguel de la Varga (
gempy,subsurface) - Santi Soler (
harmonica) - Leo Uieda (
fatiando)
Slides
Background
The tools that we have made are useful, and have big users (O&G, geothermal &c). To get stuff done, one approach is to pay people to work on the packages that we consider important. People are working on them, but can we speed things up.
Is there scope for Software Underground to be something akin to NumFOCUS for the subsurface/geoscience sphere?
Improve tools:
- Improve docs
- Make faster/easier installation
- Better tutorials
A conversation with some people, especially Leo and Santi, during and after Transform 2021 led to today's meet-up sessions. We're holding 2 of them to try to be less bound to time-zones and more global in character.
Dec-Mar internship
What is missing as a community?
- Miguel: permanence for projects. Permanent developer positions.
- Matt: internship would be an experiment to see how this works.
- Hope is that it leads to different conversations with organisations who would be able to fund this sort of thing.
- It will also let us figure out how to find which projects need help and how does this work logistically.
- DC: if we only have three months, what is the most complete thing that we can get done? And how do we find those things within the projects within our purview?
- Matt: Ask the maintainers (rate your project on X, Y, Z) to see what is most common
- Matt: Notion of affiliate projects - quality level of projects to be considered an affiliate. Can look at making improvements to projects that are nearly there.
- Miguel: Maybe we need a vertical slice through the projects to ask organisations what they would need to see in order to fund people. - Can maybe be partial things, so if X can supply a small piece, then a bigger organisation can pay for that if we get something solid shown.
- Matt: We will need to be pretty flexible about the tasks, but for the first go, we need to know what the maintainers want.
- DC: Generally what companies need is fast and continuous development on features that they want.
- DC: We need to make sure that we can sell things as steady development, not stops and starts. A good question is what do developers need to move things forward at a steady pace.
- DC: Also need to consider reliability and stability - companies want this. So what do maintainers need to keep a project going and up to date for 5-10 years.
- Miguel: The above is critical. The long-term stability is something that the Software Underground can add.
- Matt: Things like OSDU and OpenEarth got a good amount of financial support from companies, which should be possible by Software Underground as well. There is now a pathway to get money to developers.
- Matt: this is especially the case for things that make it easier for people to contribute to our projects.
- Matt: The goal is to start building a bit of momentum to be able to take this sort of thing forward.
- Matt: We have been looking at Outreachy. They facilitate the hiring and transactional side of things for interns with an eye towards diversity of hires and humanitarian projects.
- Matt: Current cohort of interns is ~70, including folks working on fairly large-scale projects.
- Matt: There are two annual cohorts, one May->August, and one in December->March.
- Matt: There is a bit of work from the applicant projects, so tasks need to be well-defined.
- Leo: Would the goal be to apply as projects or as Swung?
- Matt: As Swung. Probably working on 1-3 projects, depending on what projects needed and potential impact and a few other things.
- Leo: One thing that we will probably need is to consider what the intern would get from joining us. Not just the experience and some money, but the focus of Outreachy is to provide a platform for students to improve and join the open-source community.
- Leo: This means that tasks need to be more than menial things that maintainers do not want to do.
- Santi: Agreement. Needs to be more than just fixing the broken CI. Something that has helped projects would be faster and more code reviews. So maybe we need to try and push for such things: more and better code reviews.
- Santi: For the internship, maybe we want to encourage people to own the project. Ideally we get someone from the geoscience background.
- Miguel: Most of the bugs that I have seen are with interconnections, not within a single function. So in order to do such reviews, people need to understand the whole codebase, which might be too much for an internship
- Santi: Yes, but possibly more reviews, even if they are not as detailed would help.
- Leo: Something that an outsider can do very well is to look at what user-facing code and APIs are like - do they make sense?
- Matt: True, but we may get someone who is really interested in HPC, and not user-facing things.
- Martin: Outreachy would probably tailor the interns to the projects, at least to some extent. If we want UX-type things to be looked at, then an intern interested in HPC would presumably not actually be looking at it.
- Matt: Outreachy may well be willing to meet us and explain some of these issues.
- Matt: One approach would be to go to organisations and say 'we have been doing this, but look at what else we could do if you matched'.
- Matt: Before bringing in sponsors as well, let us make sure that it works between Swung, maintainers, and interns.
- DC: Sponsors may also be interested in interns who have practical experience with the subsurface tools that we are interested.
- Miguel: Would it be possible to offer this internally within Swung first?
- Matt: The main issue here is if we get someone who is not in Canada - I can handle that Canadian case, but for anywhere else I have no idea how to do it right.
- Matt: I would also like to get a bit of separation from our initial bubble - avoid bias, etc.
- Matt: We can almost certainly point Swung people at Outreachy doing things, so we can feed the applicant pipeline.
- Martin: We can maybe do internal calls for things that are very geoscience-based that need additional domain knowledge, while Outreachy is likely to be more copmuter science based.
- DC: It might be good to look at someone who is interested in project management type things, working above individual projects.
- Matt: This is good, but it might be harder to make the value of such a project visible to someone else.
- Miguel: In some cases, like the public organisation here in Germany they are interested in the vision. Being able to say that we are not just a group of people, but saying that in 5 years we want to have hired three people full-time, and all the projects interconnect might be good.
- DC: This comes back to reliability and longevity that sponsors will be interested in. Laying the groundwork for that early is good.
- Miguel: These are maybe two different positions: a funded position for someone to oversee the entire Swung Stack would not just be an intern but a senior with a number of years of experience.
Path forward
- Matt: Is a questionnaire or similar for the maintainers useful?
- Leo: I think speaking to Outreachy is the first step that needs to happen, this will affect the types of project that we pitch.
- Matt: Are there other projects similar to Google SoC that we should look at?
- Leo: Offering something in the vein of Outreachy but with other societies like Geolatinas or Earth Science Women's Network. We may be able to involve them in some development.
- Leo: GMT is offering a three-day bootcamp on how to contribute to pyGMT to the Women's Network. Similar sorts of initiatives might be very attractive, especially if we can offer a paid internship for doing additional open source contributions.
- Matt: I am not sure if there are organisations that might be able to help with that sort of thing.
- Leo: Maybe Swung can offer that sort of thing.
- Matt: Yes, but not yet. We would need someone as an executive director to make sure that this would actually work.
- Martin: Could also look at global conferences and similar for shorter-term things, like the pyGMT contribution.
Would X Help Your Project?
A few ideas have been floated in the past, and several of them resurfaced in the 'call for agenda items' for this meeting. Thank you to Leo and John for proposing them.
Matt chose to word these as "Would X Help Your Project", because this seems like a more pointed way to think about it than, "Should Swung do X?".
- Web analytics server, eg: for project docs.
- A lot of people don't want to use Google Analytics. We turned GA off on the Swung website and tighted up our Privacy Policy to be responsible for less data.
- Would let maintainers gather information about users
- Forum for projects on something like Discourse:
- Means that projects are not running these for themselves, and can have things kept in one place.
- Doesn't seem connected but is actually connected question: Is Slack the right platform for Swung? Introducing a new forum/chat risks forking the conversation. So be careful what we wish for. Therefore: how would we do this experiment and minimize that risk?
- Hosted JupyterHub and notebook repository
- This is happening, although it is very experimental.
- Contact: Filippo.
- A full subsurface Python distribution and/or Conda channel
- Something like Anaconda - curate packages and make sure that it is all working.
- Packages would come from conda-forge, so they would need to be compatible with everything else in there.
- Use https://github.com/conda/constructor - provide a list of packages and their versions. This would then build an installable distribution.
- Would need to test everything via CI.
- This is not a trivial amount of work, but might be worthwhile.
- Justin Gosse's recent work on the depency / connection graph might be useful to look at.
Summary
-
There was general support for the idea of hiring an intern to work on 'something'. We noted:
- Swung must offer a positive experience to the intern, and work to contribute what that individual needs from the project.
- We could choose a 'meta'/Swung project (building capcity through better coherence/organization), or tool-oriented project (building capacity through better tools).
- Swung and/or Maintainers (depending on the type of project) must be able to identify useful tasks — any work done must be useful to them too.
- Swung and/or Maintainers/projects must be ready to mentor, which means time.
- We will all need to be somewhat flexible as we adjust course to meet the needs of the intern, the projects, Swung itself, and possibly future sponsors.
-
It's not yet quite clear what the best project(s) are. Perhaps we'll get some clarity as we talk to Outreachy and we all think about it some more:
- Spread across three different tools.
- Focused on one tool.
- More of a meta / Swung project, e.g. the Conda / installer project, or the 'affiliate' program.
-
Plan:
- Step 1 is to talk to Outreachy to answer some of our questions about the program. — Matt has contacted Outreachy to ask about this. See below for questions.
- Then we can poll the maintainers to find out who is up for mentoring, and what kind of tasks they have in front of them. Another way to spin this might be to say, "What would an 'affiliate project' look like and which projects lack what to get there?"
- Then we can select some short-list projects and start the search process with Outreachy. This would be in (roughly) August.
- Once we have defined a plan, we can socialize/market it with sponsors to potentially match our funding and perhaps even hire a second intern for this period or in 2022.
-
Questions for Outreachy (these are drawn from the questions that came up in the 2 sessions, and in the May board meeting):
- What do the interns want to get out of their experiences? (Is is co-op type experience, a job, etc? Or does it vary?)
- What kind of background do most applicants have? (Is it CS, IT, domains, or does it vary?)
- Can we feed people into the applicant pool directly? E.g. promote the internship in our own and affiliated communities?
- Outreacy seems to be a project of the Software Freedom Conservancy, is that right? What's the potted history?
- Are we eligible, and if so, what does the process look like for an organization like us?
- Based on previous experience, is it okay to run the project across several tools/maintainers? Or would it be better to choose a single one, or a 'meta' (Swung) one?
- How do you avoid the search process (of contributions) becoming too onerous — for the applicants or the maintainers?
