# [Notes] Ideas for creating Usable Open Source Scientific Software
This is a scratchpad of ideas/things we could do/things other people should do to make software more usable/sustainable.

## Contributors
-

## Usable Software

### Preliminary List of Items defining "Usable Software"
MUST have:
- free, permissive license - otherwise academic users may not be allowed to apply a particular software
- availability of documentation; [this approach](https://documentation.divio.com/) to structuring documentation is great. But, I must say that documentation cannot save badly designed software. Here, I only have vague ideas about a badly designed software (non-intuitiveness even for the target audience, etc.). Usage examples are an integral part of the documentation!
- getting started ought to be easy. E.g. a Software must be deployable - developers ought to provide a package, a make script, a container, etc.
- FOSS should provide a way for users to give feedback, e.g. by issue trackers. We recommend providing an example on how to give feedback (e.g. within the README constituting the landing page).
- code should be bundled in releases, releases ought to be versioned, a changelog/release notes should be maintained
- An effort to standardize version numbers, for example: [Semantic versioning](https://semver.org/).
- Using a proprietary code base (e.g. Matlab, Mathematica, SAS) limits portability.
- clear prioritization and decision-making processes, in general. And specifically for usability fixes (?)

NICE to have:
- efficient software (efficiency determines sustainability)
- scalability - if a software is intended for big data sets, it needs to be able to cope with big data sets
- in documentation / tutorial / usage example: a benchmark example is nice to have, to know what to expect in terms of the ability of this software
- for reference: Usability can be described as the capacity of a system to provide a condition for its users to perform the tasks safely, effectively, and efficiently while enjoying the experience. (source: https://en.wikipedia.org/wiki/Usability)

### Pathways to achieving "Usable Software"
- adequate time, money, resources, and support (e.g. overhead for RSEs)
- scientific IT more than administrative IT [something about IT knowing what to do]
- sustainable employment
- overcoming "advantages to not share"

### Examples of Usable OS Scientific Software
- [OpenFOAM](https://https://www.openfoam.com/) is an example of a large community
- [Snakemake](https://github.com/snakemake/) is an example for a workflow system, which is able to a) auto-deploy needed software and b) is highly portable (from local server to HPC systems and various cloud providers)
- [GSTools](https://github.com/GeoStat-Framework/GSTools)

## Software Maintenance

- if software == web service , there should be an institutional commitment to keep the server running ...
- clear on-boarding guides, documentation, processes, code of conduct, playbooks for _contributors_
- availability of test suites, perhaps on different levels (unit, integration, etc.)
- a software should be built from pieces / modules, such that there is no monolithic huge code base, where no single contributor can achieve overview. In contrast, modularized software can be huge (ranging up to millions of code lines) and still provide overview, because contributors will be able to find their niche.
- maintainability is often driven by the philosophy of the first or last author.
-

## Open questions
- are maintainability and usability helpful to each other? In what ways are they conflicting?
- I think this was clarified by Lennart and David this morning: Increasing usability of a research software, especially adding GUI, entails major work not only to develop it but also to maintain it, thus reducing the software's maintainability.

## Other things to do this week
- ponder more about maintainability.
- explain some usability methods. mapping interventions to the SWDL
- draft 3-5 narrative stories (https://www.wqusability.com/articles/personas_storytelling.html)

## How Meag thinks about usability -> experience at NIH

* see the graph on Mattermost for an overall landscape of methods and ideas
* **Approaches** (see https://www.nngroup.com/articles/which-ux-research-methods/ , linked by Meag on Mattermost)
* **Surveys** for large-scale data gathering, often a good place to start, likewise focus groups or brainstorming -> most abstract level, understand the problem that should be solved before starting to write code
* **Diary studies** collected when users start using a tool, the keep a log of their experience and experiences, etc.
* **Unmoderated testing** -> task-based, users are given a a task to complete and are ovserved / keep a log of their experiece completing it
* **Card sorting** -> Different concepts are provided on cards, and users sort them into groups, typically for structuring GUIs
* **Gorilla testing / hallway testing** talking to random members of the public in the 'real world' / on social media -> this is the 'dollar a day' testing
* **User journey** -> flowchart or timeline of sequential steps, describing positive and negative experiences with a given service, visualize painpoints and raise awareness around issues (example at https://en.wikipedia.org/wiki/User_journey#:~:text=A%20user%20journey%20is%20the,users%20interact%20with%20software%20experiences)
* These are not exclusive -> mixed methods augment each other
* How to choose/prioritize? **RICE method** -> Reach, Impact, Confidence, Effort, e.g. https://www.productplan.com/glossary/rice-scoring-model/
* Knowing which problem you want to solve first also guides the decision -> do you want more users? more loyal users, happier users?
* How to map this onto scientific software development?
* Projects pivot frequently, maybe goal is not clear at first?
* Goals of developers and users may diverge
* Development projects, especially smaller onces, might especially benefit from these methods but might find it more difficult to do so, workflows might combine several tools and require different specializations -> not a single product that any entity has influence over
* Meag: Usability and UX can happen at different scales, opportunities for eas of use and clarity exist at all layers -> maybe high-level documentation or cookbook/tutorial methods for specific tasks could help
* Workflow tools such as snakemake etc. can help combine and encapsulate different steps in a sequence, but may constituent packages are already suboptimal in terms of usability
* Possible goals for the retreat
* Guidelines for scientific software developers? Checklists for journal reviewers and editors
* Maybe privacy/security angle also, could garner interest from policy makers / regulation?
* Maybe automated checker that takes burden from reviewers?
* Who could define criteria?
* Example: [Web Content Accessibility Guidelines (WCAG)](https://www.w3.org/WAI/standards-guidelines/wcag/) for accessibility, provide browser plugin to e.g. check contrast
* [System usability scale](https://en.wikipedia.org/wiki/System_usability_scale) as a generic baseline evaluation or user survey
* Examples of UX in command-line software
* `pip` [UX design notes](https://pip.pypa.io/en/stable/ux_research_design/)

## List of Reviewing Criteria

See [the JOSS list of review criteria](https://joss.readthedocs.io/en/latest/review_criteria.html) as a reference - as this work heavily borrows from it.

## Further Reading
- For measuring usability:
- [System usability scale](https://en.wikipedia.org/wiki/System_usability_scale) as a generic baseline evaluation or user survey
- Examples of

## Practical things to improve usability
Landscape of user research approaches based on your question.
(Source: https://designinfluences.com/erika-hall/)

- Understand your user researh question then pick an approach. We discussed specific methods like surveys