## About
The CIL [software sustainability and accessibility hackathon](https://ccpi.ac.uk/events/software-sustainability-and-accessibility-hackathon/) will take place from the 7th-9th February 2024 at the Rutherford Appleton Laboratory

We will be working in small groups to look at topics, including:

- Modifying our current web page with the right information, e.g. a quick start guide.
- Infrastructure
- Creating versioned documentation probably using read the docs.
- Embedding testable code snippets and notebooks into webpages
- Developing automatic testing for our code examples and training notebooks.
- Developing contribution policies and a contributor guide

Please add any ideas or code snippets below!

### Draft timetable
| Date | Time | Activity | Location |
|--------------------------|---------------|--------------------------------------------|------------------------------------------------|
| Wednesday 7th Feb | 9:30-12 | Planning and group formation | Access Grid R89 and online |
| | 12-1 | Lunch | RAL Canteen |
| | 1-3 | Group Working | Access grid, F51, F22, atrium, desk space |
| | 3:30-4 | Group updates | Access grid and online |
| | 4-5 | Group working | Access grid, F51, F22, atrium, desk space |
| Thursday 8th Feb | 9.30-10:30 | Group updates and priorities for Day 2 | Access grid and online |
| | 10:30-12 | Group Working | Access grid, F51, F22, atrium, desk space |
| | 12-1 | Lunch | RAL canteen |
| | 1-3 | Group working | Access grid, F51, F22, atrium, desk space |
| | 3:30-4 | Group updates | Access grid and online |
| | 4-5 | Group working | Access grid, F51, F22, atrium, desk space |
| | 5-6 | Pre-Dinner Boardgames | The Atrium, R89
| | 6:30-9 | Christmas Dinner | Lusso, Didcot |
| Friday 9th Feb | 9-10 | Group working | Access grid, F51, F22, atrium, desk space |
| | 10-10:30 | Quick updates | Access grid and online |
| | 10:30-12 | Group working | Access grid, F51, F22, atrium, desk space |
| | 12-1 | Lunch | RAL canteen |
| | 1-2 | Group working | Access grid, F51, F22, atrium, desk space |
| | 2-4 | Conclusions and next steps | Access grid and online |

### Locations

R89 floor 1.


### Contributor sign-up
If you'd like to attend, please sign up at https://forms.office.com/e/ysed4K3Y33
and fill in the form below if you'd like to be involved with any specific task
Name | Skills/interests | Preferred tasks
------ | ------ | ---------------
Hannah | ??? | Landing page
Joshua | Yes | Refactoring (e.g. `framework.py`), typehints (3.6+)+`mypy`
Laura | CIL Knowledge |Improving content of CIL docs
Casper | website framework | [#1579](https://github.com/TomographicImaging/CIL/issues/1579): versioning, redirects to one site, dev-vs-user docs?
Margaret | CIL knowledge | Contributor guide, quick start guide, PR and issue templates, style guide and style enforcing (vis [eqt#98](https://github.com/TomographicImaging/eqt/issues/98))
Franck | ??? | UML class diagrams, publication lists
Gemma| | |
Danica |user interface| expanding landing page design, quick examples, tests, citations, partners, contact us etc
Iwan | IA & Python | Dev-facing doc [docstring/typehinting, etc] (Also some random nitpicks)|
Sunny | Python |quick start, examples, user guide
Edo | | mention SIRF [#1570](https://github.com/TomographicImaging/CIL/issues/1570)?

Team | Topic
---- | ------
|Laura, Joshua & Iwan|Docstrings & documentation
|Casper & Edo| Versioned documentation & Landing page (embedded and tested notebooks)
|Margaret & Danica| Quick start, contributor guide & coding style, Landing page sections
|Sunny & Bill & Hannah|Tutorials and 'how to'
|Franck & Nick| UML generation for documentation and development
|Gemma| CIL installation verification

### What we've got at the moment

- [Documentation on github.io ](https://tomographicimaging.github.io/CIL/nightly/index.html)
- [CCPi website](https://ccpi.ac.uk/cil/)
- [Github README](https://github.com/TomographicImaging/CIL?tab=readme-ov-file#cil---core-imaging-library)

## Material to review

Lots of useful ideas in the CIL software sustainability evaluation
- https://docs.google.com/document/d/1ds6_kEmdF6eBxfmVdlKLb-t6MtGoyIZJTR0kWPYd9tQ/edit
- https://github.com/TomographicImaging/CIL/discussions/1004
- Some questions which don't fit clearly into any of the tasks below
- 7.2: open communications protocols
- 9.1: accessibility conventions and standards
- 10: having a stable and backed-up code repository
- 12: tests
- 13.8: If your software is developed as an open source project (and, not just a project developing open source software), do you have a governance model?

We already have some github issues
- https://github.com/TomographicImaging/CIL/issues/1233
- https://github.com/TomographicImaging/CIL/labels/documentation

See other useful links at https://discord.com/channels/929016277266219038/972063873228238878

## Task 1: Expanding website

- ### Github templates for issues and PRs
https://github.com/electron/electron/issues/new/choose
https://github.com/mantidproject/mantidimaging/issues/new/choose

- ### Landing page with necessary information
- Inspiration:
- https://the-turing-way.netlify.app/welcome
- https://cuqi-dtu.github.io/CUQIpy/
- Link to user showcase demos
- Rendered on the webpage? [Like the matplotlib gallery?](https://matplotlib.org/stable/gallery/index.html)
- Link to publications quoting CIL
- Relevant [sustainability evaluation](https://docs.google.com/document/d/1ds6_kEmdF6eBxfmVdlKLb-t6MtGoyIZJTR0kWPYd9tQ/edit) questions:
- 1.2: who is the software for?
- 4.2: quick-start guide
- 5.2: describe what support is available for users -> how to contact us
- 13.2: list projects and users associated with your project
- 13.4: list important partners and collaborators
- 13.6: list third-party publications that refer to your software or link to them
- 15.7: recommended citation for your software
- 16.1: project roadmap
- 16.2: describe project funding and time scales
- 16.3: timely announcements of the deprecation of components

- ### Developers guide
- Inspiration:
- https://docs.vtk.org/en/latest/developers_guide/index.html
- Add ideas here
- UML class diagram
- e.g. `pip install pylint; pyreverse -o html Wrappers/Python/cil/framework/framework.py` :)
- ### Contributors guide
- Inspiration:
- https://github.com/alan-turing-institute/the-turing-way/blob/main/CONTRIBUTING.md
- https://the-turing-way.netlify.app/project-design/persona/persona-contributors.html
- Relevant [sustainability evaluation](https://docs.google.com/document/d/1ds6_kEmdF6eBxfmVdlKLb-t6MtGoyIZJTR0kWPYd9tQ/edit) questions:
- 14.2: Do you have a contributions' policy
- 14.3: Is your contributions' policy publicly available
- Legal:
- Contribution License Agreement ([CLA](http://oss-watch.ac.uk/resources/cla))
- CIL [PR template](https://github.com/TomographicImaging/CIL/blob/master/.github/PULL_REQUEST_TEMPLATE.MD)
- Need to [keep track of CLA](http://oss-watch.ac.uk/resources/cla#associating-commits-to-clas)
- GH keeps track of comment history so current PR template is enough
- Casper strongly objects to any CLA framework more complex[^cla-google] than the current PR template. More templates (e.g. dev vs extern contrib) are fine :)
- NB github automatically renders `https://github.com/TomographicImaging/CIL/commits?author=<github_username>` including [commit-level coauthors](https://docs.github.com/en/pull-requests/committing-changes-to-your-project/creating-and-editing-commits/creating-a-commit-with-multiple-authors) - though it's probably better to rebase + merge rather than squash
- Add ideas here

[^cla-google]: https://github.com/google/code-review-bot

- ### User guide
- Inspiration:
- https://cuqi-dtu.github.io/CUQIpy/user/index.html
- Getting started
- Links to demos
- Links to code tutorials and snippets (see task 3!)
- Relevant [sustainability evaluation](https://docs.google.com/document/d/1ds6_kEmdF6eBxfmVdlKLb-t6MtGoyIZJTR0kWPYd9tQ/edit) questions:
- 4.3: how to deploy and use the software
- 4.5: troubleshooting guide
- 5.1: guide for how to get help with using software
- 5.2: Describe what support is available for users
- 11.8: Do you have tests that can be run after your software has been built or deployed to show whether the build or deployment has been successful?

## Landing page draft
------------------


### Danica and Margaret comments DAY2:
- TO DISCUSS: Publishing the automatic testing in the webpage? (Question 12.4)

all version are re run and tested.
every merge is tested. add this information in "what is CIL"

### Notes on the landing page:
- TBD: How often do we update each section in the webpage? Who is responsible?

### TO DO:
- Move videos near tutorials.
- Add github link. add to image draft.
- Add showcase links.
- done. Add github projects links to show the roadmap. Add this to the FAQs.
- Add to image draft. Add section: "Our projects" near meet the team, contact us. This includes project roadmap. Add other projects we work on. Make list.

### Website HackMD files:

- Installation guide https://hackmd.io/@Rd1qr8wCSOSs4LqQqF4NPQ/HkiOqmfoT
- About CIL https://hackmd.io/@Rd1qr8wCSOSs4LqQqF4NPQ/Skqw5mMja
- Landing page https://hackmd.io/@Rd1qr8wCSOSs4LqQqF4NPQ/S1oRYQzja/edit
- Contributor guide https://hackmd.io/@Rd1qr8wCSOSs4LqQqF4NPQ/rkpr5QMi6
- Help and FAQs https://hackmd.io/@MDuff/BJXPJBzsa
- User community https://hackmd.io/@MDuff/HJo0tvGi6
- Quick start guide
- Demos and tutorials https://hackmd.io/@XVREuqqbQqyweSiwagjjRw/BkJo8wmo6
- Contact us

### Page sections

- Search button + accessibility options eg change font size and colour scheme
- Welcome sentence:
- What we are (user driven community)
- Our software is for...
- Our Licence is.
- Please cite us
- Please contribute.
- Link to GitHub tomographic imaging (not only CIL, we also have the showcases and demos)

- What is CIL
- User driven community
- Our two user groups
- Free software
- Born out of the CCP
- Quick intro to CT
- Quick intro to iterative reconstruction
- Add link to CIL repository.
- Talk about automatic testing.

- Quick start guide + basic examples
- Simplest installation
- How to test if installed correctly
- One example of using CIL
- FDK reconstruction?
- Quick start for Mathematicians
- Quick start for XCT users
- Installation guide:
- ~~Third party dependencies (version number)~~
- binary distribution with conda, docker
-
- Tested for Windows and Linux, not for MacOSX
- Automatic verification
- Help and FAQs
- Dictionary of our specific terminology.
- Alphabetical order or divide into categories.
- FAQs from discord and others.
- Link to the "contact us" pages. State how long we take to reply normally and how best to get a quick response (discord).
- See townhall notes to draft this.

- Our community.
- List third party publications + our publications that use/cite CIL.
- List our presentations at conferences (kept up to date from the presenter).
- Discord, talk about how many people we have, our channels, perhaps screenshots.
- News from our community
- List projects in which developers/contributors work.
- CIL-User-Showcase, add link and explain what it is.

- Demos and tutorials.

- **How-to's**


- Contributor guide
- What we define as a contribution
- Issues, code examples, full on PRs
- When talking about the showcase, link to the readme of that repository.
- On any and all parts of CIL
- Example workflows
- E.g. you send us an error message, we help you fix it. You send us an error message, it exposes a bug, we put in an issue crediting you and it gets fixed.
- E.g. you send us some code that you have been using. We clean it up, optimise for CIL, add unit tests and merge it into CIL, crediting your contribution
- E.g. you put in a PR, we work with you to get it ready to merge and it gets reviewed by 2 developers...
- Developer installation and guide to build locally and modify the code.
- Style guide for CIL
- Explain the rules about the copyright of the contribution and our licencing rules.
- Deprecation policy (nothing on this yet, Gemma?)
- Template for contributor issue
example:
https://github.com/electron/electron/issues/new/choose
https://github.com/mantidproject/mantidimaging/issues/new/choose

We like the simplicity of mantid. We like the form from electron.
We want to keep the form as simple as possible.
Forms
- Bug report
- Feature request
- Link to discord
- Link to contact us section in the landing page.

- Contact us
- Discord
- Group email (TBD)
- Mailing list?
- Github

- Video guides (TBD).

- CIL documentation
- User documentation.
- Developer documentation.
- API
- UML diagram.

- Meet the team.
- List core developers.
- Pics + short bio.
- Link to the "Contact us" section.

- Our projects.
- Mention our interests.
- Link GitHub projects to show our roadmap.
- CIL GUI
- DVC
- ...(Please add)

- Upcoming events
- These will include our trainings and hackathons and other meetings organised/co-organised by us.
- Add smaller below "View all events"
- past
- future
- Link to ccpi for the list of events.
- We could be general and say "We run one user meeting and X hackathon per year."
- Add links to discord event.

- How to cite CIL

- Funded by
- CCPI
- ALC
- STFC
- State our guarantee of funding.

- Website copyright statement in the bottom.

## Task 2: Improving documentation
- ### Versioned documentation
- Ideas
- Read the docs

- ### Documentation content
- Tackle open issues
- Review the docs for each module of CIL and check for missing/outdated info
- Relevant [sustainability evaluation](https://docs.google.com/document/d/1ds6_kEmdF6eBxfmVdlKLb-t6MtGoyIZJTR0kWPYd9tQ/edit) questions:
- 4.4: comprehensive guide to all commands, functions and options
- 11.4: does documentation list all third-party dependencies?
- 11.5: does documentation list the version number for all third-party dependencies?
- 11.6: list the web address, and licences for all third-party dependencies and say whether the dependencies are mandatory or optional?

## Task 3: Embedding testable code snippets and notebooks into webpages
- ### Some simple tutorials
- Inspriation:
- https://cuqi-dtu.github.io/CUQIpy/user/_auto_tutorials/index.html
- ### Code snippets
- How do we host them?
- Download notebook/ python file button
- Copy button on cells
- Ideas:
- https://jupyterbook.org/en/stable/intro.html

### How to embed notebooks

## Task 4: Plan video content
- What videos would be useful?

## Task 5: Refactoring codebase

Modifying the structure of CIL would make it much easier to document and contribute to. Some suggestions (in no particular order):

- Decide to properly drop Python 2 support. It's depricated as of 2020 so potential CIL users should not be relying on it, and I'm pretty sure maintaining Python 2 support blocks us from using typehints (Python 3.5+).
- Have proper interface conditions (probably using `ABC`s). It might be helpful to use `typing.Protocol` to infer the interfaces of some things which don't currently have a proper interface.
- Reduce or remove instances in which functions (particularly constructors) are taking `**kwargs` as the main input. Where this is happening we should consider revising the class structure, as it could be that different designs (e.g. favouring composition over inheritance) would allow us to have a slightly larger number of simpler objects with straightforward constructors. In cases where keywords are being used, the keywords should be spelled out as much as possible in the function input signature. E.g. `Algorithm` could be split into an `Iterator` part for the flow control and a `Strategy` pattern object which describes the update step.
- Move towards functions having a single, consistent return type where possible. Functions which operate on input using side-effects could return the mutated input.
- CIL's version numbering seems to be done based on date rather semantic versioning, and thus doesn't tell the user whether an update should be expected to be backward-compatible or not.
- Typehint where possible.
- Naked strings or numbers used in type definition should be eliminated wherever possible as they can't be typechecked and are easy to get wrong; use literal or enumerated type instead if this needs to be done. Also having lots of switching during construction probably indicates a problem; maybe those should be separate classes.
- > Iwan: Can always use Literal's for typechecked strings, rather than enums

- Maybe use something like `xarray` for multidimensional labelled arrays, would be pretty compatible with numpy and HDF5 I/O would come basically for free.
- Base class definitions are currently stored in unintuitive places (e.g. `framework.py`). Some of these files are also absolutely enormous (several thousand lines) and should certainly be split up.

### UML Diagrams

**Packages have circular dependencies due to python imports**

For example: in the case of cil.recon:

cil/recon/
|- __init.py__
|- FBP.py
|- Reconstructor.py

In FBP.py, Reconstructor from Reconstructor.py is imported:

cil/recon/FBP.py:
from cil.recon import Reconstructor

This causes a circular dependnecy because "`from cil.recon`" imports "`__init__.py`", which contains:

__init__.py:
from .Reconstructor import Reconstructor
from .FBP import FBP
from .FBP import FDK

Fix is to use relative improts:

cil/recon/FBP.py:
from cil.recon import Reconstructor

### Standards / style guide [Master Issue: #1683](https://github.com/TomographicImaging/CIL/issues/1683)

- Numpy doc format
- https://numpydoc.readthedocs.io/en/latest/format.html#documenting-classes
- If the function has typehints, omit type hinting from the docstring
- e.g:

def __init__(self, input:AcquisitionData, image_geometry:Optional[ImageGeometry]=None, backend="tigre"):
    """Abstract class for a tomographic reconstructor.

    Parameters
    ----------
    input
        Input data for reconstruction
    image_geometry
        Image geometry of input data, by default None
    backend
        Engine backend used for reconstruction, by default "tigre"

    Raises
    ------
    TypeError
        Raised if input data is not an AcquisitionData class
    """

- Class Structure *(basically an inverse of c)*:
1. `__init__`
- Init is first due to its documentation existing in the class's docstring, and properties may be large
2. Properties
3. Static functions
4. Functions
6. Private functions
7. Special functions (`__str__()`)
- Functions should return `out` if passed in, not `None` #1657

## Notebooks

1. Binder directory
2. Geometry https://github.com/TomographicImaging/CIL-Demos/blob/main/demos/1_Introduction/00_CIL_geometry.ipynb