# Wrapper-Merging Task Force

### Status 2021-04-16:
* **AH:** Need to connect up ISPyB service via recipe
* **MG:** Add cryolo envionment `module load EM/cryolo/1.7.6` to the recipe and indirector script.
* **DH:** looking at simulated data collections

### Status 2021-04-15:
* DH has been making releases
* **AH:** Need to connect up ISPyB service via recipe
* **MG:** `cryolo_relion_it.run()` calls need to use correct environments:
* relion: `module load EM/relion`
* cryolo: `module load EM/cryolo/1.7.6`
* Could be encoded in the recipe, eg.:

  "prepare-environments": {
     "relion": ["module load EM/relion"],
     "cryolo": ["module load EM/cryolo/1.7.6"],
  }

* **DH:** looking at simulated data collections

### Status 2021-04-14:
* licensing resolved, branch merged, python-relion 0.3.1 released and installed
* Can trigger relion processing using `zocalo.go -p 6844019`
* Does create an ISPyB program entry, cluster job running underneath
* `/dls/m12/data/2021/cm28212-1/tmp/zocalo/raw/_1`
* Found (and fixed) the first few bugs
* [Synchweb page](https://ispyb.diamond.ac.uk/dc/visit/cm28212-1/) still broken, no reply from BAG

### Meeting on 2021-04-08:
* See *if* it works
* kind of
* can't test it beyond a certain point because the cluster jobs are lacking the correct environment
* Figure out what needs to be done to make it work
* PR https://github.com/DiamondLightSource/python-relion/pull/81
* PR https://github.com/DiamondLightSource/python-relion/pull/82
* housekeeping: move external jobs to `relion.external_jobs`
* we can use entry points to enumerate them
* threading is not race-condition free yet
* **DH:** data analysis resends information every time it goes through the `while` loop
**Completed** :heavy_check_mark:
* **AH:** synchweb stop-file logic needs revisiting
* This is for Synchweb to tell the wrapper to stop
* the stop file will be created by the zocalo recipe `relion-stop.json`
* possibly by abusing the archiver service? :thinking_face:
* location possibly `${VISIT}/tmp/${processingid}.stop`, to be defined in the recipe, and can then be accessed as:
* `self.params["stop_file"]`
* Need to define this value in the `ispyb-relion` recipe
* Wrapper needs to periodically check for the stop file and then go into some abort-sequence
* make a release (post CP-signoff)
* **DH:** have a look at simulated data collections/analysis

### Meeting on 2021-04-07:

* We'll work on the branch `future` until CP has signed off the pull request
* **DH:** Make flake8 happy
**Completed** :heavy_check_mark:
* **DH:** Rewrite unittests to pytest
**Completed** :heavy_check_mark:
* **DH:** Make tests pass
**Completed** :heavy_check_mark:
* **AH/MG:** Consolidate the three wrappers:
* **A**: `/zocalo/wrapper.py` -- delete that one
* **B**: `/src/relion/cryolo_relion_it/relion_zocalo_wrapper.py`
* **C**: `/src/relion/zocalo/wrapper.py` <- this is the one that will remain
* **AH** to integrate B into either C or elsewhere (eg `/src/relion/zocalo/__init__.py` - the exact place doesn't really matter yet)
* end goal is that **B** is deleted
* **MG** to sort out C including the thread logic, leaving the `start_relion()` function itself
**Completed** :heavy_check_mark:
* **MG:** housekeeping: symlink stuff is in zocalo package now
* https://github.com/DiamondLightSource/python-relion/pull/76
**Completed** :heavy_check_mark:
* **sort of everyone:** Further work on the wrapper inside python-relion
* Use separate threads to run `relion.cryolo_relion_it.cryolo_relion_it.run_pipeline` and Anna's analysis code
* Suggestion is the relion/cryolo parts run in the main thread
* Interpretation of results runs in a daemon thread alongside
* Then just need to make sure if the main thread exits the interpretation thread catches up and finishes up
**Completed** :heavy_check_mark:

### Meeting on 2021-03-29:

Todo-list then:
* **Dan:** Merge https://github.com/DiamondLightSource/python-relion-yolo-it/pull/3
* Dan to clean up: add option to use old code or new code, and check that the "old code" version runs exactly like the actual old version
* CP *maybe* to review
* Dan to verify that the old control flow still works and produces the same output
* then merge
**Completed** :heavy_check_mark:
* **Markus:** Basically copy more or less everything from the `d_rel3.1` branch of the `python-relion-yolo-it` repository into `python-relion`
* Copy files with history
**Completed** :heavy_check_mark:
* **Anna:** Merge the dlstbx wrapper work - relion-wrapper branch
**Completed** :heavy_check_mark:
* **Anna:** Move the wrapper in dlstbx to python-relion (remember entrypoints)
**Completed** :heavy_check_mark:
* **Anna/Markus:** Create a new environment/module maintained by Jenkins to deploy python-relion
**Completed** :heavy_check_mark:

Next meeting:
* 5 minute progress report in Thursday meeting
* Wednesday, 7th April, in/following catchup