DIALS meeting 2021-06-23

--- name: DIALS meeting 2021-06-23 tags: meeting notes --- # DIALS meeting 2021-06-23 ## Agenda 1. BHW: Experiments and identifiers (again!) — proposal for retiring imageset ID in favour of `Experiment`s without a `Crystal`. Related to [previous discussions](https://hackmd.io/MK6KYBMJS6ykO5XyKK3T1Q#Experiment-Identifiers). Further context [on Slack](https://diamondlightsource.slack.com/archives/CJJM7LA21/p1624292883384400). Shall we agree a day for a sprint? ## Minutes ### Experiments and identifiers BHW suggests a sprint (GW suggests four-ish people for two days). Since we have a good idea of what the output should look like, perhaps this is a good opportunity for test-driven development and we should start by writing some tests. DGW suggests that DMcD's approach ot the neutron development might be appropriate here — start at the beginning of the pipeline, fixing as you work through from import to scaling, then do a second pass to tidy up. GW suggests a GH project. DGW points out that [there already exists one](https://github.com/dials/dials/projects/19). We should use this to coordinate. BHW volunteers to do admin. GW suggests that we should wait until RJG is back from parental leave. DGW suggests that the week of the 19th July would be his preference, for offering consultation and review to those involved in the sprint. #### Interested parties: - EP - DMcD - BHW - JBE - DGW (consult & review) - GW (consult & review) - RJG? (not present in this meeting) #### Thoughts on specific implementation details: ###### DGW, `dials.refine` From experience, trying to pepper the code with `if experiment.crystal:` gets messy. `dials.refine` already makes a copy of the `Experiment`s on which it is working at the start. If we restrict this to only taking a copy of those `Experiment`s with `Crystal`s, then we might not need to make many changes to the rest of the code. This then suggests that a sensible place to start would be the functions required to get `Experiment`s and associated reflections that do/don't have `Crystal`s, and to overwrite just those reflections that are associated with a given `Experiment`. ### Updates on PRs - GW: [#1761](https://github.com/dials/dials/pull/1761) fixes [#1734](https://github.com/dials/dials/pull/1734), now without antagonising anyone by causing a regression of [#1213](https://github.com/dials/dials/pull/1213). DGW is happy with this now. - GW: [#1758](https://github.com/dials/dials/pull/1758) — one-line fix to yet another image viewer bug. - ND: [#1760](https://github.com/dials/dials/pull/1760) fixes `dials.combine_experiments`'s handling of unindexed reflections. This doesn't fix the comparable behaviour in `dials.scale`. It buys us time until we fix the bookkeeping of experiment IDs. It needs testing. ND requests test data and reviewers. GW & BHW volunteer to review. BHW to send ND data. ### Other updates - ND: Flumpy is held up behind the next release of CCTBX. - LFM: Hasn't shown DUI to the newer members of the group. Suggests a demo. General agreement that this needs to be a separate dedicated meeting. - DGW: Scan-varying refinement — JBE solved a tricky multi-turn rotation data processing problem for a user at a recent workshop. The solution to that problem was so neat (involving use of sparse matrices to avoid manipulating masses of zeros), that perhaps we should use it more generally. Needs assessment. There already exists a discussion, [#1521](https://github.com/dials/dials/discussions/1521). ## Actions - [ ] BHW to put some dates in the diary for `Crystal`-free `Experiment` work and retirement of imageset ID and coordinate via [the relevant GH project](https://github.com/dials/dials/projects/19). - [ ] DMcD, EP, JBE, BHW to divvy up the writing of test cases for multi-sweep bookkeeping amongst themselves. - [ ] BHW or other to bring RJG up to speed when he returns from parental leave. - [x] BHW to send ND data to test [#1760](https://github.com/dials/dials/pull/1760). - [x] BHW to test [#1760](https://github.com/dials/dials/pull/1760) independently. - [ ] LFM to advertise a date for a DUI demo.