--- name: DIALS meeting 2020-09-30 tags: meeting notes --- # DIALS meeting 2020-09-30 [![hackmd-github-sync-badge](https://hackmd.io/dzOFHOnVSp-CckUG8BjFbg/badge)](https://hackmd.io/dzOFHOnVSp-CckUG8BjFbg) ### New flex data types It looks like the institutional knowledge about how flex array innards work has dried up. Consequently adding new supported types is difficult. Could we only keep the functions exposed to C++ rather than Python? NKS may have the needed knowledge. A question to cctbxbb hasn't evoked any replies yet. ### JBE bitshuffling Change dxtbx `get_raw_data` functions to use bitshuffle directly, as we have more knowledge about the data format than generic HDF5 calls do. ([dxtbx #219](https://github.com/cctbx/dxtbx/issues/219)) This would put performance at par with Durin. Takes 20-30% off read decompression time, plus avoids needs for object conversion/copying. Opens up parallel chunk processing as we don't need to go through HDF5 chokepoint. ### proper NeXus support `NearlyNexus` should go away. NearlyNexus is not tolerant. Anything not matching the exact layout will fail, eg. must contain the string `Eiger` in a magic place. We also don't support any non-trivial geometry ([dxtbx #200](https://github.com/cctbx/dxtbx/issues/200)), some hacking towards that was made in [dxtbx #206](https://github.com/cctbx/dxtbx/pull/206), but that is not merge-ready by any measure. BW and DGW to follow up after meeting. Test data exist in `dials/data-files` — see [PR dials/data-files#12](https://github.com/dials/data-files/pull/12) and [PR dials/data#161](https://github.com/dials/data/pull/161). A test seems to have been introduced in [PR dxtbx#164](https://github.com/cctbx/dxtbx/pull/164). ### fp3 (Faster parallel processing pipeline) This is "fast-dp powered by dials", PR is [dials-scratch#5](https://github.com/dials/dials_scratch/pull/5). It is already much faster than xia2. Would benefit from parallel integrator and any improvements in data handling should be very visible, too. DGW: Can we use this to bootstrap a more careful DIALS processing run? Separate new package? Probably yes, because separate documentation and use-case. Merge into fast_dp? Not at this time. ### 2D detector projection [dxtbx#224](https://github.com/cctbx/dxtbx/pull/224) This would fix the image viewer for single panel detectors. Obviously not useful for metrology and/or I23. If useful it could go into the detector module. Or a projection mode in the image viewer. And in the report. We need to throw different detector images at it, to see how it fares. ### Maths questions: Randy and PHZ Q: Are we interested in detecting different spot classes early on in processing? This is relevant for order-disorder (OD) structures. AP: GW to sort out conference call ### image viewer Discuss next time.