DIALS core meeting 2024-08-08

--- name: DIALS core meeting 2024-08-08 tags: core meeting --- # DIALS core meeting 2024-08-08 ## Previous Actions - [ ] **ND** Investigate getting `psana` tests running on the DIALS xfel-regression testing - [ ] **ND**: Investigate using stable-nightly conda builds of cctbx (e.g. cctbx tests) longer term - [ ] **JBE**: H5 on-disk experiment: review practicalities of using this & welcome people looking at James's work above. round trip example files could be useful - exists in dials.export. - [ ] **ND**: Sync new releases with current state of dials/cbflib and main branch. Keep synced. ## Agenda ### OptionParser - Dials east has reserved time to look into removing some of the limitations on this. Graeme notes: https://github.com/dials/dials/pull/1265 - Set for Tues 21st - [https://github.com/dials/dials/compare/main...new-argument-parser](https://github.com/dials/dials/compare/main...new-argument-parser) branch, Ben not cleaned for review yet ### JBE Demos - Previously [[prev](https://dials.github.io/kb/core/2024-04-04)]: - Draft PR https://github.com/dials/dials/pull/2567 - GW suggested making a special release in order to test this changeset at DLS (in new year). - Wait until PR notes addressed, then build this as a test release for wider consumption - Not been touched much since NY - Keeps coming up as something that would be useful - James has rebased this, updated tests, Rick and Kevin feel OK with this => need to pick up some PR comments - Hope to merge in near future (next month) - Aaron points to [notes](https://github.com/cctbx/cctbx_project/tree/master/xfel/conda_envs) - Previous Actions: Still relevant? - **AB**: Run this against Xfel-regression tests: AB pls can you run this James? - ~~**ND**: Make test release using these branches~~ - Few more issues outstanding - 3.19 out now so good time to merge testing - providing xfel-regression works - Paper taking next two weeks (pick up next time) - Jun 13th - Still writing, starting to pick up things. Next time should be better. - H5 Reflection Table PR: https://github.com/dials/dials/pull/2677 - Discussion about shoeboxes ### Making ellipsoid integrated data work with cctbx.xfel.merge - JBE: I had prepared this, but now I'm not sure this is the best approach, discuss: https://github.com/dials/dials/pull/2661 Discussion of how to handle the difference between partiality and so on between the DIALS ellipsoid model / xfel.merge postrefine was discussed: since the intensities are modified "it's complicated." Need to make sure not to correct for partiality _twice_. Per-image scale factors are computed before postrefinement but Wilson B is only optimised during postrefinement - so this is meaningfully different from how dials.scale works. Conclusion: share custom script - 6/13 - One further complication - in cctbx.xfel.merge, only 'outlier rejection' can occur as part of repredictions during postrefinement, so this not available for ellipsoid at moment - probably want to be able to handle ellipsoid integrated data properly in cctbx.xfel.merge, rather than just applying the partiality correction coming out of dials + turning off postrefinement. ## AOB? - Quick discussion of https://github.com/dials/dials/pull/2718 - `const_iterator` came up, now included in subsequent commit - Rick wants to talk about including laue-dials tests in the wider dials test suite - how long does this take to run? Tests take a few minutes to run. For PR's we only run xfel tests on one combination of build which would probably be OK. Rick should be able to look at the Azure test scripts and add Nick points at https://github.com/dials/dials/blob/b9aae978b6d818a7dc17e7b89b66890c2e040a4e/.azure-pipelines/unix-build.yml#L109-L143 We probably should document how to get downstream tests integrated. GitHub actions could work but again a big job of work. In future, maybe adding hooks to allow downstream software to run tests based on DIALS PRs could be useful. - to look into the Azure permission issue - Adding the fast-feedback-indexer to DIALS https://github.com/dials/dials/pull/2717 How should we proceed in terms of (i) optional dependencies and (ii) things which depend on GPU. This has a Python interface to C++ / CUDA which needs building & installing. We maybe need to reload state on how the loadable algorithm stuff works. This is on list to look at. For reference, nanobragg does similar things. https://github.com/cctbx/dxtbx/pull/744 - tests fail please investigate because they shouldn't be - picking up numpy 2 causing failures - need to pin numpy in dxtbx https://github.com/dials/dials/pull/2711 - incremental creation of masks - accumulate masks from reason A, B etc. so passed in masks get propogated. to review - CC1/2 sigma-tau? Why factor of 2 in https://github.com/cctbx/cctbx_project/blob/master/cctbx/miller/__init__.py#L5269 - one for ? ### Next meeting Thursday, August 22nd, 4pm (BST), 8am (PDT)