DIALS core meeting 2022-04-28

--- name: DIALS core meeting 2022-04-28 tags: core meeting --- # DIALS core meeting 2022-04-28 ## Previous Actions - [ ] dxtbx and read-only base - [ ] Aaron to run testing, check this works - [ ] Planned to merge week of 18th April - [ ] Dan: Reduce zenodo dataset for contiguous nexus - [ ] ND to review: TOFBeam - A time-of-flight beam in dxtbx [`cctbx/dxtbx#492`](https://github.com/cctbx/dxtbx/pull/492) ([prev](https://dials.github.io/kb/core/20220303#proposal-for-a-time-of-flight-beam-in-dxtbx)) - [ ] `std::string` support for messagepack. This is used in XFEL module, should try to fix. [`dials/dials#1858`](https://github.com/dials/dials/issues/1858). ## Agenda ### Contiguous Nexus - [ ] "Contiguous Nexus" [`cctbx/dxtbx#356`](https://github.com/cctbx/dxtbx/pull/356) - [ ] Non-draft state is pending checks against issues @dwpaley raised, and checking for non-contiguous cases [[prev](https://dials.github.io/kb/core/20210715#contiguous-nexus-pr)] - AB wants this to be applied against new nexus writing - RG: Probably useful to mock up an example in the nxmxtests - **Action: Aaron** Reduce down zenodo dataset used in the issue ([`cctbx/dxtbx#93`](https://github.com/cctbx/dxtbx/issues/93)) so that we can include it in dials-data. - Afterward: Coordinate with RG to make sure the ideas in this are integrated into new nexus reading - WIP - Dan has reduced to Six images - 300mb, wondering if this is enough. Discussion about what is required for the files and how small to absorb into the dials-data repository. ### DXTBX read-only base - Issue [`cctbx/dxtbx#444`](https://github.com/cctbx/dxtbx/issues/444) - Read only base by default would be the ideal - Previously discussed downstream use cases and if DIALS needs to be included - PHENIX - depending on the readonly dxtbx stuff - until that is fixed DIALS will be pinned at 3.2 or similar. They also have a hard stop at Python 3.7, due to a slowly changing dependency. - Defer for two meetings - pinning on DIALS 3.2/virtualenv script working around for now - Originally merged in, but turned out to be more involved - **Top Priority** - Reopened under [`cctbx/dxtbx#511`](https://github.com/cctbx/dxtbx/pull/511) - main change is regenerating dispatcher immediately during configuration, this mostly seems to work - AB: going to run in next few days ### DIALS `src/` layout - https://github.com/dials/dials/pull/2077 - Moves to the same `src/` based layout as dxtbx, xia2. This enables us to write a standard `setup.py` to actually install DIALS and it's entry points, like a normal python package. It also provides isolation against accidental namespace imports, namespace pollution. - Once merged, will go through other open PRs (including draft) to update them the the new layout, though this isn't expected to be much of a problem. - Uses the same mechanism as above dxtbx PR, e.g. regenerates the DIALS dispatchers once the environment is updated to properly recognise the package layout - _Relies_ on [`dials/dials@cafaf4`](https://github.com/dials/dials/commit/cafaf4df9454b1a63cac2940322784d0ca020fc7) to force dials to configure before xfel, so that xfel picks up the correct locations - Planning to merge this in over the next couple of weeks #### Possible fallout - Possible issue in the XFEL regression tests that Aaron will communicate over - Running on cctbx Azure XFEL CI on cctbx_project ### Circular XFEL dependency - [`dials/dials#2076`](https://github.com/dials/dials/issues/2076) - xfel has crept up from optional to a hard dependency - `combine_experiments`, `cosym` and a few others use as unqualified imports - Not sure how difficult it would be to resolve this and keep all functionality - `stills_process`/`/indexing/nave_parameters.py` look reasonably dependent - might have to accept that these won't work outside a libtbx bootstrap. ### Dropping Data Blocks https://github.com/cctbx/dxtbx/pull/504 - [x] GW PR - [ ] After PR/PRs, ASB to contact e.g. Mike Wall about this, too. - GW thinks reasonable to merge when these concerns addressed - Waiting on AB now, will get sorted as soon as can ### Next meeting Thursday, May 12th, 4pm UK (BST), 8am PDT.