--- name: DIALS core meeting 2021-08-26 tags: core meeting --- # DIALS core meeting 2021-08-26 [![hackmd-github-sync-badge](https://hackmd.io/RQncDJGAThKzc3FcdfD8Mw/badge)](https://hackmd.io/RQncDJGAThKzc3FcdfD8Mw) [Previous meeting: [2021-07-29](https://dials.github.io/kb/core/20210729)] ## Previous Actions * [ ] ND: conda-forge pycbf: Make new release to use dials-data directly for tests - [ ] "Contiguous Nexus" [`cctbx/dxtbx#356`](https://github.com/cctbx/dxtbx/pull/356) - [ ] Non-draft state is pending checks against issues @dwpaley raised, and checking for non-contiguous cases [[prev](https://dials.github.io/kb/core/20210715#contiguous-nexus-pr)] - [ ] dxtbx `src/` layout [[prev](https://dials.github.io/kb/core/20210715#src-layout-for-dxtbx)] - [ ] Ask cctbxbb if anyone has knowledge or experience of re-export of dispatchers on windows - [ ] Nonconventional P1 indexing [[prev](https://dials.github.io/kb/core/20210715#nonconventional-p1-indexing)] - a possible problem with indexing nonstandard settings in P1 - [ ] Dan: try and demonstrate this on a simple/lysozyme example and file an issue ## Agenda ### Removal of DataBlock - Moving to 3.7 for all tests has exposed warnings that datablock is still being used in tests - Do we need to keep datablocks? The argument before was that since it's only post-import, re-importing the images isn't overly burdensome - dials refers to datablocks in tests and documentation. - Upgrading datablock `DeprecationWarning` to `UserWarning` worth doing? - Would also suggest deprecating all other functions/classes in `datablock.py` - 2020 LS49 paper apparently depends on datablocks quite heavily - Would need to be checked - Make a roadmap plan for timescales to do this over - make a DC - long term plan - ~ 6 months ### dxtbx `src/` layout - Aaron has feedback from Billy; Two issues with current src/ layout - Modifies the `conda-base` folder - this is causing the windows problems - pip install fails on supercomputer bases when you cannot alter the base python that are read only - `pip install --no-deps -e` apparently contacts the internet - cannot do on - Until fixed will probably pin dials - All this applies to xia2 also - MG: - Sounds like a bug in `pip`. Did you report this? - ND found a related issue - Try installing with `python setup.py develop`. `strace` says no network interaction. - Or, if you don't care about editable installations, just drop the `-e`. According to `strace` this should also be fine. - Outcome: This is all resolved. ### Mirror when stable - Action occasionally fails - even when everything else passes - e.g. [`dff4fc1`](https://github.com/cctbx/cctbx_project/runs/3178538760) - Sends an email to developer of most recent commit - MG: https://github.com/cctbx/cctbx_project/pull/646 should fix - Note: DIALS selects the python version to install using linux.txt with the conda option eg. [`conda-forge::python=3.7.*`](https://github.com/dials/dials/blob/d895ad2d0b5a6be86da402566d357556ab005afc/installer/bootstrap.py#L118) - Remove old environment files ([PR #1865](https://github.com/dials/dials/pull/1865)) ### [cbflib conda-forge package/pycbf](https://dials.github.io/kb/core/20210520#cbflib-conda-forge-package) * Outstanding: [dxtbx#368](https://github.com/cctbx/dxtbx/pull/368) for optional usage in dxtbx, no binary dependence on cbflib/_adaptbx if present. * Still stray issue with dataset `SPring8_ADSC_SN916` being the only one requiring `FormatCBFFull`. One possible solution is adding a special Format: [`dials/dials#366`](https://github.com/cctbx/dxtbx/pull/366) * Still need to test with `labelit`/`labelit_regression` once the test suites actually work * All except one labelit tests are now working. On NKS todo list to fix. * Progress on CBFlib 0.9.7? ### flumpy -- flex/numpy bridge - Additional issues seen: - ~~[`cctbx/dxtbx#406`](https://github.com/cctbx/dxtbx/issues/406) -Converting numpy arrays that don't own their own data~~ - ~~[`cctbx/dxtbx#405`](https://github.com/cctbx/dxtbx/issues/405) - Converting non-contiguous numpy arrays~~ - ~~scitbx arrays appears to have origin property that could be used for this?~~ doesn't appear possible. Throw helpful errors. - Had additional issue: - ~~[`cctbx/dxtbx#415`](https://github.com/cctbx/dxtbx/issues/415): Still some rogue `size_t` issues~~ - All know/outstanding flumpy issues dealt with. Have started using e.g. [`dials/dials#1813`](https://github.com/dials/dials/pull/1813) and reports are so far good. ### Deterministic Experiment Identifiers - PR: [`dials/dials#1864`](https://github.com/dials/dials/pull/1864) ## Deferred to next meeting ### Next meeting Thursday, September 9th, 4pm UK (BST), 8am PDT.