---
name: DIALS core meeting 2020-08-11
tags: core meeting
---

# DIALS core meeting 2020-08-11

[](https://hackmd.io/WASunvTEQHScXh-zzoSAsQ)

## Previous Actions

* [ ] ND: rewrite contribution guidelines (as PR) to include towncrier/newsfragments
* [x] GW: write comment on [dials#1327](https://github.com/dials/dials/issues/1327)
* [x] MG: fix xia2 issues re. Python 3.8 ([xia2#510](https://github.com/xia2/xia2/pull/510))

## Relevant conda-forge migrations

package | status
-- | --
Boost 1.72 | 91% complete
Python 3.8 | 94% complete
HDF5 1.10.6 | 79% complete

[SCons 4.0 went into conda-forge over the weekend](https://github.com/conda-forge/scons-feedstock/pull/16).

## Agenda

### Python 3.6 deprecation

Python 3.6 deprecation proposal / [NEP-29](https://numpy.org/neps/nep-0029-deprecation_policy.html) / [dials#1327](https://github.com/dials/dials/issues/1327)

Discussion outcome:
* We will follow #1327
* GW wrote comment on issue
* Item can be removed from agenda
* Worth documenting outcome in the `/kb` repository?
* GW: How about collecting in a GitHub project in columns 'proposed' & 'implemented'

### Next default Python version

Suggestion: moving to Python 3.8 as the next default, skipping 3.7 (cf. [#1328-comment](https://github.com/dials/dials/pull/1328#issuecomment-655812577))
* Development platform is independent of supported versions. Those will include conda-forge-supported versions at the minimum.
* xia2 currently has issues with 3.8
* these have now been resolved ([xia2#510](https://github.com/xia2/xia2/pull/510))
* XFEL regression tests run on 3.8
* Action **MG**: Make it so.

### Overall architecture discussion

*left for next meeting as ND absent*

full conda or alternatives?

* move from libtbx dispatchers to normal python dispatchers
* allow regular python commands access to the libtbx-modules namespaces
* bigger issue, needs more discussion and figuring out how to do this and how to migrate
* Should we remove the DIALS installer from cctbx bootstrap?
* We don't advertise or test or use cctbx bootstrap any more. Anyone using it may end up with an unsupported environment.

### New installer

*left for next meeting as ND absent*

Following from the overall architecture discussion: How should a new installer look like?

### Open up DIALS core meetings

Open up these DIALS core meetings to DIALS-W and the world in an effort to [move DIALS towards a stage 3 project](http:/urssi.us/blog/2019/02/25/software-incubator-workshop-a-synthesis/)
* [x] Invite to meetings in dials-support and gitter
* [ ] Change time to later in the day
* try 4pm UK time, 8am US time
* [x] Make meeting minutes public
* Item can be removed from agenda?
* Yes

### DIALS master stable

DIALS master stable / [dials#1353](https://github.com/dials/dials/issues/1353)
* Currently evaluating on xia2 (1 month = 24th of August)
* How would we distinguish with pull requests that should be reviewed and pull requests that can be merged as soon as the tests have passed?
* MG: labels may be an option? GitHub plans to introduce an auto-merge feature for pull requests later this year. We could probably do this right now using github actions.
* GW: pull requests can be moved to draft status to express "hang fire, I would like to look at"
* MG: A label saying "merge as soon as tests pass" may alleviate concerns from ND and myself about pull requests with a very short shelf-life
* ASB: Sounds like something that should be mentioned in `CONTRIBUTING`
* ASB: cctbx is running CI jobs in branches in batch mode. May want to consider that for DIALS/dxtbx
* Fundamentally it's possible that tests fail because of an upstream repository change (eg. cctbx/scitbx/constants breaking xia2)
* May have to monitor this. This could be resolved by using semantically versioned upstream releases (for cctbx_project) and stable build mechanism (for dxtbx) but comes with other drawbacks.

### dxtbx json/msgpack performance

ASB reports json being problematic when importing 91208312 experiments, as everything is stored in a single file, so everything needs to be read at once.

* GW: cf. [Load before heat death (dxtbx#118)](https://github.com/cctbx/dxtbx/pull/118). At the core of it are bad design choices in dxtbx.
* Future discussion topics:
1. treatment of large serial datasets
2. using an HDF5 backend for reflection files
* ASB: I want `check_format` to go away.
* GW: discuss in a separate meeting early UK, late US time

### Typing

*left for next meeting due to time constraints*

Options:
* [mypy pre-commit, mypy action (#1357)](https://github.com/dials/dials/pull/1357)
* [pytype action (#1364)](https://github.com/dials/dials/issues/1364)

What do we want to do?

### DIALS 1.14 tutorials

Can we remove the DIALS 1.14 tutorials from the website?
CCP4 is now distributing 2.2 and 1.14 is no longer supported.
* Action **MG**: remove

### DIALS documentation build

*left for next meeting due to time constraints*

Should the DIALS documentation build outside a cctbx environment?

* Currently you can only build the DIALS documentation inside a cctbx environment. This means a remote documentation build needs to build cctbx first, so we can't easily run the documentation build on every commit. When things break they are difficult to fix. We can't use readthedocs.
* MG: I believe the main obstacles are three cctbx Sphinx plugins that may need to be extracted or removed from DIALS documentation, and some phil parsing logic.

### Any other business
* renaming `master` branch :arrow_right: `main`
* Nothing much will happen on this any time soon - either way we'll [wait for GitHub support on this one](https://github.com/github/renaming)
* General assent that this is useful

### Next meeting

GW suggests first week of September rather than today + 2 weeks