---
name: DIALS core meeting 2023-10-19
tags: core meeting
---

# DIALS core meeting 2023-10-19

## Previous Actions

- [ ] **ND**: PR to remove `std::string` support for messagepack from dials code
- [ ] **ND** Investigate getting `psana` tests running on the DIALS xfel-regression testing
- [X] **JBE:** Solve "multiplex: Duplicate Batch offsets detected" [xia2#430](https://github.com/xia2/xia2/issues/430) ([prev](https://dials.github.io/kb/core/2023-06-15#xia2multiplex-fails-because-duplicate-batch-offsets-detected)) (James has fixed, David to review but ticking off)
- [ ] **ND** or **DW**: Cannot view reciprocal lattice using dials.reciprocal_lattice_viewer [#2443](https://github.com/dials/dials/issues/2443)
- Probably needs someone to dig into windows gltbx behaviour, or at least work out where issue lies
- Reports from CCP4 workshop that people are having issue on latest ubuntu
- David reproduced the problem which is progress...
- **ND**: Investigate using stable-nightly conda builds of cctbx (e.g. cctbx tests) longer term
- [x] **ND**: Update CI to use nightly on dials/dxtbx serialtbx merges, let AB know when ready
- [x] ~~**ND**: Test cosym Wij PR on diamond systems and merge~~

## Agenda

### SerialTBX status update ([prev](https://dials.github.io/kb/core/2023-06-15#removing-circular-xfel-dependencies-with-serialtbx)).
- [cctbx#872](https://github.com/cctbx/cctbx_project/pull/872) now marked ready for review
- Awaiting NKS approval
- dxtbx will break when changes applied, until next cctbx release
- Planning to switch to cctbx nightly for dxtbx/dials tests
- All went through successfully, after a couple of days of nightly tweaks

### small cell serialtbx
https://github.com/dials/dials/pull/2477 - small cell code did not make it into serialtbx yet.

- AB: Did small cell xfel recently. Had sequences indexer success over stills.
- Ran sweeps indexer. Many bad indexing solutions, suspect differences down to index assignment tolerance at 0.3 hkl by default
- Can still move code into serial toolbox, but don't think best end solution is obvious yet

Dan experience using new small cell algorithm - does index a lot of "nonsense" when you give it an incorrect unit cell, so went back to small_cell_process command line tool. Now need to dig into why it claims to work when it didn't (yay false positive debug) - on the whole good step in right direction. Work continues.

### DIALS-support "Error importing .cbf files using DIALS"
- [dials#2488](https://github.com/dials/dials/issues/2488)
- Pilatus3 1M at BL02B1, SPRING-8 appears to be transposed!!??!?
- Graeme got some info from Kunio, Takanori seems to be looking at
- Seems to be no explanation for the very odd inverted-detector behaviour
- Done for Crysalis convention
- Takanori appears to have Format class in hand
- Okay to ignore the exceptionally different files as no timestamp and no longer generated
- Take a look at state of things in a couple of weeks
- No progress

### dials.cosym finds the right operator but puts it wrongly
- [dials#2486](https://github.com/dials/dials/issues/2486)
- Draft PR at https://github.com/dials/dials/pull/2497
- Pulled in other problems that need to be worked through
- Potential fix opened a large box of problems
- GW believes needs a rethink and a rewrite and patching is essentially impossible
- GW and DP to discuss on 27th
- Met! DP has made some progress but had other obligations
- Recent attempt... regression test pass but Takanori's case still fails
- We are back where we started, but with a better understanding of the problem and some belief that what we are doing is correct => the issue is perhaps somewhere else
- DP to continue on the quest

### Add properties table to scan
- https://github.com/cctbx/dxtbx/pull/620
- Outstanding review request from GW
- Approve. **ND to sort out merges.**
- Still unmerged 2023-10-19 -> remains outstanding

## AOB?

### Build fails on latest ventura
- https://github.com/dials/dials/issues/2522
- GW to update from 13.3 to 13.6 to try this out

### Reports of build failures on RHEL8
- JBE noticed some CI tests started to fail on linux / bootstrap but not regular linux
- Should we think about moving the default to a cmake build (DP notes this would diverge further from the XFEL regression tests)
- JBE to test clean bootstrap in RHEL8 (ws448)

### Starting January DLS Loan of RSE
- We have someone who can do some work for 6 months on the actual codebase, refactoring, tidying etc. with a view to future optimisation

### Add support for Pilatus 4
- GW has data, has promised to ensure works - apropos to this the current issue is
- Pls can has review on https://github.com/cctbx/dxtbx/pull/663
- Apropos to nothing - debugging this took an *extra day* because I did not _re import_ the data ergo the imported.expt was inconsistent 🤦‍♂️

--------------------------------------------------------------------------------
Finding strong spots in imageset 0
--------------------------------------------------------------------------------

Finding spots in image 1 to 240...
Setting chunksize=20
Extracting strong pixels from images
 Using multiprocessing with 1 parallel job(s)

Traceback (most recent call last):
  File "/Users/graeme/git/dials/build/../modules/dials/src/dials/command_line/find_spots.py", line 242, in <module>
    run()
  File "/Users/graeme/git/dials/conda_base/lib/python3.10/contextlib.py", line 79, in inner
    return func(*args, **kwds)
  File "/Users/graeme/git/dials/build/../modules/dials/src/dials/command_line/find_spots.py", line 236, in run
    results = do_spotfinding(experiments, params)
  File "/Users/graeme/git/dials/build/../modules/dials/src/dials/command_line/find_spots.py", line 127, in do_spotfinding
    reflections = flex.reflection_table.from_observations(experiments, params)
  File "/Users/graeme/git/dials/modules/dials/src/dials/array_family/flex_ext.py", line 183, in from_observations
    return spotfinder.find_spots(experiments)
  File "/Users/graeme/git/dials/modules/dials/src/dials/algorithms/spot_finding/finder.py", line 704, in find_spots
    table, hot_mask = self._find_spots_in_imageset(imageset)
  File "/Users/graeme/git/dials/modules/dials/src/dials/algorithms/spot_finding/finder.py", line 820, in _find_spots_in_imageset
    r, h = extract_spots(imageset[j0:j1])
  File "/Users/graeme/git/dials/modules/dials/src/dials/algorithms/spot_finding/finder.py", line 417, in __call__
    return self._find_spots(imageset)
  File "/Users/graeme/git/dials/modules/dials/src/dials/algorithms/spot_finding/finder.py", line 519, in _find_spots
    result = function(task)
  File "/Users/graeme/git/dials/modules/dials/src/dials/algorithms/spot_finding/finder.py", line 79, in __call__
    mask = self.imageset.get_mask(index)
RuntimeError: Please report this error to [e-mail]: dxtbx Internal Error: /Users/graeme/git/dials/modules/dxtbx/src/dxtbx/model/panel.h(397): DXTBX_ASSERT(data.accessor()[1] == image_size_[0]) failure.

### Conda-forge renaming of boost packages

- https://github.com/conda-forge/cctbx-base-feedstock/pull/74/files
- Will need applying to downstream dials, dxtbx
- conda envs

### Removing setting batch offset when creating imagesets

- https://github.com/cctbx/dxtbx/pull/662
- We no longer need the batch offset => we can just remove it after changes by DGW
- Much testing and reviews please
- DP also to test via XFEL-regression

### Next meeting

Due to time zone changes the normal meeting time must change:

Thursday, November 16nd, 4pm (GMT), 8am (PDT)

DP, ASB unable to make November 2nd date - ND to cancel meeting request for that date