# DIALS + conda ## DC2: DIALS installer 1. **WP1:** We can use [`conda-pack`](https://github.com/conda/conda-pack) to pack up the environment. * This does not depend on anything else really, so we can start here. * Having this would mean that we have a valid (non-relocated) conda environment in the installation, which would resolve eg. the readline/ncurses issues. So this has an immediate benefit. * The installer should be able to run offline. * Workflow would be: * use `conda-pack` to pack up the correct environment * unpack that manually * do the equivalent of the create-installer script using the `conda-pack`'d environment * And then on installation: * unpack the `conda-pack`'d environment * run the `conda-pack` post install fixer-upper * do the remainder of the DIALS installer bit * done * Also: disconnect the DIALS installer scripts from libtbx 2. **WP2:** Support an offline developer installation. Currently this is covered by downloading a release. * Add a `--download` option to bootstrap that * downloads and unpacks miniconda * tells miniconda to resolve and download the environment (not: install) * clones all sources repositories * Add an `--offline` option to bootstrap that * takes all the above files and does the base & build step 3. **WP3**: have a run-time spec and a development environment * Split up eg. `linux.txt` into run/dev, and then there will be some overlap between platforms and it can probably be simplified * Not on the critical path. * This would allow us to remove C compilers, alabaster, sphinx, scons, pytest, ..., and make 4. **WP4**: The installer should only install user installation, not development installation * Not reasonable to provide an 'upgrade' path, as that would need to rebuild everything anyway. We can provide manual instructions for converting. * With this we can remove all C++ source code and intermediate files Ancillary: * Make bootstrap print out '*you're in uncharted territory*' if you change the Python version on a release build that has otherwise pinned conda-forge packages. Unpin the CF packages. ## DC1: per-module `setup.py` The intention is to create a `setup.py` file for DIALS in the `modules/dials/` directory that is generated, installed, and `develop`'d through `dials/libtbx_refresh.py` 1. Making `import` work with standard `conda/bin/python` - no `libtbx.python` dispatcher required. * this means we no longer need to generate a `pytest` dispatcher * this makes all normal python dispatchers able to import dials/dxtbx/xia2/..., which is relevant for all external packages that make use of DIALS and enables them to be absolute standard Python packages with zero required additional dark magic. For example such packages are `DUI`, `fast_dp`, `iota`, `screen19`, ... * with this activating the conda environment would be sufficient to set everything required in the environment, thus there is no more need to generate special libtbx dispatchers 2. Metadata -- generating the dials and dxtbx format entrypoints as part of this new `setup.py` * this makes for example the dials version number accessible to other packages in the regular packaging way, which allows external packages to declare version dependencies (eg. `iota` V1.7 requiring `DIALS>=2.2<4.0`) 4. Generate standard python dispatchers from `command_line/*.py` * This requires restructuring code inside the `command_line/` files in order to provide a named entry point for the dispatcher to call into. * This can be done on a per-package basis, and is trivial for DIALS, xia2, and dxtbx. 1. Wherever we import `*ext*` modules, do it in a ```python try: from . import dials_whatev_ext except ModuleNotFoundError: import dials_whatev_ext ``` way. That allows us to build and/or distribute in-source builds at some point. This should be very straightforward as we already agreed that we only import the C++ libraries in a single place, and indirectly import from this place everywhere. 5. If we supported version numbered dependencies for DIALS then at that point we could transform xia2 into a regular python package with normal releases on pypi and conda-forge. Considered alternatives: * put `setup.py` in the `build/` directory. This is an option if the source repository can not be used for any particular reason (ie. incomplete environment available at the time). We would lose version control and entry points would not be in a module `setup.py` where people may expect them.