# 3D model selection task

### Yuriy's email
> Hi Colin,
Here is my AP follow up from our previous Relion meeting:
I have tested idea of using FSC at 0.5 for selecting best model from InitialModel classes. It seem to work. Relion resolution criteria worked fine to exclude really bad model, which was initially selected (see below), but could not distinguish between two better models, classes 1 and 2 – it returned the same resolution. However, FSC at 0.5 criteria was better for model 2.
>
> The steps are below:
> 1. Select every class into individual group. I had to do it from GUI for every class, since I could not find how to do this with a single step.
> 2. Split every subset into 2 randomised subsets
> ```bash
> `which relion_star_handler` \
> --i Select/job026/particles.star \
> --o Select/job030/particles.star \
> --split --random_order \
> --nr_split 2 \
> --pipeline_control Select/job030/
> ```
>
> 3. Reconstruct 3d for every half, every model subset:
> ```bash
> relion_reconstruct \
> --i Select/job038/particles_split1.star \
> --o Select/job038/3d_half1_model4_unfil.mrc \
> --ctf true --mask_diameter 350
>
> relion_reconstruct \
> --i Select/job038/particles_split2.star \
> --o Select/job038/3d_half2_model4_unfil.mrc \
> --ctf true --mask_diameter 350
> ```
> 4. Create spherical mask. ~~I could not find how to do soft edge, so had to do this in two steps; also for some reason it would not work if pixel size was specified. Other important thing, outer radius for first, hard mask was 0.98 of the mask used in reconstruction:~~
> ```bash
> relion_mask_create \
> --denovo true --box_size 96 \
> --outer_radius 32
>
> relion_mask_create \
> --i mask.mrc --o mask_soft.mrc \
> --width_soft_edge 5
> ```
Updated by Colin (18/3/21): using a recent version of Relion 3.1, the following command works in one step. (Unfortunately we still can't use the standard Relion MaskCreate job type for this because it doesn't expose the `--denovo` option.)

relion_mask_create --denovo true --box_size 96 \
--outer_radius 32 --angpix 1.06 --width_soft_edge 5

For calculation of the required box size and angpix values in `cryolo_relion_it.py`:
* If `opts.extract_downscale == True`:
* Particle and mask box size = `opts.extract_small_boxsize`
* Particle and mask angpix = `opts.angpix * opts.motioncor_binning * opts.extract_boxsize / opts.extract_small_boxsize`
* If `opts.extract_downscale == False`:
* Particle and mask box size = `bin_corrected_box_even`
* Particle and mask angpix = `opts.angpix * opts.motioncor_binning`

> 5. Run relion postprocess job to get FSC:
> ```bash
> `which relion_postprocess` \
> --mask Select/job030/mask_soft.mrc \
> --i Select/job038/3d_half1_model4_unfil.mrc \
> --o PostProcess/job039/postprocess \
> --angpix 5.36 --auto_bfac \
> --autob_lowres 10 \
> --pipeline_control PostProcess/job039/
> ```
> 6. The last step was manual, since Relion reported same resolution for models 1 and 2, but at 0.5 threshold model 2 performed better. In a perfect word, we would need to do sort of linear interpolation between frequencies, since reported pixel values are rarely at FSC 0.5 exactly (see below):

### Relevant concepts:

* RELION External jobs -- see section 14.4 of the [RELION 3.1 tutorial](ftp://ftp.mrc-lmb.cam.ac.uk/pub/scheres/relion31_tutorial.pdf)
* [An example](https://github.com/DiamondLightSource/python-relion-yolo-it/blob/master/relion_yolo_it/cryolo_external_job.py) of an External job for running the crYOLO program for particle picking
* The [current logic](https://github.com/DiamondLightSource/python-relion-yolo-it/blob/d_rel3.1/relion_yolo_it/cryolo_relion_it.py#L1771-L1798) for selecting the best class
* Learn about new Relion Schedules (tutorial section 14.5) and the upcoming, yet-to-be-named CCP-EM relion-pipeliner successor that does something more python-y. This is independent of the work below.

### Suggested approach:

1. An External job for selecting particle groups from a set of classes
2. An External job for splitting a particle group into two halves and making 3D reconstructions of each
3. For Yuriy's step 4, mask creation, use a new External job is better to run ~~both of Yuriy's suggested commands at once~~ the single command that replaces Yuriy's ones (see above)
4. Use the normal PostProcess job type for Yuriy's step 5
5. An External job to analyse the FSC curves from PostProcess, report the resolution values and decide on the best class. (PostProcess writes out FSC curves in both XML and STAR format. Using STAR is probably better because it's the standard output format from all RELION jobs. XML is an extra output from PostProcess only.)
6. RelionWrapper to interpret the output of the external jobs and to pass them into Zocalo for 'further processing'

### Questions now answered by Yuriy:

* Should we also be checking the position of the centre of mass of each class, to avoid intense artifacts near the edges?
* Might be helpful but the FSC (with a slightly tighter mask) should remove most of these artifacts
* Any strong opinions on how best to interpolate the FSC curves? Should we pick the first or last crossing of the 0.5 threshold?
* Lowest resolution at which FSC drops below 0.5
* Should we also run this class selection algorithm after 3D classification? (Used in relion_it.py to select a 3D reference for auto-picking in the second pass.)
* Yuriy hadn't thought about this, might be good to try but is likely to be sensitive to class sizes and risks overfitting because it's non-gold-standard FSC