# Clim-recal status ## To-do: - [ ] Ruth to look at debiased dataset, see if it makes sense. - [x] Camila to implement 360 day calendar on HADs at resampling stage. - [x] Camila to modify debiasing script to loop over large periods of time. - [x] Camila to run resampling in all variables. - [x] Ruth to understand if we need to group by time.dayofyear and report back - [x] Aoife/Camila to debug why code in quantile delta mapping is braking for groups. - [x] Make sure environment works both in linux VM and Macs - [x] Our VM is not good enough to run it due to memory needed. - [x] Once we have have a fully debug CMethods.py, fork library, copy the modified script and create it as submodule (Camila). - [x] Run debiasing for tasmax for a fixed run (Camila). - [x] All avalaible methods as implemented (linear_scaling’, variance_scaling’, delta_method,quantile_mapping’ and quantile_delta_mapping’) - [ ] After all methods have been ran, fix bug and try to run quantile_mapping’ and quantile_delta_mapping with month grouping - [ ] Documentation (Aoife/Camila) - [ ] Handover to UA team for both clim-recal and scp ## 2nd March ## Debiasing methods with c-methods, learnings - For linear_scaling’, variance_scaling’, delta_method grouping default is month. This means that for each month it builds a time series of the historical data for that month and estimates the the 'correction'. This can also be done for day of the year. - For quantile_mapping’ and quantile_delta_mapping’ (based on [this](https://link.springer.com/article/10.1007/s00382-020-05447-4)) this is supposed to happend but there is a bug that doesnt allow for groupings. Currently uses full time series, is faster to run that grouping but might not be comparable. - I think fixing the bug should be simple. - Proposal: - Run debiasing for tasmax for all avalaible methods as implemented (linear_scaling’, variance_scaling’, delta_method,quantile_mapping’ and quantile_delta_mapping’) - Currently running quantile_delta_mapping running, first 10 years found in fileshare under `/vmfileshare/ClimateData/Debiased/tasmax/`. - After all methods have been ran, fix bug and try to run delta_method,quantile_mapping’ and quantile_delta_mapping with month grouping. ## 24th Feb 2022 ### Data - UKCP - Reprojection - Scripts to handle merged [into repo](https://github.com/alan-turing-institute/clim-recal/commit/6058dae61e65d12febd6b865bae7e1f4162fd3c0) - Running on VM in `screen` session under `ahughes` user - 3418 / 10701 as of 2023/02/24 - 12:56 - HADs 1. Decided to resample with 'linear' and latest grid. 2. Issue with 365 days per year, need to convert to 360, looking at code from the met office. This should be done at the resample stage. 3. Once 2 is sorted, will run resampling in all variables (only tasmax for now, with 365 day still). 4. Dropping 5 days ### Methods - [cmethods](https://github.com/btschwertfeger/python-cmethods) copied into the repo and adapted to run on our data. Library has a GPL-3.0 licence, which is has a strong copyleft clause, we need to consider for clim-recal. Methods that work in this library. - linear_scaling' - variance_scaling' - delta_method - quantile_mapping' - quantile_delta_mapping' - Fork the repo Still have questions about buggy 'grouping' step. Need to check it runs in the VM. - [pycat](https://github.com/wegener-center/pyCAT). - Not going well [documented here](https://github.com/alan-turing-institute/clim-recal/issues/6#issuecomment-1441568301). - [name=Aoife] (one of) my suggestion would be prioritise scoping other candidate libraries rather than delving into potentially complex issues with this - But maybe would take way too long, given just a few days - Alternatively, if we can get in touch with someone who has used it before and can record their setup, then this would be useful and potentially get things started - Another potential issue, is how these libraries are loading data, it makes it somewhat awkward. Needs thought about and potentially harder to use multiple libraries. ### Documentation - Resampling - Documented in python readme - Reprojection - Aoife to update. - Data loading scripts - Documented in python readme - Datasets - Not documented yet. - methods - cmethods - Not documented yet. - Main README - Basic documentation in how to run an environment. Nothing about the project or what is available.