SCope: Work Plan

# SCope: Work Plan **Website**: [scope.aertslab.org](http://scope.aertslab.org) **Source repository**: [GitHub](https://github.com/aertslab/SCope) ## Summary This document summarises "work items" which are high-level, conceptual tasks that add value to SCope. Each "work item" is described in high-level terms and justified by the value it adds to the SCope project. An estimate of the work required to complete each work item is provided as a range measured by [_full-time developer-hours_](https://en.wikipedia.org/wiki/Man-hour) worked. Estimate to complete all work items: 575 - 1820 full time developer-hours (Confidence: Moderate; Uncertainty: Moderate). In order for this project to survive into the future it requires continuous maintenance, more than the sporadic maintenance effort that is currently occasionally applied. Continually adding new features without concern for the resulting accumulation of [technical](https://en.wikipedia.org/wiki/Technical_debt) [debt](https://hackernoon.com/the-fallacy-of-technical-debt-202f7406337e) puts SCope at [very great risk](https://sites.google.com/site/unclebobconsultingllc/a-mess-is-not-a-technical-debt). This cannot be placed into a "work item", it must be dedicated and continuous. ## Work Item 1: Compare scRNA and scATAC data SCope already comes equipped with a "compare" feature, but it needs extra work such that new features, such as comparing scRNA and scATAC data, can be added. This work item will extend and improve this existing compare feature. **Value**: SCope is at the forefront of single-cell data visualisation. Adding a feature like this will help to keep it there and prevent it from being left behind. [Stein Aerts](https://aertslab.org/)[^1] believes this to be the most impactful feature to be added in terms of manuscripts published. **Time Estimate**: 120 - 300 hours * **Confidence**: Low * **Uncertainty**: High * **Factors**: not clear what needs to be done, potentially very large volume of work ## Work Item 2: Allow users to visualise more than 1 million data points Currently, SCope visualisations struggle to display more than approximately 200 thousand data points. Unfortunately this current mechanism is tightly integrated into other parts of SCope making the use of faster, [GPU](https://en.wikipedia.org/wiki/Graphics_processing_unit) based rendering difficult. This work item will involve creating a clear interface between the data visualisation and other parts of SCope then creating a GPU rendered data visualisation component that has all the features of the older, slower, visualisation system. **Value**: Data sets are constantly growing in size. There are already many data sets that contain more than 1 million points that SCope simply cannot handle. This work item will keep SCope useful into the future as datasets grow even larger. **Time Estimate**: 40 - 240 hours * **Confidence**: High * **Uncertainty**: Low * **Factors**: Prototype exists, potentially large volume of work ## Work Item 3: Make collaboration between SCope users more intuitive Collaboration between SCope users is currently difficult, error-prone, and often requires manual administrator intervention. The current mechanism involves sharing long link URLs that collaborators can visit and interact with. This has a number of issues, at least including lack of permissions enforcement (all users with the link have equal permissions), concurrent access to SCope (more than 1 user at a time) severely slows down the system, and automated use of the system (e.g. integration with other tools) is extremely difficult. This work item will solve all the above described issues by [leveraging](https://fastapi.tiangolo.com/) appropriate [modern](https://developers.google.com/protocol-buffers/) [technologies](https://redux.js.org/), [techniques](https://en.wikipedia.org/wiki/Relational_database), and [standardised mechanisms](https://restfulapi.net/) for interacting with web-based applications such as SCope. All of this while keeping the ability to collaborate by simply sharing a link URL. **Value**: As SCope becomes more and more popular as a useful tool withing the single cell community the issues with sharing data and collaboration will become more stark and require ever-increasing time to manually solve administrative problems. This work package will reduce the effort required to administer the SCope application, allow users to collaborate easily and safely, and also make adding new features to SCope in the future easier and less error-prone. **Time Estimate**: 240 - 600 hours * **Confidence**: Moderate * **Uncertainty**: Moderate * **Factors**: Potentially very large volume of work ## Work Item 4: Handle community data formats The native SCope data format is an extended (non-standard) [Loom](http://loompy.org/). However, the single cell community has not converged on Loom (or any other format) for data interchange. Therefore, it has become necessary to be able to work with other formats. **Value**: Until the single cell community converges on a data interchange format the utility of SCope is limited to the formats it can accept. **Time Estimate**: 80 - 320 hours * **Confidence**: Low * **Uncertainty**: High * **Factors**: Unclear best approach or which formats or how data should be stored, SCope is closely tied to Loom ## Work Item 5: Link SCope with community gene repositories This work item involves providing a public mechanism to integrate with other community tools such as [Flybase](https://flybase.org) and ensemble. **Value**: Integration with other tools strengthens SCope as a useful tool and aids all the single cell community by reducing wasted effort. **Time Estimate**: 15 - 120 hours * **Confidence**: Moderate * **Uncertainty**: Moderate * **Factors**: dependency on work item 3, unclear how other services should use SCope ## Work Item 6: Automated software testing, documentation, and deployment SCope currently suffers from a critical lack of testing and documentation. There are no automated tests for any component and almost no programmers documentation. This makes SCope difficult and risky to change, especially for developers new to the project. **Value**: _Automated testing_ [provides](https://doi.org/10.1109%2F32.6191) [several](https://fortegrp.com/the-importance-of-unit-testing/) [advantages](https://www.codemag.com/Article/1901071/10-Reasons-Why-Unit-Testing-Matters) [for](https://msdn.microsoft.com/en-us/library/ee330950%28v=vs.110%29.aspx) [developers](https://www.drdobbs.com/tools/change-code-without-fear/206105233): a way to experiment with changes, a clear way to evaluate design, a mechanism to fix bugs as early as possible, amongst many others. A robust, correct, and error free SCope is more likely to please users and result in a larger user (and citation) base. _Documentation_ provides maintainers a place to reflect on overall software architecture, an easy reference to parts of the system they have not worked on in a long time, and a mechanism for evaluating design changes. It provides new contributers and users with clear explanations for how to set up and work with SCope. _Automated deployment_ removes the potential for human error from the repetative series of steps required to deploy SCope. **Time Estimate**: 80 - 240 hours * **Confidence**: High * **Uncertainty**: Low * **Factors**: Groundwork is already being laid [^1]: Personal communications.