Design document

# Design document ## Background A microscope is a robot with at least one imaging system. Typically the imaging is focused on small things and the "robotic" part is typically trivial. Many microscopes involve a camera, shutters, and a 2 or 3-axis stage system used to position the sample. They require little to no software automation. However, many important forms of microscopy involve imaging without a camera or with more than just a camera. These require coordinating a systems of devices, often with tight timing tolerances, in order to form images. Multiple imaging systems may be used, sometimes in coordination with targeted interogation of a sample with any of a number of devices including micromanipulators, target light delivered via a stearable laser beam or as a hologram. Multiple imaging systems (cameras or things like them) may provide different views of a sample that need to be co-registered. Computational forms of microscopy require some non-trivial manipulation of recorded samples to generate contrast. Control software that enables all these forms of microscopy will ... ## Goals * Microscope makers can develop ### non-goals * The GUI is not part of the value. We probably will need GUI's to develop and test or to provide a baseline of interactivity. But, ideally this product enables multiple different front-ends and enables folks to develop user-interfaces focused on their particular task. ## High-level components ### Device Manager ### Runtime ### User interface ## # SCRATCH TEXT - OLD OLD OLD ## TODO Top matter, goals, requirements, problems, strategy, approach ## Background A microscope is composed of a collection of sensors and actuators which are coordinated by a peice of software. One or more of these devices is responsible for producing images. Images might come from different sources imaging different modalities. The microscope is a computer-vision enabled robot - in some respects similar to a self-driving car or the curiosity rover. There's a variety of commercial and open-source microscope software available. We can distinguish two categories: those targeting a fixed set of hardware or type of microscopy and those supporting custom-built microscopes. Here, we focus on the latter. We aim to support microscope builders as they develop new devices. However the initial focus will be to support a single type of microscope for a custom application - a modified [diSPIM] designed for fast, high resolution imaging. This design is guided by this initial use case, Focusing on that application highlights some of the These conventionally involve some graphical user interface (GUI) that is used to manipulate a set of registered devices and display data. The GUI interacts with a central controller, typically in the same process. Exposing the controller capabilities via an API allows ## Questions requiring validation - [ ] Q: Integration with ROS ecosystem - [ ] Q: Integration with µmanager ecosystem - [ ] Q: Target operating system - [ ] Q: Target language(s) ### Q: Operating system - Hardware support risk - windows is by far lowest risk - is the risk important? ## Stories ### Video subsystem The video system comprises a controller device (computer) with attached cameras, attached data storage and suitable integration with analysis to enable real-time feedback. Goals - It should be possible to simultaneously stream from more than one camera. - Cameras should not need to be synchronized. - When recording, the system should ensure that any acquired image is written to disk. - The system's sustained bandwidth for recording and persisting data shoule exceed 2 GB/s. Enabling feedback - - Based on detected events, precisely control how video is stored to disk. - Images may be streamed to analysis while persisted storage is disabled. - Images need to be timestamped as close to acquisition as possible. - Images should be retained in memory for a controllable amount of time (buffer 1 sec of video). - Within the buffer window, an analysis process should be able to look at the available data, apply a process to detect events of interest, and then inform the storage process. - Events can be used to split the video stream so videos always start the frame of the event. - Events can be used to initiate recording. - For example, persist a 1 second window around when an input TTL f ### Device management - Consistent start - Validated configuration Questions - How to aggregate devices into subsystems? This should be dynamic and flexible so devices can be reused in different contexts during an acquisition. - Dynamically acquire resources for a subsystem as it's needed. - Examples - Using a stage as a scanning device or as a static positioner during different phases of the acquistion. - Reuse an externally switchable input/output line for different subsystems ### User - #### ## Requirements ## Systems ## Device Model ### State ### Configuration ### Capability ### Transports ## References and related work * [ROS - Robot operating system](ros.org) * [ROS2 Design](design.ros2.org)