Reporter and Output Handlers Design Document

The following document is an explanation of how output rendering in conda works. The document contains a detailed explaination of the software design and certain justifications behind these design decisions. The intended audience for this document is someone who wants a thorough understanding of how this system works and is not intended to be a tutorial or how-to article. Please see those for a quicker overview of output rendering and how to begin using it.

High level overview

Output rendering in conda is a comprised of two different abstractions: reporter and output handlers. Reporter handlers define how the output will look and the output handlers define where the output is rendered to. The interaction between these abstractions is defined as an exchange of a string object from the reporter handler to the output handler. When conda runs, a pair of at least one reporter handler and one output handler must exists for the program to emit any output for the user of the application. The flexibility of this system also allows us to define an arbitrary number of reporter and output handlers simultaneously.

Furthermore, the reporter and output handlers are written as plugin hooks, which means that additional reporter and output handlers can be added to augment and override conda's default behavior.

To explore this concept further, let's look at an example. The default reporter handler in conda is the console reporter handler. This is the standard output most users are used to seeing when they run conda commands. Conda also ships with a default output handler called stdout. As the name implies, this handler is responsible for taking strings returned by the console reporter handler and sending them to stdout. So, the pairing of these two forms the basis of output rendering in conda.

This can be configured via the reporters configuration setting. The default value for this setting look like this:

reporters:
  - backend: console
    output: stdout

Why this design?

The primary reason we chose this design is to allow for simultaneous reporter/output handler pairings. Using this design we are able to funnel the entirety of our output rendering through this system which is then able to send it wherever it is configured to go. One compelling example of this usage is being to run conda with normal output to the terminal while being able to save the JSON output in various log files. This could be helpful for site administrators wishing for better insight on how users are runing conda or could even be useful in CI runner scenarios.

Another justification for this design is that the definition of reporter handlers as plugin hooks will allow plugin authors to modify the look and feel of conda itself. The maintainers of conda originally pursued this project as a way to update the look and feel of conda without disrupting those who have become accustomed to how it currently looks, especially those who may be parsing "console" output in scripts. By making these reporter handlers customizable, there exists many possibilities for further customizing conda via the usage and installation of plugin hooks.

Lower level overview: API Design

Below is a lower level discussion of exactly how the reporter and out handler APIs work with examples for illustration.

Reporter handlers

The ReporterHandler is a user defined type in Python. The class itself is an abstract base class which defines a number of different methods its child classes must implement. It's one concrete method is render which will attempt to convert an object to a string so that it can be passed to an output handler. This method can be overriden to provide a different default behavior which may be desired for example when rendering JSON.

Each abstract method other than render is responsible for render some component that we normally see in the output of conda such as the detail view seen in conda info or the activation message shown at the end of the conda create command. Although most methods return strings, some do not. These are methods which instead return dynamic components such as progress bars or yes/no dialogs.

Example

Below is an example of how a concrete class for a reporter handler may be defined:

class ConsoleReporterHandler(ReporterHandlerBase):
    """
    Example implementation for console reporting in conda
    """

    def detail_view(self, data: dict[str, str | int | bool], **kwargs) -> str:
        table_parts = [""]
        longest_header = max(map(len, data.keys()))

        for header, value in data.items():
            table_parts.append(f" {header:>{longest_header}} : {value}")

        table_parts.append("\n")

        return "\n".join(table_parts)

    def envs_list(self, prefixes, **kwargs) -> str:
        context = kwargs.get("context")

        output = ["", "# conda environments:", "#"]

        def disp_env(prefix):
            active = "*" if prefix == context.active_prefix else " "
            if prefix == context.root_prefix:
                name = ROOT_ENV_NAME
            elif any(
                paths_equal(envs_dir, dirname(prefix)) for envs_dir in context.envs_dirs
            ):
                name = basename(prefix)
            else:
                name = ""
            return f"{name:20} {active} {prefix}"

        for env_prefix in prefixes:
            output.append(disp_env(env_prefix))

        output.append("\n")

        return "\n".join(output)


    def progress_bar(self, description, io_context_manager, **kwargs) -> ProgressBarBase:
        """
        Determines whether to return a TQDMProgressBar or QuietProgressBar
        """
        if context.quiet:
            return QuietProgressBar(description, io_context_manager, **kwargs)
        else:
            return TQDMProgressBar(description, io_context_manager, **kwargs)

In the example above, we have a defined a ConsoleReporterHandler that implements three separate abstract methods. Two of them (envs_list and detail_view) return strings and can be considered static whereas one of the methods (progress_bar) returns a ProgressBar object. What is shown is a small snippet of what this concrete class will ultimately look like because it will hold all the logic for how displays are constructed.

Using this design, we define every place where the look and feel of the output can be customized for plugin authors. We use the familiar abstraction of a display component to help plugin authors better rationalize the system as well. Using inheritance, plugin authors will also be able to selectively customize only certain parts of the output rendering. This opens up the possibility for easily making vendor specific modifications to conda's output (e.g. an Anaconda upsell message at the end of the activation display component).

Output handlers

Output handlers are responsible for rendering output to a particular output stream (e.g. stdout, network, file, etc.). The output handler must be defined as a context manager that returns typing.TextIO object. We use this object because it is a common way to handle working with text streams in Python and can be easily passed in to other libraries such as "tqdm" and "rich".

The simplest implementation for this is the stdout output handler. This will essentially be a wrapper around sys.stdout function which implements the TextIO interface. Anything implementing the same interface as TextIO can be used, including the open function as long as it is using text and not bytes.

Example

Below is an example of how the StdoutOutputhandler is defined:

import sys
from contextlib import contextmanager
from typing import TextIO

@contextmanager
def stdout_output_handler() -> TextIO:
    yield sys.stdout

As you can see, this implementation is trivially simple, but lays the framework for more complicated approaches. Let's look at another one for writing output to files instead:

@contextmanager
def file_output_handler() -> TextIO:
    try:
        with open("file.txt", "w") as fp:
            yield fp
    except OSError as exc:
        logger.error(f"Unable to create file: {exc}")

Here, we can define the appropriate level of error handling to use when we are unable to write to a file. This can be expanded more and more with different output streams to write out to the network.

Reporter manager: putting it all together

So far, we have only examined how to define both reporter and output handlers, but how are these claseses actually used in the code itself? and how are both of these abstractions connected to each other?

The ReporterManager is what does this work for us and provides a higher level API for use inside the code. In order to provide this high level API, the ReporterManager is aware of the configuration context in which it operates. This means that it not only knows which reporter/output handlers are available (i.e. registered via plugin hooks) but also which are currently configured for use. Because of this, it can provide an API where the rendering is called at a single point but may be rendered via multiple reporter/output handler pairings.

Let's quickly show an example of how this is used. Afterwards, we will break down the example and discuss what's going on behind the scenes. We use part of the implementation from the conda info command as the example:

from ..base.context import context
from ..common.io import get_reporter_manager


def execute(args: Namespace, parser: ArgumentParser) -> int:
    """
    Simplified version of the ``conda info`` command
    """
    reporter_manager = get_reporter_manager()
    data, component = get_display_data(args, context)

    reporter_manager.render(data, component=component, context=context)

    return 0

The ReporterManager object in conda is a singleton and is retrieved with the get_reporter_manager function. Using a function to do this allows use to utilize the functools.lru_cache decorator to ensure only one object is created per program run.

Next, a function called get_display_data is called to return the data we want to display and the component we want to use to display it. The data and component variables are then passed to the reporter_manager.render method. The render method can be thought of as the main dispatch method and will iterate through all configured reporter/output handler pairings to render the output. As render iterates over these pairs it will first attempt to call a method on the reporter handler whose name matches the value in component. If the value of component is None it will simply revert to using the default render method mentioned previously in this document. The render method will then pass the data it receives from the data variable to the reporter handler. The reporter handler returns a string and this is then passed to the output handler callable which finishes the call.

It should be noted that the above is only appicable for reporter handler methods that return strings. Interactive components, such as ProgressBar, are handled differently and will be explained in a later section.

The ReporterManager class will have several methods for handling the rendering of both static and interactive components. For static components, the rendering will occur via a render function that accepts the name of the display component it will be rendering and the data that should be passed to it. The interactive components will be defined individually. This means the ReporterManager class will have progress_bar, prompt and spinner methods that will return specific objects the conda code can use.