# Making Boson Community Friendly

The Boson project has been built from the beginning with an eye towards community. Throughout the architecture, where there exist vendor-specific capabilities, we have endeavored to make these configurable or otherwise pluggable. This document will describe where these "pluggable" points exist, and highlight any potential concerns about community adoption.

## TL;DR

To provide a vendor-neutral version of `func`, we will need to have new stack images, builders and buildpacks. In addition, we will need to come to agreement on the use of existing Boson invocation frameworks for Node.js and Python, or create new ones. These frameworks are not vendor-specific, but should be called out as a fundamental part of the `func` project that must exist in some form.

## Templates, Signatures, and Frameworks

### Templates

Function projects in Boson often look very much like other projects written for that language. For example, a Node.js Function project directory contains an `index.js` and a `package.json` file. The `func` binary contains [function templates](https://github.com/boson-project/func/tree/main/templates) embedded in the binary for several runtimes: Node.js, TypeScript, Go, Python, Quarkus and SpringBoot. When a new function project is created, these template files are written to disk in a project directory.

To use function templates other than those that are embedded into the binary, function developers can use templates on the local file system. These customized templates can be placed in `$HOME/.config/func/templates` or a path may be set in the `FUNC_TEMPLATES` environment variable, or specified with `--templates` in the `kn func create` command.

This, however, is not a fantastic user experience. Having the templates compiled into the binary allows users to get started much easier than if they need to have a directory of template files somewhere on the local system before a project can be created. It would be best if the upstream community can agree on a base set of templates to include built in to the binary.

### Signatures

In some languages, the function signature and available APIs are very neutral. For example, the Go templates expose only APIs built in to the language, or community APIs such as CloudEvents.

A Go function signature, may look like the following.

// Handle a CloudEvent.
func Handle(ctx context.Context, event event.Event) error {
    // function user code
}

A Quarkus function signature, on the other hand, has the runtime specific `@Funq` annotation.

@Funq
public CloudEvent<Output> function(CloudEvent<Input> input) {
    // function user code
}

These function signatures are fairly neutral and while they may be tailored for a specific runtime, they are not vendor specific. We anticipate that, generally, most of the Boson templates with their corresponding function signatures should be accepted by the community.

### Frameworks

Some languages, such as Java already have frameworks that enable function-like development. This is why we don't have a generic "Java" template. Instead, we simply make use of what Quarkus and SpringBoot both offer. These frameworks simplify things and generally allow us to provide non-controversial templates and signatures as described above.

[Node.js](https://github.com/boson-project/faas-js-runtime) and [Python](https://github.com/boson-project/parlaiment) functions, on the other hand, both utilize Boson runtime frameworks that are responsible for loading and invoking user functions. Similar to other runtimes such as Quarkus or SpringBoot, they each expose APIs that are specific to the frameworks. For example, in Python there is an annotation used to indicate that a function's expected input/output is a CloudEvent.

@event
def main(context: Context):
    """
    The context parameter contains the Flask request object and any
    CloudEvent received with the request.
    """
    # print(f"Method: {context.request.method}")

    # The return value here will be applied as the data attribute
    # of a CloudEvent returned to the function invoker
    return { "message": "Howdy!" }

Both of these frameworks are quite specific to `func` and are therefore not widely used outside of this project. While Red Hat is happy to donate these frameworks upstream, the Knative community may choose to take a different approach. If so, we should specify the capabilities that these frameworks provide.

The responsibilities these frameworks bear include the following non-exhaustive list.

* Load a user function from a local source file
* Receive incoming HTTP requests on port 8080
* Expose a function's invocation URI path as /
* Expose readiness and liveness endpoints at /health/readiness and /health/liveness
* Determine if the request includes a CloudEvent, and if so extract it from the request headers and body
* Invoke the user function with the CloudEvent and HTTP context as function parameters
* Interpret the return value from the function and respond to the caller with the apppropriate headers, data and response codes

Some of these responsibilities, such as "receive HTTP requests", are well defined. Others are a little looser. For example, can a function developer specify the response code or set a header via the return value, or should there be an API to achieve this? What are the properties of the HTTP context object that the function may be invoked with? These are decisions that need to be made by the invocation frameworks, and for which the Boson project has put stakes in the ground with Node.js and Python.

If the community chooses to take a path other than what is currently there, specification or guidance about these concerns will be required. Even if the community chooses to accept these frameworks, a set of requirements should be codified so that as much as possible, we ensure consistency across runtimes. Additional languages and runtimes will be added in the future. It would be good to be clear about the expectations.

## Stacks, Builders, and Buildpacks

Function code is converted to a runnable OCI image through Buildpacks. The "buildpacks" term is a bit overloaded, however, as there are really three separate artifacts that comprise the buildpack dependency: stacks, builders, and buildpacks. Boson provides all three.

### Stacks

Stacks are defined on the [buildpacks.io](https://buildpacks.io/docs/concepts/components/stack/) site.

> A stack is composed of two images that are intended to work together:
>
> - The build image of a stack provides the base image from which the build environment is constructed. The build environment is the containerized environment in which the lifecycle (and thereby buildpacks) are executed.
> - The run image of a stack provides the base image from which application images are built.

Boson stacks use [Red Hat UBI](https://catalog.redhat.com/software/containers/ubi8/ubi/5c359854d70cc534b3a3784e) images. These images are designed to be the base for containerized applications, are derived from Red Hat Enterprise Linux (RHEL), and are therefore vendor-specific.

Upstream will need to replace these with stack images that are more neutral, presumably something like Alpine. The buildpack org on GitHub provides an [example of an Alpine stack](https://github.com/buildpacks/samples/tree/main/stacks/alpine) that may be useful for understanding what would be needed.

### Builders
Builders are OCI images that contain everything necessary to execute a build. They have a 1:1 relationship with a stack image pair, and contain one or more buildpacks. The "build" image of the stack is used by the builder when it executes a buildpack, and the "run" image of the stack is used by the builder as the base for a function's runnable OCI image.

Builders contain one or more buildpacks and the "lifecycle", which is part of the buildpack spec. When a builder is executed, it runs a detection process for each of it's known buildpacks and then executes each buildpack that returns a success value from this "detect" process.

Because builders have a 1:1 relationship with stacks, and the builders that Boson currently uses depend on the Red Hat UBI stacks, a vendor neutral builder will be needed as well.

The builder image is the only image directly referenced by `func`. It is specified in the `func.yaml` file, and is used by `func` to build a function project. The `func` binary uses the Buildpack Go APIs to execute a builder's `detect` and `build` phases, but has no specific knowledge about a builder other than its name. As long as a builder contains buildpacks that can convert a function project into a runnable image, `func` can use it.

#### Side Note: Unified Builder Image
Boson currently has a separate builder image for each language/runtime. This is only the case due to the fact that Red Hat uses RPMs to install its supported runtimes. Per the buildpack specification, a buildpack cannot write system files. This means a buildpack cannot install RPMs. Therefore, Node.js, for example, must be installed statically on the stack images in advance of a function's build phase. This is why there are multiple builders specified in the `func` code.

In an upstream version of `func` there should only be a single builder image which references multiple buildpacks, each responsible for installing its needed runtime/language on the application image during the `build` phase.

### Buildpacks

Buildpacks are the images that acutally do the work of turning a function project into a runnable image. A buildpack inspects the function source code and formulates a plan to build a function application.

For Boson, this means that a buildpack is responsible for combining function code with a scaffolding/framework that does the work of accepting network connections and invoking the function. Currently Boson provides buildpacks for the following languages/runtimes.

* **Go** - invocation framework is [part of the buildpack](https://github.com/boson-project/buildpacks/tree/main/buildpacks/go/faas)
* **Quarkus** - [Quarkus Funqy](https://quarkus.io/guides/funqy) is the invocation framework
* **Node.js/TypeScript** - [faas-js-runtime](https://github.com/boson-project/faas-js-runtime) is the invocation framework
* **Python** - [parliament](https://github.com/boson-project/parliament) is the invocation framework
* **SpringBoot** - [Spring Cloud Function](https://spring.io/projects/spring-cloud-function) is the invocation framework

These buildpacks generally have no vendor specific dependencies and could move into a common/community organization for the most part unchanged. However, Boson buildpacks assume that the language/runtime already exists on the build image (see [above](https://hackmd.io/MjKONED2S2yE1IivTaXErg?both#Side-Note-Unified-Builder-Image)). Typically, the buildpack is responsible for installing it, but for Boson images, it's already there. In moving these buildpacks upstream, there would need to be some small changes to address this.

As noted in the [Frameworks](https://hackmd.io/MjKONED2S2yE1IivTaXErg?both#Frameworks) section above, both Node.js and Python buildpacks have a dependency on invocation frameworks that are vendor neutral, but otherwise used only by Red Hat. If these Boson invocation frameworks were to be replaced with alternate technology, changes to the Node.js and Python buildpacks would be needed.

#### Side Note: Buildpack and Framework Considerations
You may be thinking, "why bother publishing these frameworks? Just add them to the buildpack". It's a fair question. The main reason is that it's much easier to iterate on, version, and release the buildpacks and invocation frameworks independently from one another. As independently versioned frameworks, users can manage the dependency version in function projects after creation. This enables developers to chose when and how to pull in an updated/patched version of the framework. But this is not a hard requirement, and we are open to discussion around how to deal with this.

## A Strawman

Given the above, here is an initial proposal for the minimal work required to make Boson "community friendly".

* Adopt the existing function signatures for each language/runtime, and work with the community to define common expectations across all languages and runtimes
* Adopt the existing function invocation frameworks for Node.js and Python, and work with the community to codify an SPI that specifies expectations (ports, filenames, health endpoints, etc.) which an invocation framework must support to satisfy both Knative Serving/Eventing and function buildpacks
* Create a new, or reuse an existing stack that is not dependent on Red Hat UBI8 images
* Create a builder image that uses this new community stack and is aware of the buildpacks noted below
* Adopt the existing buildpacks with some modifications, including:
* A buildpack for a given language/runtime is responsible for installing the language/runtime that it uses, and should use community/upstream distributions. For example, the Node.js buildpack should install the upstream distribution of Node.js from nodejs.org.

The below diagram is a rough approximation of the architecture of the Boson project. Items shaded in blue are user artifacts. Items shaded in green are existing Boson tooling that are relatively non-controversial, and should be usable upstream with some minor changes. Items shaded in yellow are existing Boson tooling which the community may choose to adopt, but which folks might have strong feelings about, so should be discussed. Items shaded in orange highlight new artifacts that would be required to replace existing Boson tooling that is specific to Red Hat as a vendor.


## Not Covered

This document does not address several important issues that were raised in the initial Boson presentation to the Knative community. For example, the current proposal is not concerned with event streaming, on-cluster builds, or event source/sink declarations. The Boson team is aware of these concerns, and in some cases may have design documents or other supporting material. But there is no existing code or implementation to reference for these features. We assume community adoption would imply that implementation of these features would be a natural growth of the project under the Knative umbrella, and that the specifics of their implementations would be derived from the community.