# OpenFunction Deep Dive

<div align=center><img src=https://github.com/OpenFunction/OpenFunction/blob/main/docs/images/openfunction-logo-gif.gif?raw=true></div>

[OpenFunction](https://openfunction.dev/) is a Cloud Native FaaS (Function as a Service) platform for running serverless workloads with ease.

Recently, OpenFunction announces its v0.6.0 release. This is a release with important new features and significant improvements that makes a big move towards its v1.0.0 release. The major changes in v0.6.0 include:
- Refactor function serving definition, the core API version has been upgraded from v1alpha2 to v1beta1
- Add Dapr support to Knative sync functions which also add a HTTP trigger to async functions.
- Unify the scale options of sync and async functions.
- Add function plugin mechanism that enables user to add custom logic before and after a function.
- Integrate OpenFunction with SkyWalking, now users are able to use SkyWalking to analyze the performance of sync and async functions.
- Optimize Go and Nodejs builder, the function images of Go and Nodejs are significantly reduced.
- Now users can choose not to install some components and the function controller will no longer report an error anymore. For example, users can only install Knative and Shipwright without installing Dapr and KEDA if only sync functions are used.

<!--
## OpenFunction v0.6.0 介绍

OpenFunction v0.6.0 已于近日发布，是 OpenFunction 发布以来变化最大同时也是最完善的版本，为最终发布 v1.0.0 打下了坚实的基础。

v0.6.0 比较显著的变化包括：
- 重构了函数运行时定义，core API 的版本从 v1alpha2 升级为 v1beta1.
- 将 Dapr 与 Knative 同步函数集成，增强同步函数能力的同时，给异步函数增加了 HTTP 触发器
- 统一了同步函数与异步函数的伸缩设置，使用户可以更方便的配置函数扩展
- 增加了函数插件机制，使用户可以自由定义函数执行前后的自定义逻辑
- 与 SkyWaking 社区紧密合作，用户现在可以利用 SkyWalking v9 分析同步函数和异步函数的性能
- 对 Go 和 Nodejs 的 Builder 进行了优化，极大减小了 Go 和 Nodejs 函数镜像的大小
- 用户可以不安装不需要的组件而不会报错，比如可以只安装异步函数运行时 Dapr 和 KEDA，而无需安装 Knative 以及 Shipwright
-->

## The design and architecture of OpenFunction


OpenFunction is designed with several principles in mind: open，flexible，and pluggable
- By introducing Shipwright in the function build phase, users are able to build function images with Cloud Native Buildpacks. At the mean time, users can also choose to use Kaniko, Buildkit, and Buildah to build his application images. Shipwright provides an excellent framework that enable users to switch between these build technologies.
- In the function serving phase, OpenFunction introduces OpenFunction async runtime, Knative sync runtime, and OpenFunction sync runtime(will be released in v0.7.0 or later). Users are able to choose or switch the appropriate runtime for different scenarios in a pluggable way.
- By introducing Dapr, both sync and async function runtimes are able to utilize the power of distributed application runtime including state management, data binding, and pubsub etc. Different types of functions are able to communicate with various middleware, data sources like MySQL, PostgreSQL, Kafka, NATS Streaming, MQTT and cloud providers' MQ in a unified, low cost and pluggable manner.
- The EventBus of OpenFunction Events is built on Dapr too with pluggable feature.
- By introducing KEDA, OpenFunction async functions are able to scale precisely based on the specific metrics of various event sources like Kafka, Redis, NATS Streaming, cloud providers' message queue or event bus etc.
- By introducing a plugin mechanism, users are able to define custom logic before and after function code. OpenFunction's integration with SkyWalking is based on this mechanism.

<!--
## OpenFunction 设计与架构详解

OpenFunction 设计上秉承着开放、灵活、可插拔的原则，具体来说：

- Function Build 阶段通过引入 Shipwright，使得用户在 Tekton 的控制下不仅可以使用 Cloud Native Buildpacks 技术不依赖于 Dockerfile 从源码直接构建函数镜像，还能使用 Kaniko, Buildkit 或 Buildah 基于 Dockerfile 构建应用镜像。
- Function Serving 阶段通过引入 OpenFunction 异步函数运行时(async runtime)、Knative 同步函数运行时(knative runtime)、OpenFunction 同步函数运行时(sync runtime, 计划 v0.7.0 或 v1.0.0 引入) 以可插拔的方式给用户提供了多种选择。
- 通过引入 Dapr, 使得同步函数和异步函数都可以充分利用分布式应用运行时的各种基础能力如状态管理(state)，数据绑定(binding), 发布订阅(pubsub)等，可以用统一的、低成本、可插拔的方式对接各种中间件和数据源如 MySql, PG, Kafka, NATS Streaming, MQTT 等以及多个云厂商的 MQ.
- 以 Dapr 为基础设计的 OpenFunction Events 事件框架也具有灵活可插拔的特点
- 通过引入 KEDA，使得异步函数可以根据数据源的特性进行更精确的伸缩如 Kafka 的消费延迟、各云厂商 MQ 或 EventBus 特有的指标、多种开源中间件特有的指标如 Redis， NATS Streaming 等。
- 通过插件机制使得用户可以在函数运行前后执行自定义的插件逻辑，OpenFunction 与 SkyWalking 的集成就是通过对插件机制的集成实现的
-->
## OpenFunction step by step

With OpenFunction, you can build and then launch sync or async functions.
You can also build and launch functions separately.

### Build function image only: Function source to Image
OpenFunction can be used to build function image only without launching the function.

cat <<EOF | kubectl apply -f -
apiVersion: core.openfunction.io/v1beta1
kind: Function
metadata:
  name: function-sample
spec:
  version: "v2.0.0"
  image: "openfunctiondev/sample-go-func:latest"
  imageCredentials:
    name: push-secret
  port: 8080 # default to 8080
  build:
    builder: openfunction/builder-go:v2-1.16
    env:
      FUNC_NAME: "HelloWorld"
      FUNC_CLEAR_SOURCE: "true"
    srcRepo:
      url: "https://github.com/OpenFunction/samples.git"
      sourceSubPath: "functions/knative/hello-world-go"
      revision: "main"
EOF

### Build application image only: Application source to Image

### Launch Knative sync function
Once the function image is built, you can launch the function whenever you want:

cat <<EOF | kubectl apply -f -
apiVersion: core.openfunction.io/v1beta1
kind: Function
metadata:
  name: function-sample
spec:
  version: "v2.0.0"
  image: "openfunctiondev/sample-go-func:latest"
  imageCredentials:
    name: push-secret
  port: 8080 # default to 8080
  serving:
    template:
      containers:
        - name: function
          imagePullPolicy: Always
    runtime: "knative"
EOF

Sync functions is useful when your scenario is to deliver events as plain HTTP payloads or CloudEvent payloads in a synchronous way.

You can find the source code of this Knative sync function [here](https://github.com/OpenFunction/samples/tree/main/functions/Knative/hello-world-go)

### HTTP trigger for async functions：The combination of Knative sync functions, Dapr, and Async functions

You can use Knative sync functions and async functions together like below:


This way you actually add a HTTP trigger to a async function:

- Define a Knative sync function to serve HTTP request and send the payload received to a message queue(or event bus) like Kafka:
> The function communicate with Kafka via a Dapr sidecar container, so Kafka could be replaced by any other message queue(or event bus) such as NATS Streaming, Azure Event Hub or GCP Pub/Sub with ease without changing the function code.

cat <<EOF | kubectl apply -f -
apiVersion: core.openfunction.io/v1beta1
kind: Function
metadata:
  name: function-front
spec:
  version: "v1.0.0"
  image: "openfunctiondev/sample-knative-dapr:latest"
  imageCredentials:
    name: push-secret
  port: 8080 # default to 8080
  build:
    builder: openfunction/builder-go:latest
    env:
      FUNC_NAME: "ForwardToKafka"
      FUNC_CLEAR_SOURCE: "true"
    srcRepo:
      url: "https://github.com/OpenFunction/samples.git"
      sourceSubPath: "functions/knative/with-output-binding"
      revision: "main"
  serving:
    scaleOptions:
      minReplicas: 0
      maxReplicas: 5
    runtime: knative
    outputs:
      - name: target
        component: kafka-server
        operation: "create"
    bindings:
      kafka-server:
        type: bindings.kafka
        version: v1
        metadata:
          - name: brokers
            value: "kafka-server-kafka-brokers:9092"
          - name: authRequired
            value: "false"
          - name: publishTopic
            value: "sample-topic"
          - name: topics
            value: "sample-topic"
          - name: consumerGroup
            value: "function-front"
    template:
      containers:
        - name: function
          imagePullPolicy: Always
EOF

The Knative sync function is defined as below:

package sender

import (
	"encoding/json"
	"log"

	ofctx "github.com/OpenFunction/functions-framework-go/context"
)

func ForwardToKafka(ctx ofctx.Context, in []byte) (ofctx.Out, error) {
	var greeting []byte
	if in != nil {
		log.Printf("http - Data: %s", in)
		greeting = in
	} else {
		log.Print("http - Data: Received")
		greeting, _ = json.Marshal(map[string]string{"message": "Hello"})
	}

	_, err := ctx.Send("target", greeting)
	if err != nil {
		log.Print(err.Error())
		return ctx.ReturnOnInternalError(), err
	}

	return ctx.ReturnOnSuccess(), nil
}

- Define an async function to retrieve data from a message queue(or event bus) like Kafka:

cat <<EOF | kubectl apply -f -
apiVersion: core.openfunction.io/v1beta1
kind: Function
metadata:
  name: kafka-input
spec:
  version: "v1.0.0"
  image: openfunctiondev/kafka-input:latest
  imageCredentials:
    name: push-secret
  build:
    builder: openfunction/builder-go:latest
    env:
      FUNC_NAME: "HandleKafkaInput"
      FUNC_CLEAR_SOURCE: "true"
    srcRepo:
      url: "https://github.com/OpenFunction/samples.git"
      sourceSubPath: "functions/async/bindings/kafka-input"
      revision: "main"
  serving:
    runtime: async
    scaleOptions:
      minReplicas: 0
      maxReplicas: 10 
      keda:
        scaledObject:
          pollingInterval: 15
          # minReplicaCount: 0
          # maxReplicaCount: 10
          cooldownPeriod: 45
    triggers:
      - type: kafka
        metadata:
          topic: sample-topic
          bootstrapServers: kafka-server-kafka-brokers.default.svc:9092
          consumerGroup: kafka-input
          lagThreshold: "20"
    inputs:
      - name: greeting
        component: target-topic
    bindings:
      target-topic:
        type: bindings.kafka
        version: v1
        metadata:
          - name: brokers
            value: "kafka-server-kafka-brokers:9092"
          - name: topics
            value: "sample-topic"
          - name: consumerGroup
            value: "kafka-input"
          - name: publishTopic
            value: "sample-topic"
          - name: authRequired
            value: "false"
    template:
      containers:
        - name: function
          imagePullPolicy: Always

The async function is defined as below:

package bindings

import (
	"encoding/json"
	"fmt"

	ofctx "github.com/OpenFunction/functions-framework-go/context"
)

func HandleKafkaInput(ctx ofctx.Context, in []byte) (ofctx.Out, error) {
	var msg Message
	err := json.Unmarshal(in, &msg)
	if err != nil {
		fmt.Println("error reading message from Kafka binding", err)
		return ctx.ReturnOnInternalError(), err
	}
	fmt.Printf("message from Kafka '%s'\n", msg)
	return ctx.ReturnOnSuccess(), nil
}

type Message struct {
	Msg string `json:"message"`
}

- Test the entire pipeline
Now we send some payload to the Knative sync function and test the HTTP Trigger

kubectl run curl --image=radial/busyboxplus:curl -i --tty --rm
curl -d '{"message":"Awesome OpenFunction!"}' -H "Content-Type: application/json" -X POST http://openfunction.io.svc.cluster.local/default/function-front

Take a look at the sync function's output:

kubectl logs -f \
  $(kubectl get po -l \
  openfunction.io/serving=$(kubectl get functions function-front -o jsonpath='{.status.serving.resourceRef}') \
  -o jsonpath='{.items[0].metadata.name}') \
  function

Take a look at the async function's output:

kubectl logs -f \
  $(kubectl get po -l \
  openfunction.io/serving=$(kubectl get functions kafka-input -o jsonpath='{.status.serving.resourceRef}') \
  -o jsonpath='{.items[0].metadata.name}') \
  function

Please refer to [Add HTTP trigger for async functions](https://github.com/OpenFunction/samples/tree/main/functions/knative/with-output-binding) for more details like function source code.

### OpenFunction async functions: Unleash the power of Dapr and KEDA

Serverless workloads are usually event-driven and asynchronous. Most serverless or FaaS frameworks convert async events received from event source to HTTP payloads and then send them to a serverless backend for processing which usually scales based on the load of HTTP.

OpenFunction async functions process async events in a different and more intuitive way thanks to Dapr and KEDA:
- Unlike other Serverless or FaaS platform, OpenFunction async functions scales based on KEDA which use the metrics of event source directly.
- And the communication between OpenFunction async functions and event source(such as a message queue or event bus) is offloaded to Dapr. In this way, the job of creating async functions is significantly simplified and the async functions are more lightweight.

Lets take a look at a log alerting scenario:
- A K8s cluster sends logs to a Kafka cluster
- A OpenFunction async function is created to receive log events from the Kafka cluster, and then analyze them.
- Whenever this async function finds a `404` error in the `wordpress` pods of the `demo-project` namespace, it will create an alert message and send it to the HTTP endpoint of [Notification Manager](https://github.com/kubesphere/notification-manager)
- The [Notification Manager](https://github.com/kubesphere/notification-manager) will then send the alerts to a Slack channel.


Openfunction async function is a great fit for scenarios like this:
- Powered by KEDA, async functions autoscale based on the `consume lag` of the Kafka topic. This way the logs sent to Kafka will be processed as fast as possible with the right amout of resources (Less redundant or wasted)
- The async function's input is the Kafka topic, and its output is a HTTP endpoint. Both of them could be defined as a Dapr binding component.
- The function container only need to communicate with a Dapr sidecar container via grpc without introducing a Kafka SDK or a HTTP client.

The log alerting async function is defined as below:

cat <<EOF | kubectl apply -f -
apiVersion: core.openfunction.io/v1beta1
kind: Function
metadata:
  name: logs-async-handler
spec:
  version: "v2.0.0"
  image: openfunctiondev/logs-async-handler:latest
  imageCredentials:
    name: push-secret
  build:
    builder: openfunction/builder-go:latest
    env:
      FUNC_NAME: "LogsHandler"
      FUNC_CLEAR_SOURCE: "true"
      # Use FUNC_GOPROXY to set the goproxy
      # FUNC_GOPROXY: "https://goproxy.cn"
    srcRepo:
      url: "https://github.com/OpenFunction/samples.git"
      sourceSubPath: "functions/async/logs-handler-function/"
      revision: "main"
  serving:
    runtime: "async"
    scaleOptions:
      keda:
        scaledObject:
          pollingInterval: 15
          minReplicaCount: 0
          maxReplicaCount: 10
          cooldownPeriod: 45
    triggers:
      - type: kafka
        metadata:
          topic: logs
          bootstrapServers: kafka-server-kafka-brokers.default.svc.cluster.local:9092
          consumerGroup: logs-handler
          lagThreshold: "20"
    template:
      containers:
        - name: function
          imagePullPolicy: Always
    inputs:
      - name: kafka
        component: kafka-receiver
    outputs:
      - name: notify
        component: notification-manager
        operation: "post"
    bindings:
      kafka-receiver:
        type: bindings.kafka
        version: v1
        metadata:
          - name: brokers
            value: "kafka-server-kafka-brokers:9092"
          - name: authRequired
            value: "false"
          - name: publishTopic
            value: "logs"
          - name: topics
            value: "logs"
          - name: consumerGroup
            value: "logs-handler"
      notification-manager:
        type: bindings.http
        version: v1
        metadata:
          - name: url
            value: http://notification-manager-svc.kubesphere-monitoring-system.svc.cluster.local:19093/api/v2/alerts
EOF

You can find the source code of this async log alerting function [here](https://github.com/OpenFunction/samples/tree/main/functions/async/logs-handler-function).

### Knative sync functions and state: The combination of Knative and Dapr

Functions are usually considered stateless and easily to scale. But inevitablly there're scenarios in which you want to store or retrieve some sort of states in functions, for example:
- Connect to MySQL or PostgreSql and then execute SQL to store or retrieve data
- Store and query data as key/value pairs to or from Redis, MongoDB, MySQL, PostgreSql, AWS DynamoDB, GCP Firestore，and Azure CosmosDB etc.

Functions is meant to be simple, but dealing with various state stores introduce additional complexity, and this is where Dapr could help.

OpenFunction is planning to introduce Dapr state components and rdb output bindings to solve the function state management requirement in a simple and pluggable manner.


### OpenFunction sync function preview

### Analyze Knative sync functions performance with SkyWalking

### Analyze Openfunction async functions performance with SkyWalking

### OpenFunction Events