# Sum Types in FTL
**Author**: @worstell @deniseli
<!-- Status of the document - draft, in-review, etc. - is conveyed via HackMD labels -->

## Description

Support for sum types in the FTL type system.

## Goals

- Users can express sum types in their runtime code and annotate them for extraction to the FTL schema
- Approach must integrate with runtime compilers such that sum type behavior is upheld in application code, outside of FTL
- FTL-managed API layer handles sum types; verb and ingress calls will accept values of any of the allowed type variants

## Design

### Sum types as enums (inspired by [Swift enums](https://docs.swift.org/swift-book/documentation/the-swift-programming-language/enumerations/))

Enums can be either "value" enums, a collection of constant values with assigned names (consistent with C or Kotlin enums) or "type" enums (consistent with sum types/discriminated unions).

See [Enums in FTL](https://hackmd.io/oIEwp1IRR8avvPNOUrGU1Q) for discussion around declaring "value" enums. Discussed below are ways to declare "type" enums.

**Tagging struct types**

_Go_

package structs

//ftl:enum
type Structs interface { tag() }

type Foo struct {
}
func (Foo) tag() {}

type Bar struct {
}
func (Bar) tag() {}

_Kotlin_

package ftl.structs

data class Foo(
    ...
)

data class Bar(
    ...
)

@Enum
sealed class Structs {
    data class Foo(
        val value: ftl.structs.Foo
    ) : Structs()

    data class Bar(
        val value: ftl.structs.Bar
    ) : Structs()
}

_In Kotlin: Sealed class is the discriminator and data classes contained therein are the variants._

_We will require the `value` field (and validate that only this field is defined) on each of the data class variants. The name of the data class is the enum variant name and the `value` field type is the variant type._

_FTL_

module structs {
    enum Structs {
        Foo structs.Foo
        Bar structs.Bar
    }
}   

**Tagging non-struct types (underlying type must be supported in FTL)**

_Go_

package aliases

//ftl:enum
type ScalarOrList interface { scalarOrList() }

type Scalar string
func (Scalar) scalarOrList() {}

type List []string
func (List) scalarOrList() {}

_Kotlin_

package ftl.aliases

@Enum
sealed class ScalarOrList {
    data class Scalar(
        val value: String
    ) : ScalarOrList()

    data class List(
        val value: List<String>
    ) : ScalarOrList()
}

_FTL_

module aliases {
    enum ScalarOrList {
        Scalar String 
        List   [String]
    }
} 

**Tagging raw values**

_Go_

package rawvalues

//ftl:enum
type Color string

const (
  Red   Color = "Red"
  Blue  Color = "Blue"
  Green Color = "Green"
)

_Kotlin_

package ftl.aliases

@Enum
enum class Color(name: String) {
  RED("Red"),
  BLUE("Blue"),
  GREEN("Green"),
}

_FTL_

module rawvalues {
    enum Color {
        Red   String = "Red"
        Blue  String = "Blue"
        Green String = "Green"
    }
}

**Mix of multiple types with raw values**

_Go_

package mixed

//ftl:enum
type Mixed interface { mixed() }

type Int int
func (Int) mixed() {}

type ColorAlias Color
func (ColorAlias) mixed() {}

//ftl:enum
type Color string
const (
  Red   Color = "Red"
  Blue  Color = "Blue"
  Green Color = "Green"
)

_Kotlin_

package ftl.mixed

@Enum
sealed class Mixed {
    data class Int(
        val value: kotlin.Int
    ) : Mixed()

    data class ColorAlias(
        val value: ftl.mixed.Color
    ) : Mixed()
}

@Enum
enum class Color(name: String) {
  RED("Red"),
  BLUE("Blue"),
  GREEN("Green"),
}

_FTL_

module mixed {
    enum Color {
       Red   String = "Red"
       Blue  String = "Blue"
       Green String = "Green" 
    }
    
    enum Mixed {
       ColorAlias mixed.Color
       Int        Int
    }
}

"Value" and "type-only" variants cannot be mixed within a single enum.

Does this force us to support structured enums in value enums? We previously decided not to [here](https://hackmd.io/oIEwp1IRR8avvPNOUrGU1Q#Structured-enums).

#### Encoding/Decoding in Go

**Pre-work needed to support encoder changes**


- [Decouple custom encoder from `encoding/json`](https://github.com/TBD54566975/ftl/issues/1262); custom encoder is used for FTL-internal encoding/decoding (e.g. verb-to-verb calls)
- Custom encoder will only use `encoding/json` implementations for stdlib types supported in the FTL type system. This is only `time.Time` at the time of writing

- Implement `type_registry.go`, which is a registry of types (inspired by [K8s scheme](https://https://pkg.go.dev/k8s.io/client-go/kubernetes/scheme)), associating type identifiers with their `reflect.Type`. We will use this to “instantiate” types dynamically at runtime from just their names (used for unmarshaling, discussed below)
- `type_registry.go` will also manage a map of sum types, associating discriminators with their variants

package ftl

// TypeRegistry is a registry of types that can be instantiated by their qualified name.
// It also records sum types and their variants, for use in encoding and decoding.
//
// FTL manages the type registry for you, so you don't need to create one yourself.
type TypeRegistry struct {
	// sumTypes associates a sum type discriminator with its variants
	sumTypes map[string][]sumTypeVariant
	types    map[string]reflect.Type
}

type sumTypeVariant struct {
	name     string
	typeName string
}

// NewTypeRegistry creates a new type registry.
// The type registry is used to instantiate types by their qualified name at runtime.
func NewTypeRegistry() *TypeRegistry {
	return &TypeRegistry{types: map[string]reflect.Type{}, sumTypes: map[string][]sumTypeVariant{}}
}

// New creates a new instance of the type from the qualified type name.
func (t *TypeRegistry) New(name string) (any, error) {
	...
}

// RegisterSumType registers a Go sum type with the type registry. Sum types are represented as enums in the
// FTL schema.
func (t *TypeRegistry) RegisterSumType(discriminator reflect.Type, variants map[string]reflect.Type) {
	...
}

- Modify codegen for module `main.go`:
- `main.go` will register sum types based on the module schema (using native type information). This will need to include external module sum types as well as module-owned sumtypes—all sum types referenced in the module schema must be registered
- Add the TypeRegistry to the context that gets passed to the application

_Example generated `main.go` file:_

// Code generated by FTL. DO NOT EDIT.
package main

import (
   "context"

   "github.com/TBD54566975/ftl/go-runtime/scheme"
   "github.com/TBD54566975/ftl/common/plugin"
   "github.com/TBD54566975/ftl/go-runtime/server"
   "github.com/TBD54566975/ftl/backend/protos/xyz/block/ftl/v1/ftlv1connect"

   "ftl/petstore"
)

func main() {
	verbConstructor := server.NewUserVerbServer("petstore",
		server.HandleCall(petstore.AdoptPet),
	)
	ctx := context.Background()
	typeRegistry := ftl.NewTypeRegistry()
        typeRegistry.RegisterSumType(
            reflect.TypeFor[petstore.Pet](),
            map[string]any{
               "Dog", reflect.TypeFor[petstore.Dog](),
               "Cat", reflect.TypeFor[petstore.Cat](),
               "Snake", reflect.TypeFor[petstore.Snake](),
            })
	ctx = context.WithValue(ctx, "typeRegistry", typeRegistry)

	plugin.Start(ctx, "petstore", verbConstructor, ftlv1connect.VerbServiceName, ftlv1connect.NewVerbServiceHandler)
}

**Internal encoding changes (`encoding.go`)**
- Pass context to `encoder.Marshal` and `encoder.Unmarshal`
- `encoder.Marshal`:
- Sum type variants will be marshaled with the addition of a field containing their variant name—e.g: instead of `{"greeting":"woof"}`, `{"name":"Dog", "value":{"greeting":"woof"}}`
- `encoder.Unmarshal`:
- If we're unmarshaling into an interface:
- Check if it's a discriminator using the TypeRegistry from context. If it's not, error
- If it's a discriminator, find a match between the value of the `"name"` field from the input bytes and any variants for this discriminator
- Once identified, use the TypeRegistry to load the matching variant from the types registry
- Unmarshal the contents of `"value"` into this derived type using `decodeValue` and the contents of the input bytes

**Ingress encoding changes**

We will expect the same JSON structure on ingress as we use internally for sum type (enum) variants; i.e. values should be wrapped in an outer type with fields "name" and "value": `{“name”:”Dog”, “value”:{"greeting":"woof"}}`.

### Non-schema changes to support handling

* Modify request validation to accept any of the allowed types when comparing against a sum type in the schema
* `jsonschema` should reflect the structure discussed above—e.g.`{"name": string, "value": oneOf{<variant types>}}`.

### Rejected Alternatives

#### Sum types as their own concept (not enums)

**FTL**

module petstore {
    sumtype Pet = Dog | Cat | Snake
    
    data Cat {
        ...
    }
    
    data Dog {
        ...
    }
    
    data Snake {
        ...
    }
    
    data AdoptRequest {
        name   String
        animal petstore.Pet
    }
    
    data AdoptResponse {
        success Bool
    }
    
    verb AdoptPet(petstore.AdoptRequest) petstore.AdoptResponse
}

**Go**

package petstore

import ...

//ftl:enum
type Pet interface { pet() }

type Cat struct {
    ...
}

func (Cat) pet() {}

type Dog struct {
    ...
}

func (Dog) pet() {}

type Snake struct {
    ...
}

func (Snake) pet() {}

type AdoptRequest struct {
   Name   string
   Animal Pet
}

type AdoptResponse struct {
   Success bool
}

//ftl:enum
func AdoptPet(ctx context.Context, req AdoptRequest) (AdoptResponse, error) {
	...
}

The Go schema extractor must validate that all interface functions are private.

In the future, if we extend the FTL type system to support interfaces in other contexts, we will determine sumtypes on the basis of the interface containing a private function.

#### Kotlin (prospective, to be implemented at a later date)

package ftl.petstore

import ...

data class Dog(
    ...
)

data class Cat(
    ...
)

data class Snake(
    ...
)

@Export
sealed class Pet {
    data class Dog(
        val value: ftl.petstore.Dog
    ) : Pet()

    data class Cat(
        val value: ftl.petstore.Cat
    ) : Pet()

    data class Snake(
        val value: ftl.petstore.Cat
    ) : Pet()
}

data class AdoptRequest(val name: String, val animal: Pet)
data class AdoptResponse(val success: Boolean)

@Export
fun adoptPet(context: Context, req: AdoptRequest): AdoptResponse {
  ...
}

We will require the `value` field (and validate that only this field is defined) on each of the data classes contained in the sealed class. The type of this field will be the type of the variant itself.