Make invalid states unrepresentable - someone

Background

Some problems:

• We have redundant nil checks everywhere
• We forget to populate fields of structs when doing raw initialization

What can we do? Leverage the compiler more:

• Compiler does the heavy lifting
• Catch errors in development

Go does not make this easy for us, unfortunately. We do not have good enum types, nor generics. What can we do? Let's do a thought experiment: be as extreme as possible with the Go type system to increase safety / purity of our application.

Avoid Active Validation

Active validation has problems:

func createFoo(name string, value *big.Int, receivedAt time.Time) error {
    if !nameIsValid(name) {
    }
    if !valueExists(value) || valueLessThanRequired(value) {
   
    }
    if !receivedWithinThreshold(receivedAt) {
        
    }
}

• Captures a point in time with no guarantee of ongoing validity
• High chance we'll forget a future validation
• Hard to test, each test requires complex setup for each branch
• High-trust approach to writing code

Prefer dedicated constructors for types

func NewUserValue(val *big.Int) (UserValue, error) {
    if !valueExists(value) || valueLessThanRequired(value) {
       ... // Err
    }
    return UserValue{val}, nil
}

Then, we know that create foo has validated inputs:

func createFoo(name Name, value UserValue, receivedAt ThresholdTimestamp) error { ... }

Problem, what about someone just doing Name{someBadString} as a raw caller? Instead, we make Name an interface and prevent exporting its raw struct. The only way to get a Name type is via the constructor, which validates inputs.

Zero Values

Zero values of important types can bypass their constructor requirements we mentioned above. Anyone can create a Name{someBadString} from anywhere, bypassing our controls.

Moreover, sometimes the definition of what "allowed zero values" are is complex, as we've had many issues with using nil vs. empty slices in serialization paths.

Maybe we can mark values as complete or incomplete. Types with fields that are allowed to be zero values shouldn't affect the zero-value-ness of that type, or we should have more granular control over them.

Let's define a Complete interface which is met by types that must be complete.

type Complete interface {
    IsComplete() bool
}

The above returns false is the value is the zero value and is invalid, or for complex types, if any of its inner methods that satisfy Complete return false.

type Foo struct {
    Name Name
    ReceivedAt ThresholdTimestamp
}

func (f *Foo) IsComplete() bool {
    return f.Name.IsComplete() && f.ReceivedAt.IsComplete()
}

func doSomething(name Name, receivedAt ThresholdTimestamp) {
    ...
    f := &Foo{
        Name: name,
        ReceivedAt: receivedAt,
    }
    if types.IsIncomplete(f) {
        return types.ErrIncomplete
    }
    ...
}

Some of these ideas can be used to implement nice forking logic depending on fork versions! Super-structs such as beacon state could benefit from this if we have a notion of completeness depending on fork object

Two Kinds of Errors with Completeness

There are two categories of validation errors in applications such as Prysm:

1. Validation errors: bad input, malformed, not spec compliant, etc.
2. Completeness errors: of a single kind: only developers can cause completeness errors. Detectable at dev time

Safety Around Completeness

• Good tests make completeness errors easy to catch
• Crazy idea:
  • Build tags to compile zero-value checks during development, and panic if they fail
  • Include this in CI

//go:build release

func NewService(foo Foo) Service {
    return newService(foo)
}

and our dev build:

//go:build develop

func NewService(foo Foo) Service {
    types.ValidateCompleteness(foo)
    return newService(foo)
}

Pretty radical idea, but it's an idea nonetheless. This approach panics in development, helping us set up CI jobs that fail before they reach prod.

Options

nil as the expression of absence is pretty terrible in our design, as it can mean an accidental omission, or it could be on purpose and make our completeness checks tricky. Other languages that do not have nil instead use optional types. Here's what they look like in Go:

type Option[T any] struct {
    t T
}

func None[T any]() Option[T] {
    return Option[T]{nil}
}

func Some[T any](x T) Option[T] {
    return Option[T]{&x}
}

func (x Option[T]) IsNone() bool {
    return x.value == nil
}

func (x Option[T]) IsSome() bool {
    return x.value != nil
}

func (x Option[T]) Unwrap() T {
    return *x.value
}

Why? Well we can express absence safely without nil pointer dereferences, and because option types are non-nilable, we will receive compile time errors if we forget to add them to a struct:

type Transaction struct {
    From common.Address
    To common.Address
    Data []byte
    Value Option[*big.Int]
}

Compile time error below! We forgot Value

func foo() {
    ...
    tx := &Transaction{
        from: from,
        to: to,
        data: data,
    }
}

Instead,

func foo() {
    ...
    tx := &Transaction{
        from: from,
        to: to,
        data: data,
        value: None[*big.Int](),
    }
}

Combining Options With Completeness

We can also protect our option values from invalid states by adding the type constraint to our None constructor and checking for completeness when building an option:

func None[T Complete]() Option[T] {
	return Option[T]{nil}
}

func Some[T any](x T) Option[T] {
    if cpt, ok := x.(types.Complete); ok {
        if !x.IsComplete() {
            return ErrIncomplete
        }
    }
    return Option[T]{&x}
}