> Make invalid states unrepresentable - someone # Background Some problems: - We have redundant nil checks everywhere - We forget to populate fields of structs when doing raw initialization What can we do? Leverage the compiler more: - Compiler does the heavy lifting - Catch errors in development Go does not make this easy for us, unfortunately. We do not have good enum types, nor generics. What can we do? Let's do a thought experiment: be as extreme as possible with the Go type system to increase safety / purity of our application. ## Avoid Active Validation Active validation has problems: ```go func createFoo(name string, value *big.Int, receivedAt time.Time) error { if !nameIsValid(name) { } if !valueExists(value) || valueLessThanRequired(value) { } if !receivedWithinThreshold(receivedAt) { } } ``` - Captures a point in time with no guarantee of ongoing validity - High chance we'll forget a future validation - Hard to test, each test requires complex setup for each branch - High-trust approach to writing code Prefer dedicated constructors for types ```go! func NewUserValue(val *big.Int) (UserValue, error) { if !valueExists(value) || valueLessThanRequired(value) { ... // Err } return UserValue{val}, nil } ``` Then, we know that create foo has _validated_ inputs: ```go! func createFoo(name Name, value UserValue, receivedAt ThresholdTimestamp) error { ... } ``` Problem, what about someone just doing `Name{someBadString}` as a raw caller? Instead, we make `Name` an `interface` and prevent exporting its raw struct. The **only way** to get a `Name` type is via the constructor, which validates inputs. ## Zero Values Zero values of important types can bypass their constructor requirements we mentioned above. Anyone can create a `Name{someBadString}` from anywhere, bypassing our controls. Moreover, sometimes the definition of what "allowed zero values" are is complex, as we've had many issues with using nil vs. empty slices in serialization paths. Maybe we can mark values as **complete or incomplete**. Types with fields that are allowed to be _zero values_ shouldn't affect the _zero-value-ness_ of that type, or we should have more granular control over them. Let's define a `Complete` interface which is met by types that must be complete. ```go type Complete interface { IsComplete() bool } ``` The above returns false is the value is the zero value and is invalid, or for complex types, if any of its inner methods that satisfy `Complete` return false. ```go! type Foo struct { Name Name ReceivedAt ThresholdTimestamp } func (f *Foo) IsComplete() bool { return f.Name.IsComplete() && f.ReceivedAt.IsComplete() } func doSomething(name Name, receivedAt ThresholdTimestamp) { ... f := &Foo{ Name: name, ReceivedAt: receivedAt, } if types.IsIncomplete(f) { return types.ErrIncomplete } ... } ``` > Some of these ideas can be used to implement nice forking logic depending on fork versions! Super-structs such as beacon state could benefit from this if we have a notion of completeness depending on fork object ## Two Kinds of Errors with Completeness There are two categories of validation errors in applications such as Prysm: 1. **Validation errors**: bad input, malformed, not spec compliant, etc. 2. **Completeness errors**: of a single kind: only developers can cause completeness errors. Detectable at dev time ## Safety Around Completeness - Good tests make completeness errors easy to catch - Crazy idea: - Build tags to compile zero-value checks during development, and panic if they fail - Include this in CI ```go! //go:build release func NewService(foo Foo) Service { return newService(foo) } ``` and our dev build: ```go! //go:build develop func NewService(foo Foo) Service { types.ValidateCompleteness(foo) return newService(foo) } ``` Pretty radical idea, but it's an idea nonetheless. This approach panics in development, helping us set up CI jobs that fail before they reach prod. ## Options `nil` as the expression of absence is pretty terrible in our design, as it can mean an accidental omission, or it could be on purpose and make our completeness checks tricky. Other languages that do not have `nil` instead use optional types. Here's what they look like in Go: ```go! type Option[T any] struct { t T } func None[T any]() Option[T] { return Option[T]{nil} } func Some[T any](x T) Option[T] { return Option[T]{&x} } func (x Option[T]) IsNone() bool { return x.value == nil } func (x Option[T]) IsSome() bool { return x.value != nil } func (x Option[T]) Unwrap() T { return *x.value } ``` Why? Well we can express absence _safely_ without nil pointer dereferences, and because option types are non-nilable, we will receive compile time errors if we forget to add them to a struct: ```go! type Transaction struct { From common.Address To common.Address Data []byte Value Option[*big.Int] } ``` Compile time error below! We forgot `Value` ```go func foo() { ... tx := &Transaction{ from: from, to: to, data: data, } } ``` Instead, ```go= func foo() { ... tx := &Transaction{ from: from, to: to, data: data, value: None[*big.Int](), } } ``` ## Combining Options With Completeness We can also protect our option values from invalid states by adding the type constraint to our `None` constructor and checking for completeness when building an option: ```go= func None[T Complete]() Option[T] { return Option[T]{nil} } func Some[T any](x T) Option[T] { if cpt, ok := x.(types.Complete); ok { if !x.IsComplete() { return ErrIncomplete } } return Option[T]{&x} } ```