6/29/2023

This proposal attempts to do two things:

1. Describe the current and future needs of Hyperlight's public Sandbox API
2. Describe how to meet these needs with a single, type-safe implementation in Rust

This proposal is linked from issue #754

Current state

Currently, in the dev branch, we have several different types of Sandboxes, detailed in this section.

UninitializedSandbox

As its name implies, the UnintializedSandbox - represents a sandbox that has not yet been initialized and thus is not ready to execute guest code.

The primary purposes of this component are twofold:

1. Allow a caller to create a VM and load guest code into its memory, but stop short of actually running any computation
2. Allow a caller to initialize a new fully-functional sandbox with custom logic
   • At the time of this writing, this custom logic feature is used most frequently to load a web assembly module into the Hyperlight-WASM guest

(Initialized) Sandbox

As described in the previous section, the UninitializedSandbox has an initialization function attached to it. The return value of this function is a Sandbox type, which represents a fully initialized Hyperlight virtual machine (VM) inside of which the user can execute code.

Ths Sandbox has two variants of note:

• reusable - you can call guest functions on these two or more times. internally, a reusable sandbox takes snapshots of VM memory between function calls, thus allowing the end user to "hot-swap" different guests into it
• non-reusable - you can only call one guest function, then this Sandbox is destroyed. No further action is possible

Current design notes

As implied easlier, this overall design was put in place to fulfill the requirements primarily of the first major user of Hyperlight, Hyperlight-WASM. As we look toward the future of Hyperlight and its users, we see several developments, listed below, that require a more flexible, generic design:

• Hyperlight-WASM's requirements have evolved
• We want to support a more diverse set of workloads beyond only Hyperlight-WASM

Aside: who's a user of what?

This document attempts to define a clear separation between three different types of code, listed below.

1. Hyperlight-Core: the project at github.com/deislabs/hyperlight
2. Libraries - code that uses Hyperlight-Core, including tests inside Hyperlight-Core and Hyperlight-WASM
3. End users - code that interacts with libraries from (2), such as the systems that run within AFD and more

This distinction is important because this proposal moves a significant amount of implementation and API design from Hyperlight-Core to libraries.

Future requirements

As requirements have changed over time, one general idea continues to arise. We would like to implement a generic way to implement a sandbox that is "partially loaded", allow libraries to load only part of a guest's code and/or memory, and then load the remainder at a later time. What "partially loaded" actually means, and how to load the remainder of a guest depends greatly on the library.

The Hyperlight-WASM project would enlist this feature set by doing the following steps, in both forwards and backwards order:

1. Load the WAMR-based guest, but not yet load a customer's WASM module, into a VM
2. Load the customer's WASM module into the partially-loaded Sandbox from (1)

This feature set would also allow an end user to "roll back" from a fully-loaded Sandbox in (2) to a partially-loaded Sandbox in (1) that has just the WAMR-based guest, or rollback from one intermediate state to another in other circumstances.

Current limitations

In C#, we have implemented a single Sandbox class that internally has a recycleAfterRun boolean flag to indicate whether that Sandbox can be "recycled", and memory can be swapped in and out of it (this flag is passed in via the SandboxRunOptions.RecycleAfterRun argument). This strategy solves the hot-swap functionality described above.

As implied by the "future requirements" section, though, the C# Sandbox does not directly allow you to "partially load" it. In our Rust implementation, however, we have started to represent a "partial" Sandbox state with our UninitializedSandbox struct.

This design currently limits us in several ways with respect to the "future requirements" section:

1. We can only allow libraries to implement one "partial" state rather than an arbitrary number N partial states
2. We do not provide the ability to roll back from a more "advanced" state to a previous one

Proposed design

From our requirements and our plans for developing the Hyperlight ecosystem going forward, a few themes emerge:

• In Hyperlight Core, we're currently trying to define application-specific rules like sandbox initialization states. Instead, I think we should get out of the business of doing that and build a general enough API to empower library authors to do that safely.
• We have a fairly clear, general set of states and transitions, and thus need to be able to model a state machine. We can use Rust's rich type system to model both the states and their transitions. Doing this enlists the compiler to check and eliminate an entire class of bugs and runtime errors, greatly increasing the safety of resulting libraries.

I propose Hyperlight provide a set of traits and reusable logic to do both these things. The code that follows illustrates this approach.

Sandboxes in the new design

/// The minimal functionality of a sandbox
pub trait Sandbox {
    fn generation_num(&self) -> u8;
}

/// A "final" sandbox implementation that can be reused and devolved, but not
/// evolved.
///
/// Thus, `ReusablsSandbox`es implement `DevolvableSandbox`
/// and have `run` methods that borrow `self`, so they can be called more
/// than once.
pub trait ReusableSandbox {
    /// Borrow `self` and run this sandbox.
    fn run(&self) -> Result<()>;
}

/// A fully-initialized sandbox that can run guest code only once,
/// and then must be destroyed.
pub trait OneShotSandbox {
    /// Consume `self` and run the sandbox.
    ///
    /// After this call, you can no longer use this `OneShotSandbox`
    fn run(self) -> Result<()>;
}

/// A sandbox that can be evolved to a next state
pub trait EvolvableSandbox<Next: Sandbox>: Sandbox + Sized {
    fn evolve(self) -> Result<Next>;
}

/// A sandbox that can be devolved to a previous state
pub trait DevolvableSandbox<Next: Sandbox>: Sandbox + Sized {
    fn devolve(self) -> Result<Next>;
}

Pull Request #756 contains the above code and an extensive code sample to model Hyperlight-WASM.

Specifying and implementing transitions between states

It's intended that library authors will create structs as necessary to represent the states specific to their application's needs. Each state will implement Sandbox at minimum, and will implement EvolvableSandbox and DevolvableSandbox as necessary to specify each state transition. If a state transition is not implemented, it will be disallowed by the Rust compiler. A simple example follows:

struct State1{}
struct State2{}

impl Sandbox for State1 {
    ...
}
impl Sandbox for State2 {
    ...
}

/// Specifies the transition from State1 => State2
impl EvolvableSandbox<State2> for State1 {
    fn evolve(self) -> State2 {
        State2{}
    }
}

/// Specifies the transition from State2 => State1
impl DevolvableSandbox<State1> for State2 {
    fn devolve(self) -> State1 {
        State1{}
    }
}

Implementing states

We've shown an example set of possible states and transitions above, but it's important to note almost all end users will interact with libraries like Hyperlight-WASM rather than Hyperlight Core.

Thus, for all practical purposes, this API, along with much of the rest of Hyperlight's public API, is directed only at library authors. I've tried to design the components in this proposal to enable library authors (like us, as we build Hyperlight-WASM) to build ergonomic, intuitive APIs for their end users.

To this end, I believe we can make available a set of "building block" APIs to ease the process of building the desired Sandbox, EvolvableSandbox, DevolvableSandbox etc… implementations in a library like Hyperlight-WASM. I believe as Hyperlight Core is used more widely, our ability to provide well-design building block APIs alongside our sandbox/transition abstractions will become more essential.

The sub-sections herein describe these building blocks in detail.

Sandbox, OneShotSandbox, and ReusableSandbox

Most of the primary functionality within these implementations is concerned with dispatching calls to guest functions from the host, and vice-versa. The major pieces of this process are as follows:

1. Allow guest functions to be easily dispatched, accounting for guest callbacks back into the host and optionally doing state snapshot/restore operations appropriately
2. Allow host functions to be registered where appropriate
3. Allow users to react to errors in guest calls from (1) with custom code.

If we provide a construct to library authors that provides a set of easy-to-use, safe APIs to do these things, I believe we'll make the task of implementing these three traits nearly trivial.

EvolvableSandbox and DevolvableSandbox

These traits represent state transitions, so any implementation thereof represents an action of moving from one sandbox to the next. Generally speaking, this doesn't involve a lot of work, but it may involve taking memory snapshots and/or changing the set of registered host functions (e.g. functions the guest can call on the host) available to guests.

I expect if we implement utilities for the previous section, we will have most of what we need to implement utilities for this one.

Final notes

Those familiar with Hyperlight's C# codebase will notice this proposal represents the conceptually-large, but practically-small, task of breaking up the monolithic Sandbox C# class into utility functions as we rebuild them in Rust, and ending up with no single Sandbox implementation. In fact, this document proposes we end up with no single Sandbox-like "entrypoint" to Hyperlight whatsoever.

This design would enable a kind of inversion-of-control between Hyperlight-Core and its libraries like Hyperlight-WASM. If this proposal is implemented, library authors, like us as we build Hyperlight-WASM (after we rewrite it in Rust), would use the aforementioned "building blocks" to build a set of Transition and Sandbox implementations, then use those to implement a single HyerlightWasm.Sandbox class in that library.

I believe this design in Hyperlight Core can safely scale to a wide, diverse set of libraries and can accommodate arbitrary design changes within any one of them.