# Bevy Editor-UI research

## Target

https://github.com/bevyengine/bevy/issues/254

Before we can start work on the Bevy Editor, we need a solid UI implementation. Bevy UI already has nice "flexbox" layout, and we already have a first stab at buttons and interaction events. But Bevy UI still needs a lot more experimentation if we're going to find the "right" patterns and paradigms. Editor-Ready UI has the following requirements:

- Embraces the Bevy architecture: Bevy ECS, Bevy Scenes, Bevy Assets, Bevy Events
- A Canvas-style API for drawing widgets with shapes and anti-aliased curves
- Define a consistent way to implement widgets
- A core set of widgets: buttons, inputs, resizable panels, etc
- Theme-ability
- "Interaction" and "focus" events
- Translation-friendly. We can't be anglo-centric here

Additional "target" quotes from Discord:
> @cart It must be both modular (not just a bunch of hard coded functionality), well integrated into Bevy ECS, and express-able via Bevy Scenes. Bevy UI will be used for the editor, but what Bevy UI looks like (and if we base it on something) is still up for debate

> @KamBam For bevy’s editor, would it likely include a text editor like Godot?
> @cart It's a pretty low priority. Doesn't seem worth the investment. We should be building game engines, not text editors

## Concerns

### ECS & Functional Reactive Programming
As Cart has stated multiple times any UI solution should feel bevy native, therefore using ECS

> is the idea that Bevy UI is built using the ECS i.e. UI elements are entities, or merely that Bevy UI is optimised specifically for interacting with the ECS
mem
> @cart yup both!

However in the web world there is some degree of concensus and state of the art emerging regarding functional and reactive programing, perhaps best popularised by React, but also others like the Elm framework and Cycle js (where the latter are more "pure" functionally). These ideas also show merit for Native development, as shown by React Native and Flutter.

It's not immediately clear how FRP and ECS could live together

> james.i.h
> i think you could do an elm style thing with ECS, components are basically just data models, you can setup separate systems to handle rendering ("views") and updates, the main disconnect is ecs has no built in concept of a messaging system, but bevy has events, which probably covers that

https://raphlinus.github.io/personal/2018/05/08/ecs-ui.html

### Immediate Mode (imgui) vs Retained Mode

Many discord users extoll the virute of imgui (eg `Dear imgui` aka `imgui-rs` or `egui`). However no imgui solution is likely to the whole story.

> @Cart yeah we are definitely shooting for a "retained" api, but "immediate" also has value. it would be interesting to use immediate mode drawing with a retained wrapper around it

>@StarToaster I think the hardest part is fitting imgui or whatever into ecs.
>@cartYesterday and layout

There has been some appetite for people developing imgui tests as standalone plugins. It's true that Imgui remains very popular, as it allows one to get going very quickly. However see the Unity section for concerns about the long term viablity of imgui.

However performance seems to be a solved problem with more modern IMGUI implementations
> OvermindDL1 they build a full scene in the background and as you submit the calls it just walks down it comparing the commands, and if it doesn't match then it updates it. It's extremely fast and gives you both the speed of cached UI's while being extremely dynamic in call styles

### Layout
The layout engine is key component to any UI solution. Currently Bevy uses flexboxed `stretch`, however see the `iced` section for concerns. There is a question whether we use something more custom like flutter does, or going for something more standards based, like flexbox.

### Styling
It's true that users need to be able to customise widget (theming in the original target description). There's not been much discussion yet of what this might look like in Bevy, but to separate data and presentation is long estabilished wisdom. Obvious example being CSS.

### Rendering
Canvas style api for drawing widgets with shapes and anti-aliasing.

Concerns here probably include font rendering, and diffing to redraw only dirty elements.

Seems there are multiple bedrock options for this:
> @cart thttps://github.com/bevyengine/bevy/issues/90
> We're currently considering lyon, pathfinder, and piet as potential providers of this functionality

### Headless Editing

Cart has expressed desire to make it possible to hand editor scene files.  

> @cart a stretch goal for the editor design is for aspects of it to be embeddable in games

There has also been some opinion on whether Bevy games should be possible to be developed entirely outside an editor
https://github.com/WesterWest/bevy-editor/issues/1

## Evaluations & Prior Art

### egui

Rusty imgui library

### imgui-rs

Rust bindings for DearImgui. Dear Imgui is a very popular C++ imgui library, but Bevy has strong preference for Rust native crates

### Druid
> A data-first Rust-native UI toolkit.

https://github.com/linebender/druid

Created by Raph Levien, spun out of the now defunct xi edtior as a way of doing cross platform GUI. Raph's blog is also some of the most authoritive sources on what it takes to build GUI in Rust. See:
https://raphlinus.github.io/rust/druid/2019/10/31/rust-2020.html

Druid seems to heavily inspired by Flutter. It's not clear how ready Druid is as a complete framework, most of the work seems to be going into piet - a GPU accellerated rendering foundation.

The druid Zulip instance also hosts rust GUI chat
https://xi.zulipchat.com/

### OrbTk
> The Orbital Widget Toolkit is a cross-platform (G)UI toolkit for building scalable user interfaces with the programming language Rust. It's based on the Entity Component System Pattern and provides a functional Reactive-like API.

https://github.com/redox-os/orbtk

### Iced
> A cross-platform GUI library for Rust focused on simplicity and type-safety. Inspired by Elm.

(Created by discord User @lone_scientist)
https://github.com/hecrj/iced

#### Lessons

> Iced dropped `stretch` (Editors note: the same lib used by Bevy) in favor of a custom layout engine a while back: https://github.com/hecrj/iced/pull/52 it might be helpful and interesting to read about why they took that approach.
>
This was replaced by layout methodology taken from druid, as it seems the flexbox model create perf concerns.

It's important to keep in mind that iced used an older version of stretch which could have been the source of the performance issues.

### Flutter (Dart)

Flutter is a relatively modern cross platform native UI framework created by Google, written in Dart. And seemingly inspired by React.

It's relatively modernity means that they've been able to incorportate some of the latest thinking in UI. Also, google scale it's a good reference point on how to implement an entire frame work from scracth.

Native mobile doesn't map perfectly to "games" (eg most games don't care about having platform native looking ui), but games **DO** care about platform native UI performance.

Seems like Flutter is strong inspirtation for Druid

https://flutter.dev/docs/development/ui/layout

### Stretch

### Unity (C#)

Unity's first GUI was a bespoke Unity authored `imgui` (referred to as `OnGUI` for disambiguation from other `imgui` libraries) for both editor and runtime.

At runtime this was a major source of frustration for many developers. The `OnGUI` draw the world often had poor performance beyond even marginal complexity. Styling was based on serialized `GUIStyle` assets that would be passed into `OnGUI` functions - however creating unique looking UI (important in games) or bespoke functionality was difficult.

For the Unity editor, consistency is important, and performance is less of a concern. These concepts map well to imgui and indeed much of Unity's editor today is still powered by `OnGUI`. However it's worth noting Unity has needed to add conventions to facilitate so more stateful operations, namely serialization (persisting state) and undo/redo (rewinding state). These concepts layer additional complexity for the end user trying to write successful UI editor code - imgui often isn't enough to answer the whole story by itself.

One of the most common Editor use cases is to embellish default editor behavior - in other words you don't want to replace what the editor is doing, but just add a bit extra for efficiency. Generally achieving this with custom editor code involved a fair degree of boilerplate. However it remains true that Unity's editor extensibility is one of it's selling points.

Later based on the limitations of `OnGUI`, Unity created a bespoke retained mode UI (`UGUI` or UnityGUI) for runtime. This was much closer to a traditional OOP display graph GUI as seen in Flash/.NET/Java, with class based widgets storing state. As an end user, the biggest win is to be able to edit UI in the editor - as `OnGUI` was runtime only. This was a huge win for customization. Modification and custom elements also become easier, as now there was an API to create your own widgets. It was now also for the first time possible to UI on 3d canvases in the world.

The library also did a god job of separating UI capturing input into separate modules. In general this concept has survived well - as Mouse Pointers, Touch devices, and VR devices have all been evolutionarily supported by swapping out modules without needing to modify UI code.

Less successful was the layout routines. The esoteric layout strategy (designed to ride on top of Unity's existing transform hierarchy) historically been very difficult to get dynamic UIs to work well and layout how the author expects - a constant source of pain. And even though `UGUI` was an order of magnitude perf gain over `OnGUI` it still struggled with "typical" UI perf bottlenecks, eg drawing large lists or dynamically modifying content.

More recently Unity has created `UIElements`: a unity flavoured html+css like approach based on flexbox (under the hood I believe this uses Yoga - React Natives layout model). It's still too early to evaluate the success of this, but the long term goal is to have this be the blessed path on both Runtime and the editor. Most new editor features are built on this new framework.

As a developer this offers many wins. The markup approach means UI can be hot reloaded at runtime, however it can still be dynamically generated from code (the mark up elements are backed by class based components). So far this approach has been well received by the community. It's worth noting none of the Unity frameworks are opinionated in the in how state should flow through the application. This is mostly up to the end user.

## General Comments

@msiglreith:

> Regarding stretch, I moved away from flexbox as it's powerful but also kinda hard to predict (might be also due to my lack of experience back then with UI layouting in general). So I went further to a variant of druid layout system, which is also adopted by iced. It's quite nice and fits well into a multi-pass UI approach. My goal for an UI was to build tools and be useable in games as well, so easy to customize, integrate into the game world and also quick to setup without building the rest of the application around it. druid is on one end of the spectrum by being completely retained. I moved towards a custom approach where the passes could be either retained or immediate (logic/state non retained, only layout data) and then to fully immediate, basically dropping the layout system completely.
> Overall, the user group is important (e.g automatic vs manual layouting?) and the application while working with the constraints given by Rust.

@ncallaway:

> We should take into account device-scaling from the early stages of UI development. `bevy` already has an issue with DPI scaling on different devices (https://github.com/bevyengine/bevy/issues/195). `bevy_ui` should set a good foundation in the core of `bevy_ui` for device independence, with defaults that work across a wide arrange of devices.

> As part of this, we should take a thoughtful approach as to what units we want to present in `bevy_ui`. I would strongly encourage `bevy_ui` to default to logical pixel of some kind (in the same way CSS defines a pixel as 1/96 in), while leaving physical pixels as an available unit.