# Makepad Architecture Benefits

## 4 major design choices
Makepad’s architecture highlights can be summarized in 4 major designs choice

1. Live designable/codeable UI DSL for all styling and structure co-developed with a realtime IDE/visual designtool. It is Delphi/Visual Basic but for Rust.
2. Immediate mode drawing flow to draw all 2D shapes and output directly to the GPU’s instanced drawing APIs (https://www.youtube.com/watch?v=Ude1zZbf20s) keeping CPU cost very low. No vector processing needed.
3. Shader styling language integrated in UI DSL to style and animate widgets with simple familiar canvas-like API running on GPU
4. Support a mix of immediate mode and retained mode in terms of UI state management so you can choose the optimal form for your problem. Immediate/Retained mixing enables complex widgets (editors, infinite scrollviews, graphs)

---

### 1. Live designable/codeable UI DSL for all styling and structure co-developed with a realtime IDE/visual designtool. Think delphi/visual basic for Rust.

* Use the 'live' connection of makepad studio or write it by hand in vscode to update the application in real-time. Real time live update speeds to iterate the UI design with either a visual designtool (WIP) or code editor

* Updateable in realtime on-device, running android/ios phones or web or even apple tvs can be live updated

* Visual designtool familiar to Figma users is being developed

* Makepad has solved major design to code challenges. Makepad Live Design DSL can be parsed and live reloaded in realtime (<10ms) much faster than JS

* Makepad Live Design DSL are similar to HTML/CSS concepts so the designers have no problems to adopt it in most use cases.

* Makepad Live Design DSL provides the HTML/CSS like set/get APIs so the application logics can be implemented in separate code. Designers and programmers can work on the same code base simultaneously.

* Makepad Studio provides Figma like user experience so no need to do the design conversion, the design is the code.

* The UI structure implements prototypical UI inheritance, very similar to the Figma 'Component' model where you clone a component and specialise the styling

* A fast idea to implementation cycle is the most important part of the front-end app design (see Bret Victor’s The Future of the Programming, https://www.youtube.com/watch?v=8pTEmbeENF4 ). Thus all design tools, IDEs and application frameworks are focused on the reducing the latency of this feedback loop. Makepad studio has a unique angle for this with cross-process-textures for smoothly recompiling applications inside the IDE, and livecoding the UI styling

### 2. Immediate mode drawing API that internally uses instanced drawing on the GPU

* Makepad has an immediate mode drawing API for drawing 2D shapes that internally uses instanced drawing on the GPU. This enables the user to draw a large number of 2D shapes with very little effort, while still keeping CPU costs very low.

* An more in depth explanation of instanced drawing on the GPU can be found here:
https://www.youtube.com/watch?v=Ude1zZbf20s

* Using instanced drawing on the GPU is a very cross platform compatible solution: instanced drawing has been available in graphics APIs since WebGL 1.0 and GLES 2.0. It is likewise available in baseline graphics APIs such as OpenGL, DirectX, Metal, Vulkan, etc. We don't require new GPU functionality such as GPGPU which is not available everywhere yet.

* Because of this, Makepad is currently compatible with Wasm on the Web, Macos, Linux, Windows, Android, iOS, TvOS, and soon VisionOS.

* Widgets that are drawn using the same shader are grouped into the same drawcall. This minimises the number of draw calls that need to be issued by the CPU, which is often a limiting factor in how quickly the GPU can draw.

* Every instance of a widget is mapped to an element in an instanced array. This allows for direct manipulation of the instanced array to modify a specific instance of a widget with very little overhead.

* For certain operatons, such hovering over a button or hyperlink, direct manipulation of the instanced array drastically reduces the CPU time required to rebuild the UI in response to an event, compared to other immediate mode UI solutions such as Dear Imgui/Egui.

### 3. Live editable shaders for real time styling and animation

* Makepad uses live editable shaders for real time styling and animation. Live editable shaders fulfill most of the styling and animation needs of a typical UI, for a fraction of the complexity cost of solutions such as CSS and SVG.

* Our contention is that CSS and SVG, while more general, are massive overkill for the majority of use cases in UI design, and not worth the complexity of their implementation.

* Most things that are possible in CSS and SVG can be expressed in shader code with relatively little effort. Examples are gradients, shadows, animations, etc.

* Dynamic shader programming can be used for fast animations, such as mouse hover event triggering button color changes. This is an extremely efficient way to animate large number of instances of the same widget.

* In general, the shader system is extremely flexible: any shader that you can find on Shadertoy can be run inside a Makepad widget.


* The shader APIs provided by Makepad greatly simplify the writing of custom shaders, similar to writing Canvas or SVG code, except programmable on a per pixel basis. Animating and changing styles based on user input is also much simpler compared to manipulating vector APIs.

* Because shaders are written with imperative code, the user has much more control over how things are styled an animated than with declarative code, since the latter only allows the user to specify what the language designers thought of up front.

* Shader code runs almost directly on the GPU. This gives substantial performance benefits over CSS and SVG, both of which require complex transformation steps before the end result can be rendered on the GPU.

* Moreover, because Makepad makes very few assumptions about what graphics APIs are available, cross platform compatibility is much less of a concern than for vector APIs, which often rely on complex APIS (i.e. GPGPU) which are not available on every platform.

* The use of shader code also enables full styleability in a 3D context and will enable unique styling in an AR/VR context as well (prototype available).

### 4. Mixed immediate/retained mode APIs

* There is no one solution to ui: immediate or retained mode. Makepad recognises this, and therefore supports both paradigms, enabling us to pick the solution that is best suited for a particular problem.

* Retained mode components can be used statically, as a pure design document, and this is the principle on which the visual designtooling is based: live updating of the declarative retained UI structure.

* It is possible to use retained mode subcomponents within the immediate mode API. This allows for a mixed approach where many retained mode subcomponents are spawned in immediate mode. This feature is used in virtual viewport scroll view applications such as Robrix, Moxin or Makepad Studio's log view.

* The use of mixed immediate/retained mode allows Makepad to efficiently update its GPU data structures when things need to update. As opposed to many pure immediate mode GUI's or some React clones that need to regenerate everything, Makepad can independently update a particular widget very efficiently. Inside the IDE for instance when a code editor updates only that piece of the GPU data related to the code editor updates.

* In Makepad the immediate mode API is the lowest level API. The retained mode is built on top of this. This means that each widget draws itself in immediate mode. This means that the layout pass has to happen in immediate mode as well.

* Layout happens at the same time as drawing. There is no separate 'measure' pass, as done in Flutter. The reason this works is that Makepad has the ability to 'move' items around in the generated GPU data after it is generated. For example: Right aligned items are first drawn on the left. They are generated and measured. Once their size is known, they are moved in the GPU data to the right.

--------------
-------------

* High performance UI app like AR/VR, computer vision, large volume real-time data rendering, fast infinite list scrolling needs immediate drawing, meaning every UI element’s position, layout, color, special effects will be changed on frame-by-frame basis. Basically, game like apps.

* Retained mode works best with apps with more static UI such as text editor, e-commerce page, web page, where only small portion of the UI is changed from frame to frame thus most of the UI elements states are intact which means a large portion of the UI does not need to be re-draw to save CPU/GPU cycles. This is especially important for the mobile SoC where the CPU voltage and frequency are scaled up and down with the workload.

* Most of the UI frameworks stay with one state management architecture, either retained mode or immediate mode. Specifically, the Web frontend frameworks architectures, i.e. React, Vue, implemented a sophisticated UI state management or so-called Virtual DOM above the HTML DOM layer because HTML/CSS was designed to render the static text, image, video contents. Highly interactive animated web UI usually requires direct manipulating the WebGL which totally bypass the HTML/CSS DOM i.e. three.js which is a totally difference ecosystem. Inspired by web front end frameworks’ innovations, e.g. virtual DOM, immutable one-way dataflow (Redux), state reconciliation, many mobile declarative UI frameworks also choose to build a separate UI layer abstraction to specifically manage the retained state, i.e. Flutter. The key design choice of the retained mode architecture are tree structure(s) to represent the high level UI elements and layout, the state association with the UI elements, UI state reconciliation due to events, bridging the high level UI tree abstractions to the widget/drawing layer.

* There are few problems with this approach. In the web front end architecture, there are stable and fixed APIs with the underline HTML DOM which makes the VDOM and DOM to work and optimize separately. For example, why building VDOM if the DOM layer can implement the state reconciliation algorithm. (https://www.linkedin.com/pulse/virtual-dom-vs-real-reconciliation-algorithm-react-hanan-fadel-tnede/)

* This is not the case for the mobile where the entire low-level widget, drawing layers are open to the framework developers. There is no need to mimic the web frontend’s architecture.

* Immediate mode UI are widely used with the game engine where there is also a need to implement the user interface such as menus. For example, egui was initially designed to work with game engine and later on pivot to the robotics and AI application for the real-time data visualization. In the immediate mode architecture since everything in the UI has to be re-drawn per frame, there is no need to have a separate UI state layer because there is nothing to be retained. The UI element’s size and layout calculating and drawing are happening at the same time but must be finished with one frame period, i.e. 16ms in 60fps. This could be a lot of overdo for the static UIs and can be power consuming too which limits its use as a mobile UI framework. Also for some complicated UI layout like nested boxes where parent’s size and layout info are dependent on the children’s, the immediate mode UI could miscalculate the parent element’s position and size because the children element’s position and size were not available when the parents were being drawn.

* The real-world application can have mixed requirements for both immediate mode and retained mode. For example, in the mobile, the infinite scrolling list are widely used for any catalog applications, i.e. eCommerce. ~~The retained mode UI will need to find a way to bypass the state management to meet the 60fps or 120fps scrolling requirements while for the immediate mode, this is no extra efforts needed to support it.~~ Another example is real-time video and audio applications, there are portion of UI need to be drawn at the audio and video sampling rate while the control part of the UIs is pretty much static.

* Consider the case of infinite scrolling. When using a retained mode UI, it's challenging to limit the drawing effort to just the visible part of the list. This is because every list item, whether currently visible or not, is represented in the virtual DOM or a similar retained structure. If data changes occur for items that aren't currently visible, it's difficult to avoid wasting processing power on drawing items outside the viewport. This could hinder efforts to achieve a 60fps or 120fps scrolling rate.
The immediate mode, on the other hand, explicitly defines what to draw at any given time, making it easier to implement in many cases. It can also handle scenarios where keeping updates are needed for no longer visible items. For instance, if a user types input into an item that's no longer visible and then scrolls up, it's expected that the typed input will persist. The extent of this behavior can be limited to a fixed number of non-visible items to balance performance and functionality.

* Makepad provides mix and match of the immediate mode UI and retained mode UI in one UI page. The immediate mode parts are rendered every frame. The developer needs to provide the absolute size and position info for each immediate mode UI element for each drawing. The retained mode part of UI is like the popular declarative UI framework where each widget can be defined with relative position and size and can be nested in many levels. The framework will automatically calculate the new size and layout based on the event changes. Such design give the developer the maximum freedom to balance the UI performance and the computing efficiencies.

### Live design and DOM like APIs
* It is hard for the high-performance languages to do live reload because of the extra compile time. Swift, Java vs Javascript. Javascript is much easier to do fast live reloading. DART was created on top of Flutter engine mainly to compete with Javascript on hot reloading. It smartly designed development stage and production stage. In the development stage, DART provides a JIT mode which compiles faster but less performant to support live reloading-based development. While for the product stage, it supports AOT mode to provide the maximum performance but no hot reloading. SwiftUI and
* Jetpack Compose are slowed in hot reloading compared with JS.* The so-called declarative design pattern makes the hot reloading useless. The main user of the hot reloading is the UI designer, not the engineer. But React and React-like app frameworks, i.e. SwiftUI, Jetpack Compose, Flutter, has mingled the application logic with the UI code which makes impossible for the two teams working on the same code base.
* The HTML/CSS DOM design expose the set/get APIs to the application logics which clearly separate the UI design and application design flow. The HTML/CSS is the golden standard for the app prototype design because all the designers have HTML/CSS background. The design tools like Figma are very HTML/CSS centric and the IDE has its own UI lingo which does not always match with each other. Figma can output lossless HTML/CSS files but not so much for the app UI frameworks.
* In reality, the designer adopted the design centric tools such as Sketch, Figma and few designers can directly work with SwiftUI or Jetpack Compose for the design prototype. There are awkward process often involving manual process to convert the design tool outputs to the SwiftUI, Jetpack Compmose, react application for many reasons.