# Autosave & Autosync

Table of contents
-
[TOC]

## Know Solutions
- [OTs `Operational transformation`](https://en.wikipedia.org/wiki/Operational_transformation): Good for text based apps
- [CRDTs `Conflict-free Replicated Data Types`](https://en.wikipedia.org/wiki/Conflict-free_replicated_data_type): Designed for decentralized systems
- [Figma](https://www.figma.com/blog/how-figmas-multiplayer-technology-works/): custom Simplified CRDTs for centralized systems

## Chalanges
### Frontend Side
- [Offline Editing](#Disconnnected-from-the-Internet)
- Sync Controls (Property vs Object) [[1]](#Change-Control-properity)
- [Flickering Problem](#Flickering-Problem)
- Object Creation and Removel [[1]](#Add-Control), [[2]](#Remove-Control)
- [Reparent](#Change-Control-Parent-(Reparent))
- [Reorder](#Change-Control-Order)
- [Undo/Redo](#UndoRedo-Changes) (during editing not revision histoy)

### Backend Side [TBD]
- Auto save
- Duplicated Changes in offline editing
- Revision History
- Revision History Cleanup
- Publish Form
- Database & Cache Model

## Sequences and workflows
- Init Form Editing
- Add Control
- Remove Control
- Change Control properity
- Change Control/Section Parent (Reparent)
- Change Control Order
- Undo/Redo Changes
- Disconnnected from the Internet

### Init Form Editing
1. Client (A) Requests the latest version of the form from the server
2. Client (A) Open a webSocket(SignalR) Channel with the server to sync and track changes
3. Client Init Empty Undo/Redo Buffers

### Add Control
#### ❌ None-Valid Control
1. Client (A) Generates new `ControlId`, using `Client Session Key as a seed` to make sure Id in unique.
2. Client (A) Sends the JSON for the new field over websockets to the server.
3. Client Applays the new control in the designer with a pending state and wait for server confirmation.
4. Server validate the new control properities `ControlId`, `ParentId`, `Order` || `❌`.
5. Server Sends a message over websocket to Client (A) only to reject the Control with the reason, Such as:
- P1: Parent (Page, Section) not Found (another clint Remove the parent)
- P2: Order Conflict (Server may solve the conflict and send the new order)
- P3: Duplicated `ControlId`
- ...?
6. Client (A) Try to resolve the conflict
- P1: Add Control to a temperory parent (ex: unused hidden Page) and notify the user to locate the control || `Just an idea`
- P2: update order property and fix visual order if required
- P3: If the Id is generated using `Client Session Key as a seed`, This problem means that control has been added without confirmation ?!
- ...?
> ⚠️ [After thinking It is just a bad idea - Ignore]
> ~~Idea [not sure if good]: Consider using sections as a normal field like in Jotform with additional Section-break field~~
> - ~~Field has no reference to the parent in the form JSON~~
> - ~~Section Information could be added to the field asnwers on submit~~

#### ✔ Valid Control
1. Client (A) Generates new `ControlId`, using `Client Session Key as a seed` to make sure Id in unique.
2. Client (A) Sends the JSON for the new field over websockets to the server.
3. Client Applays the new control in the designer with a pending state and wait for server confirmation.
4. Server validate the new control properities `ControlId`, `ParentId`, `Order` || `✔`.
5. Server Sends a message over websocket to All Clients to confirm the new added control.
6. Client (A) Changes Control Status from pending to done.
7. Clients (B, C, D, ...) Recieve the new control and add it to the form.

### Remove Control
1. Client (A) Sends Remove Control command over websockets to the server.
2. Client Hides the control in the designer with a pending state and wait for server confirmation.
3. Server validate the control exists and could be removed
- `❌` Rejected: not sure yet what could cuase this
- ✅ Valid: Just remove the control and send request
- `❗` Not Found: Just confirm and ignore the request
4. Server Sends a message over websocket to All Clients to confirm the removal of the control.
5. Clients Complete the removal of the control.

### Change Control properity
> We will sync the changes on property level not page nor control
> each property change will be sent to the server to sync and save

> The Server has no conflict problem last change recieved will be applied `last-write-win`

> The Client has to solve some problems like Flickering due to `last-write-win`

#### Initial Setup:
- We have a label with color `blue 🟦`
- Two Clients A,B are editing the same form and connected with the server over websockets
#### Normal Operation (no-conflict)
1. User over Client (A) changes control property `Label Color` to `red 🟥`
2. Client (A) changes the label visual color in the builder to `red 🟥` and set the change status to pending
3. Client (A) sends a request to the server to change `Label Color`
4. Server Recieve Client (A) Request and approve it.
5. Server send the change to the Clients (A, B)
6. Client (B) applay the change.
7. Client (B) `label color` = `red 🟥`
8. Client (B) confirm the change.
9. Client (A) `label color` = `red 🟥`

#### Flickering Problem

- [00:00:01]
1. User over Client (A) changes control property `Label Color` to `green 🟩`
2. Client (A) changes the label visual color in the builder to `green 🟩` and set the change status to pending
3. Client (A) sends a request to the server to change `Label Color`
- [00:00:01] at the same moment
1. User over Client (B) changes control property `Label Color` to `yellow 🟨`
2. Client (B) changes the label visual color in the builder to `yellow 🟨` and set the change status to pending
3. Client (B) sends a request to the server to change `Label Color`
- [00:00:02]
1. Server Recieve Client (B) Request and approve it.
2. Server send the change to the Clients (A, B)
- [00:00:03]
1. Client (B) confirm the change.
2. Client (B) `label color` = `yellow 🟨`
3. Client (A) does not applay the change as it affects a pending change, ...! wait until server approve or reject my changes.
4. Client (A) `label color` = `green 🟩`
- [00:00:04]
1. Server Recieve Client (A) Request and approve it.
2. Server send the change to the Clients (A, B)
- [00:00:05]
1. Client (B) applay the change.
2. Client (B) `label color` = `green 🟩`
1. Client (B) confirm the change.
4. Client (A) `label color` = `green 🟩`

### Change Control Parent (Reparent)
Reparent Operation consist of two operations as we > drag the control to specific position in the new parent (Section - Page)
- Change parent `Set new ParentId`
- Change Order

> ⚠️ Change section parent (page) will change order of this section and its children
> We need to consider `local order inside the group`
> The control order in pages are independant from each other
> The control order in sections are independant from each other

> We has no cycling issue as the levels are static
> `Page > [Control | (Section > Control)]`

> ⚠️ [After thinking It is just a bad idea - Ignore]
> ~~`Order Based Open-Close Grouping:`~~
> ➠ ~~There is no direct relation between children or perants during building the form~~
> ➠ ~~Section will has two orders `StartOrder`, `EndOrder` like it is a two seperate controls~~
> ➠ ~~We could make this hidden in the builder just logical representaion~~
> ➠ ~~We also could present the section closing as a locked control, or draggable locked control~~
> ➠ ~~The same concept is going to be applaied to the pages~~
> ➠ ~~This Solution is good for the flat representaion of the form controls~~

#### Initial Setup:
- We have two pages (P1, P2)
- We have two sections (S1, S2) in the P1
- We have a single TextBox Control in P2
- Two Clients A,B are editing the same form and connected with the server over websockets

#### Reparent Section
1. User over Client (A) changes section `S1` parent to `P2` and set it in the `2nd position` after the Textbox
2. Client (A) calculte the new order of the section as being explained later
3. Client (A) render the result and set the changes to as pending changes
5. Client (A) sends a request to the server to reparent the section specifing `NewParentId`, `NewOrder`
6. Server Recieve Client (A) Request and approve it.
7. Server send the change to the Clients (A, B)
8. Client (A, B) applay the change.

#### Reparent Control
The same as reparent section considering the local order solution for section reparent problem

### Change Control Order
Changing the control order require to change the structure of the collection holding the controls, or changing the order property of all controls...
That will result in many changes, and will cause many conflicts.

We should consider the solution `Figma` uses `Fractional Indexing`
- At a high level, an object’s position in its parent’s array of children is represented as a fraction between 0 and 1 exclusive.
- The order of an object’s children is determined by sorting them by their positions.
- You can insert an object between two other objects by setting its position to the average of the positions of the two other objects.


- This solution cauld has a precision proplem.
- `Figma` use [arbitrary-precision fractions](https://en.wikipedia.org/wiki/Arbitrary-precision_arithmetic) instead of 64-bit doubles so that we can’t run out of precision after lots of edits.
- For compactness, `Figma` omit the leading “0.” from the fraction and we use the entire ASCII range instead of just the numbers 0–9 (base 95 instead of base 10).

#### Example by Numbers (Base 10 - omitting "0.")
- To add the first control (c1) we need to added with
- order = avarage between 0,10 = 5 || `0.5`
- To Add control (c2) after (c1)
- order = avarage between 5,10 = 7.5 || `0.75`
- To Add control (c3) before (c1)
- order = avarage between 0,5 = 2.5 || `0.25`

#### References
- [Base95](http://ftr.icerealm.org/numeric#base95)
- [JS-BigInt](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/BigInt) supports up to Base36
- [Arbitrary-precision arithmetic](https://www.youtube.com/playlist?list=PLGI5yUFVsUkWO26oPixUNYwKick1LxiWR) - Youtube Tutorial in Python

### Undo/Redo Changes
Undo in a multiplayer environment is inherently confusing, If other people have edited the same objects that you edited and then undo, what should happen?
- Should we undo local changes only?
- Should we undo other people changes?
- Is it realy a feature we need ⁉️

The most user frindly approch should **`Undo`** local changes only but we need to take others changes into account when perform a **`Redo`** operation.

#### Undo/Redo Buffers Management
- We have two buffers **Undo Buffer (UB)** and **Redo Buffer (RB)**
- When the form is initialezed the `UndoB` and `RedoB` will be empty
- When any changes recived from other pepole the buffers (UndoB, RedoB) remain the same with no change, although the form state will be updated
- When the user perform any change the current state to new state `S1 -> S2` will be pushed to the `UndoB`, `S1` includes others changes
- When the user `Undo` the last change `S1 -> S2` in the `UndoB` will be removed and its previous state will be applayed, the current state to previous state`S2* <- S1` will be pushed to `RedoB`, `S2*` is `S2` including others changes
- The same with redo, Instead of moving the last state between buffers we will push the a recent updated state including others changes

#### Example

##### Initial Setup:
- We have a label with color `red 🟥`
- Two Clients A,B are editing the same form and connected with the server over websockets
- Client (A, B) has a `red 🟥` label stored in `Undo Buffer`

##### Example sequence and steps
1. Client (A) change the label color to `blue 🟦`
2. Client (A) push label color change `red 🟥 -> blue 🟦` in `Undo Buffer`
3. Client (B) applay the change without changeing the buffers
4. Client (B) change the label color to `yellow 🟨`
5. Client (B) push label color change `blue 🟦 -> yellow 🟨` in `Undo Buffer`
6. Client (A) applay the change without changeing the
7. Client (B) perform `Undo` operation
8. Client (B) remove label color change `blue 🟦 -> yellow 🟨` from `Undo Buffer`
9. Client (B) push label color change `yellow 🟨 <- blue 🟦` to `Redo Buffer`
10. Client (A) applay the change with no change in buffers
11. Client (B) perform `Redo` operation
12. Client (B) remove label color change `yellow 🟨 <- blue 🟦` from `Redo Buffer`
13. Client (B) push label color change `blue 🟦 -> yellow 🟨` to `Undo Buffer`
14. Client (A) applay the change with no change in buffers
15. Client (A) perform `Undo` operation
16. Client (B) remove label color change `red 🟥 -> blue 🟦` from `Undo Buffer`
17. Client (A) push label color change `yellow 🟨 <- red 🟥` to `Redo Buffer`
18. Client (B) applay the change with no change in buffers
19. Client (A) perform `Redo` operation
20. Client (A) remove label color change `yellow 🟨 <- red 🟥` from `Redo Buffer`
21. Client (A) push label color change `red 🟥 -> yellow 🟨` to `Undo Buffer`
22. Client (B) applay the change with no change in buffers

### Disconnnected from the Internet
- User could go offline and continue editing with changes stored in th browser
- When Client comes back online:
- download a fresh copy of the document
- Reapplies any offline changes on top and send to the server for confiration one by one
- continue syncing updates over WebSockets
- Client should create a snapshot with the offline changes in undo buffer if the user wants to undo others changes and keeps his changes only

---

⚡ ✅ ❌ ❗ ✔ ⁉️ ⚠️

🔴🟠🟡🟢🔵🟣🟤⚫⚪🔘🛑

🟥🟧🟨🟩🟦🟪🟫⬛⬜🔲🔳

❤️🧡💛💚💜💙🤎🖤🤍

🔺🔻🔷🔶🔹🔸