Client
======
#### Implementation
- We have renamed the property Value to Content, in both Normal and RT whiteboards. In some places there're still remaining `value`s, like `ValueContainer` or `loadingValue`... _Rename those to Content_.
- We have separated collaborative whiteboards implementation from the normal whiteboards. The separation has been done at CalloutView level, all the children under them is duplicated & adapted to RT code.
| Original file | Rt File |
| --------------------------------- | ---------------------------------- |
| SingleWhiteboardCallout | SingleWhiteboardRtCallout |
| WhiteboardProvider | WhiteboardRtProvider |
| WhiteboardsManagementViewWrapper | WhiteboardsRtManagementViewWrapper |
| WhiteboardActionsContainer | WhiteboardRtActionsContainer |
| WhiteboardManagementView | WhiteboardRtManagementView |
| Whiteboard**Content**Container | WhiteboardRtValueContainer |
| WhiteboardRtDialog | WhiteboardRtDialog |
| ExcalidrawWrapper | CollaborativeExcalidrawWrapper |
_Unifying all of this would be good_
- We have (copy&pasted / adapted) some code from excalidraw-app into the client
- Everything is under `src/domain/common/whiteboard/excalidraw/collab`
- Some things are copied there because VITE doesn't "see" them, cannot import them from the @alkemio/excalidraw package, but VS.Code compiles the code perfectly, it's just when serving them VITE shows an error in the browser saying that it cannot find them. _Investigate this VITE/excalidraw package issue importing_
- Collab class is an oldstyle React class no hooks, the state is handled with jotai atoms, _Refactor/rewrite this class and remove the dependency from jotai_
- Some code can be removed
- Implications of 16.1
#### Whiteboards vs WhiteboardRTs / starting collaboration.
The way of loading the original content of the whiteboard is very similar, we have a container `WhiteboardRtProvider` and a `WhiteboardRtValueContainer` that loads the JSON from the GraphQL API. We have removed all the logic about checking-in and out the whiteboardRTs and when the whiteboard loads the `Collab.startCollaboration` function is called.
`startCollaboration` connects the websocket (Socket.IO) link to the server, which is on the same server url `env.VITE_APP_ALKEMIO_DOMAIN + '/api/private/ws/socket.io'`,
There's no way (for the moment) to turn a whiteboard into a whiteboardRt or vice-versa, other than saving the json content and loading it into a new callout manually.
On starting collaboration the client connects the websocket
........ Algorithm and messages
- first-in-room and not first-in-room
_Pending analyze and expose here the full algorithm_
The `whiteboard.id` is the Socket.IO `roomId`
#### Image sharing
Whiteboard JSONs contain the `elements` array and the `files` object.
The `files` object keys are the SHA1 of the content of the files (_confirm this_):
```typescript=
type BinaryFiles = Record<string, BinaryFileData>
type FileId = string;
export type BinaryFileData = {
mimeType: ValueOf<typeof IMAGE_MIME_TYPES> | typeof MIME_TYPES.binary;
id: FileId;
dataURL: DataURL;
created: number; // Epoch timestamp in milliseconds
/**
* Indicates when the file was last retrieved from storage to be loaded
* onto the scene. We use this flag to determine whether to delete unused
* files from storage.
*
* Epoch timestamp in milliseconds.
*/
lastRetrieved?: number;
};
export type ExcalidrawImageElement = _ExcalidrawElementBase & Readonly<{
type: "image";
fileId: FileId | null;
status: "pending" | "saved" | "error"; // whether respective file is persisted
scale: [number, number]; // X and Y scale factors <-1, 1>, used for image axis flipping
}>;
```
At the moment the client connects to the websocket and receives the init message. That init contains all the elements the way the are at the moment in the room and reconciles them with the JSON loaded from the database. But it doesn't receive the files.
Some of these elements, may be images (type: `ExcalidrawImageElement`), they have a property `fileId` that Excalidraw uses to find the file in the `files` object.
If the file was not pushed to the database, it was not comming in the original JSON, so the client will not have the file available.
The excalidraw-app implementation uploads the images to firebase, and then clients download them from there by hash.
When a user puts a image in the whiteboard, it was already sharing with the rest of the users connected to the same room the `image` element. The other clients were showing just a gray square.
What we are doing now is, when a user adds a new file to the whiteboard the file is broadcasted to the rest of the user with a FILE_UPLOAD message.
We have experimented with files and they look work well up to 1MB, from ther it looks like the broadcasted messages are dropped. Excalidraw is limited to 2MB images anyway.
There is another problem, when a user connects to an already existing room (a whiteboard that some people are already editing), and somebody has put a new image, the users that were connected, they all have that image, but the new client doesn't because it is not in the database.
What we have done is, on Init (Collab.tsx:267) check the images inserted in the whiteboard, find the missing files and FILE_REQUEST them with a request message.
When another client receives a FILE_REQUEST, it immediatelly answers with the file if they have it.
_THIS WILL NOT SCALE WELL_
The main problem of this is that we don't have client-to-client communication, all messages are broadcasted. So all clients that have that file, are going to send the file to all the clients.
##### How to improve the algorithm
A) The best way would be to implement something like excalidraw-app does, upload the files to somewhere common, ipfs... but for that we're going to need to think about the StorageBuckets and how to pass them to the Collab class and how to reference the files by the hash.
There is an interesting callback in excalidraw: [generateIdForFile](https://docs.excalidraw.com/docs/@excalidraw/excalidraw/api/props#generateidforfile) we may need to use.
B) Maybe something like sending a FILE_SEARCH, then clients that have that file can respond with a FILE_OFFER, so we can make the client request the file to one of the clients with a FILE_REQUEST sending the message only to the offerer, and not broadcasting it, so the offerer can respond with the file:
- Client A: Broadcast `FILE_SEARCH(hash)`
- Clients that have that file in memory, respond to A a `FILE_OFFER`
- Client A chooses one of the clients (for example Client B) and sends a FILE_REQUEST to B.
- Client B sends the file to A.
(All these messages through the existing websocket, not direct messages)