# Parallélisme d'upload dynamique ## Spec - L'upload engine possède un nombre maximum d'upload parallèle - Le parallélisme dynamique consiste à pouvoir modifier ce nombre à n'importe quel moment - On peut distinguer deux cas à gérer si il y a des uploads déjà en cours : - Augmentation du nombre - On commence des nouveaux upload jusqu'à atteindre le nouveau maximum - Diminution du nombre - A la fin des uploads en cours, on ne démarre plus de nouveaux upload jusqu'à atteindre le nouveau maximum ## Implem spécifique web dans un 1er temps ```mermaid classDiagram class ParallelOperationQueue~Input, Output~ { - queue: Input[] - maxConcurrency: number - currentConcurrency: number - operation: (input: Input) => Promise~Ouput~ + constructor(operation: (input: Input) => Promise<Ouput>), maxConcurrency: number) + add(input: Input) Promise~Output~ + setMaxConcurrency(concurrency: number) } ``` Draft de conception KMP (Non fonctionnel, à transformer en atelier sur la gestion de l'asynchrone en KMP) ```mermaid classDiagram class OperationQueue { -operations: List~Operation~ +addOperation(operation: Operation): Task<Output> +removeOperation() } class JSOperation~Input, Output~ { -input: Input +start() -abstract process(input: Input): Promise<Output> +cancel() } class MobileOperation~Input, Output~ { -input: Input +onOperationFinished: (result: Output) +onOperationFailed: (result: Error) -task: Task +start() -abstract process(input: Input): Output +cancel() } class UploadOperation~MediaToUpload, UploadedPhoto~ { -uploader: Uploader -preparer: Preparer // Only mobile -process(MediaToUpload): UploadedPhoto } class ExtractImageDataOperation~File, ImageData~ { -process(File): ImageData } Operation <|-- UploadOperation Operation <|-- ExtractImageDataOperation class UploadEngine { operationQueue: OperationQueue } ``` # Calcul de la vitesse de téléversement ```mermaid classDiagram class UploadQuality { <<enum>> //TODO Voir avec le produit GOOD, BAD, UNKNOWN } class UploadEngine { uploader: Uploader state: UploadState constructor() uploadSpeedEstimation: MegaBytesPerSeconds add(photo: PhotoToUpload) getUploadQuality() UploadQuality onPhotoUploaded(state: State, photo: UploadedPhoto) onUploadFailed(state: State, photo: FailedPhoto) onUploadCompleted(state: State) onUploadSpeedEstimationChanged(uploadSpeedEstimation: MegaBytesPerSeconds) onUploadQualityChanged(uploadQuality: UploadQuality) } ``` @Alexandre Guzu 13/09/24 : - Pourquoi stocker `uploadSpeedEstimation` au niveau de l'UploadEngine ? - À quoi sert `getUploadQuality()` ? C'est juste une fonction privée ? - À priori fonction publique qui servirait à l'init de l'appelant à connaître la qualité d'upload actuelle. L'appelant s'abonne ensuite aux mises à jour de la qualité et les reçoit via `onUploadQualityChanged` # Etape 3 - Adaptation de la couche d'observabilité Liste des changements - Retrait des promesses en retour de la fonction `add` - Ajout des méthodes d'observations - onPhotoUploaded(state: State, photo: UploadedPhoto) - onUploadFailed(state: State, photo: FailedPhoto) - onUploadCompleted(state: State) ```mermaid classDiagram UploadEngine --* Uploader UploadEngine --* UploadState class Uploader { upload(uploadablePhoto: UploadablePhoto): UploadedPhoto } class UploadState { uploadingPhotos: Array~PhotoToUpload~ uploadedPhotos: Array~UploadedPhoto~ failedPhotos: Array~FailedPhoto~ getUploadingPhotoCount(): Int getUploadedPhotoCount(): Int getFailedPhotoCount(): Int } class UploadEngine { uploader: Uploader state: UploadState constructor() add(photo: PhotoToUpload) onPhotoUploaded(state: State, photo: UploadedPhoto) onUploadFailed(state: State, photo: FailedPhoto) onUploadCompleted(state: State) } class FotomUploader { provider: FotomApiProvider isUploadPhotoOutsideCustomization: boolean constructor(provider: FotomApiProvider, isUploadPhotoOutsideCustomization = false) upload(photoToUpload: PhotoToUpload): Promise~UploadedPhoto~ } Uploader <|-- FotomUploader FotomUploader --> FotomApiProvider ``` # Etape 2 - Adaptation du moteur interne ```mermaid classDiagram UploadEngine --* Uploader UploadEngine --* UploadState class Uploader { upload(uploadablePhoto: PhotoToUpload): UploadedPhoto } class UploadState { uploadingPhotos: Array~PhotoToUpload~ uploadedPhotos: Array~UploadedPhoto~ failedPhotos: Array~FailedPhoto~ getUploadingPhotoCount(): Int getUploadedPhotoCount(): Int getFailedPhotoCount(): Int } class UploadEngine { uploader: Uploader state: UploadState finished: SelectionPhoto[]; onTickSubscribers: Array&lt;&lcub; callback: OnTickCallback &rcub;&gt; onQueueEmptySubscribers: Array&lt;&lcub; callback: OnQueueEmptyCallback &rcub;&gt;; constructor() onTick(callback: OnTickCallback): () => void notifyOnTick(): void onQueueEmpty(callback: OnQueueEmptyCallback): () => void notifyOnQueueEmpty(): void addToFinished(selectionPhoto: SelectionPhoto): void add(photo: PhotoToUpload): Promise~UploadedPhoto~ } class OnTickCallback { type: () => void } class OnQueueEmptyCallback { type: () => void } class FotomUploader { provider: FotomApiProvider; isUploadPhotoOutsideCustomization: boolean; constructor(provider: FotomApiProvider, isUploadPhotoOutsideCustomization = false) upload(photoToUpload: PhotoToUpload): Promise~UploadedPhoto~ } Uploader <|-- FotomUploader class UploadSuccessCallback { type: (uploadedPhoto: UploadedPhoto) => void } UploadEngine --> OnTickCallback UploadEngine --> OnQueueEmptyCallback UploadEngine --> UploadSuccessCallback FotomUploader --> FotomApiProvider ``` ## Mise à jour des models photos ```mermaid classDiagram class Photo { id: string } class PhotoToUpload { } class GooglePhoto { url: string } class FilePhoto { file: File } class UploadedPhoto { hash: string width: Length height: Length lastModified: number } class FailedPhoto { photoToUpload: PhotoToUpload errors: Array~Error~ } Photo <|-- PhotoToUpload Photo <|-- FailedPhoto PhotoToUpload <|-- GooglePhoto PhotoToUpload <|-- FilePhoto Photo <|-- UploadedPhoto ``` # Etape 1 - Adaptation des entrées / sorties + nouveau models Liste des modifications : - `upload(selectionPhoto: SelectionPhoto, file?: File): Promise<SelectionPhoto>` - `upload(photoToUpload: PhotoToUpload): Promise<UploadedPhoto>` - `UploadItem.selectionPhoto: SelectionPhoto` - `UploadItem.photoToUpload: PhotoToUpload` - `OnQueueEmptyCallback = (selectionPhoto): void` - `(): void` - `notifyOnQueueEmpty(selectionPhotos): void` - `notifyOnQueueEmpty(): void` - `UploadSucessCallback = (selectionPhoto): void` - `(uploadedPhoto): void` - `doUpload(uploadItem): Promise<void>` - `doUpload(uploadItem): Promise<UploadedPhoto>` - La mise à jour de la selection photo sera à faire dans le `.then` de la fonction `upload()` à partir de l'`UploadedPhoto` retournée ```mermaid classDiagram class Photo { id: string } class PhotoToUpload { } class GooglePhoto { url: string } class FilePhoto { file: File } class UploadedPhoto { hash: string width: Length height: Length lastModified: number } Photo <|-- PhotoToUpload PhotoToUpload <|-- GooglePhoto PhotoToUpload <|-- FilePhoto Photo <|-- UploadedPhoto ``` ```mermaid classDiagram class UploadUtils { async uploadOrNot(uploadProvider: BaseUploadProvider | undefined, selectionPhoto: SelectionPhoto,dispatch: DispatchCustomizationAction) async handleFile(file: File, uploadProvider: BaseUploadProvider, dispatch?: DispatchCustomizationAction): Promise&lt;SelectionPhoto | void&gt; } UploadUtils ..> BaseUploadProvider UploadUtils ..> SelectionPhoto class BaseUploadProvider { queue: UploadItem[]; finished: SelectionPhoto[]; onTickSubscribers: Array&lt;&lcub; callback: OnTickCallback &rcub;&gt; onQueueEmptySubscribers: Array&lt;&lcub; callback: OnQueueEmptyCallback &rcub;&gt;; constructor() upload(photoToUpload: PhotoToUpload): Promise~UploadedPhoto~ tick(): void doTick(): void start(uploadItem: UploadItem): void finish(uploadItem: UploadItem): void reject(uploadItem: UploadItem, error: Error): void doUpload(uploadItem: UploadItem): Promise~void~* count(status?: "queued" | "started" | "finished"): number onTick(callback: OnTickCallback): () => void notifyOnTick(): void onQueueEmpty(callback: OnQueueEmptyCallback): () => void notifyOnQueueEmpty(): void addToFinished(selectionPhoto: SelectionPhoto): void } class OnTickCallback { type: () => void } class OnQueueEmptyCallback { type: () => void } class FotomUploadProvider { provider: FotomApiProvider; isUploadPhotoOutsideCustomization: boolean; constructor(provider: FotomApiProvider, isUploadPhotoOutsideCustomization = false) doUpload(uploadItem: UploadItem): Promise~UploadedPhoto~ } class UploadItem { key: string; reject: UploadErrorCallback; retry?: number; photoToUpload: PhotoToUpload; status: "queued" | "started" | "finished"; success: UploadSuccessCallback; } class UploadSuccessCallback { type: (uploadedPhoto: UploadedPhoto) => void } class SelectionPhoto { key: string; width: Length<Unit.PX>; height: Length<Unit.PX>; localRef?: string; lastModified?: number; provider: SelectionProvider; providerRef: string; providerThumbnailUrl?: string; providerPictureUrl?: string; fotomKey?: string; status?: "error" | "success" | "uploading"; } class UploadError { retryable: boolean; } BaseUploadProvider <|-- FotomUploadProvider BaseUploadProvider --> UploadItem UploadItem --> UploadSuccessCallback BaseUploadProvider --> UploadError BaseUploadProvider --> OnTickCallback BaseUploadProvider --> OnQueueEmptyCallback BaseUploadProvider --> SelectionPhoto UploadError --|> Error FotomUploadProvider --> FotomApiProvider FotomUploadProvider --> UploadError ```