---
# System prepended metadata

title: Parallélisme d'upload dynamique

---



# Parallélisme d'upload dynamique
## Spec

- L'upload engine possède un nombre maximum d'upload parallèle
- Le parallélisme dynamique consiste à pouvoir modifier ce nombre à n'importe quel moment
- On peut distinguer deux cas à gérer si il y a des uploads déjà en cours : 
    - Augmentation du nombre
        - On commence des nouveaux upload jusqu'à atteindre le nouveau maximum
    - Diminution du nombre
        - A la fin des uploads en cours, on ne démarre plus de nouveaux upload jusqu'à atteindre le nouveau maximum

## Implem spécifique web dans un 1er temps

```mermaid
classDiagram
    
    class ParallelOperationQueue~Input, Output~ {
        - queue: Input[]
        - maxConcurrency: number
        - currentConcurrency: number
        - operation: (input: Input) => Promise~Ouput~
        
        + constructor(operation: (input: Input) => Promise<Ouput>), maxConcurrency: number)
        
        + add(input: Input) Promise~Output~
        + setMaxConcurrency(concurrency: number)
    }
```

Draft de conception KMP (Non fonctionnel, à transformer en atelier sur la gestion de l'asynchrone en KMP)

```mermaid
classDiagram
    
    class OperationQueue {
        -operations: List~Operation~
        +addOperation(operation: Operation): Task<Output>
        +removeOperation()
    }
    
    class JSOperation~Input, Output~ {
        -input: Input

        +start()
        -abstract process(input: Input): Promise<Output>
        +cancel()
    }
    
    class MobileOperation~Input, Output~ {
        -input: Input
        +onOperationFinished: (result: Output)
        +onOperationFailed: (result: Error)
        -task: Task

        +start()
        -abstract process(input: Input): Output
        +cancel()
    }
    
    class UploadOperation~MediaToUpload, UploadedPhoto~ {
        -uploader: Uploader
        -preparer: Preparer // Only mobile
        
        -process(MediaToUpload): UploadedPhoto
    }
    
    class ExtractImageDataOperation~File, ImageData~ {
        -process(File): ImageData
    }
    
    Operation <|-- UploadOperation
    Operation <|-- ExtractImageDataOperation
    
    class UploadEngine {
      operationQueue: OperationQueue 
    }

```


# Calcul de la vitesse de téléversement

```mermaid
classDiagram

    class UploadQuality {
        <<enum>>
        //TODO Voir avec le produit
        GOOD,
        BAD,
        UNKNOWN
    }
    
    class UploadEngine {
      uploader: Uploader
      state: UploadState
      constructor()
      uploadSpeedEstimation: MegaBytesPerSeconds
      add(photo: PhotoToUpload)
      getUploadQuality() UploadQuality
      onPhotoUploaded(state: State, photo: UploadedPhoto)
      onUploadFailed(state: State, photo: FailedPhoto)
      onUploadCompleted(state: State)
      onUploadSpeedEstimationChanged(uploadSpeedEstimation: MegaBytesPerSeconds)
      onUploadQualityChanged(uploadQuality: UploadQuality)
    }
```

@Alexandre Guzu 13/09/24 :

- Pourquoi stocker `uploadSpeedEstimation` au niveau de l'UploadEngine ?
- À quoi sert `getUploadQuality()` ? C'est juste une fonction privée ? 
    - À priori fonction publique qui servirait à l'init de l'appelant à connaître la qualité d'upload actuelle. L'appelant s'abonne ensuite aux mises à jour de la qualité et les reçoit via `onUploadQualityChanged`


# Etape 3 - Adaptation de la couche d'observabilité

Liste des changements 
- Retrait des promesses en retour de la fonction `add`
- Ajout des méthodes d'observations 
  - onPhotoUploaded(state: State, photo: UploadedPhoto)
  - onUploadFailed(state: State, photo: FailedPhoto)
  - onUploadCompleted(state: State)

```mermaid
classDiagram
    UploadEngine --* Uploader
    UploadEngine --* UploadState
    
    
    class Uploader {
      upload(uploadablePhoto: UploadablePhoto): UploadedPhoto
    }  
    
    class UploadState {
      uploadingPhotos: Array~PhotoToUpload~
      uploadedPhotos: Array~UploadedPhoto~
      failedPhotos: Array~FailedPhoto~
      getUploadingPhotoCount(): Int
      getUploadedPhotoCount(): Int
      getFailedPhotoCount(): Int
    }  


    class UploadEngine {
      uploader: Uploader
      state: UploadState
      constructor()
      add(photo: PhotoToUpload)
      onPhotoUploaded(state: State, photo: UploadedPhoto)
      onUploadFailed(state: State, photo: FailedPhoto)
      onUploadCompleted(state: State)
    }
    
    
    class FotomUploader {
      provider: FotomApiProvider
      isUploadPhotoOutsideCustomization: boolean
      constructor(provider: FotomApiProvider, isUploadPhotoOutsideCustomization = false)
      upload(photoToUpload: PhotoToUpload): Promise~UploadedPhoto~
    }
    
    Uploader <|-- FotomUploader
    
    FotomUploader --> FotomApiProvider

```

# Etape 2 - Adaptation du moteur interne

```mermaid
classDiagram
    UploadEngine --* Uploader
    UploadEngine --* UploadState
    
    
    class Uploader {
      upload(uploadablePhoto: PhotoToUpload): UploadedPhoto
    }  
    
    class UploadState {
      uploadingPhotos: Array~PhotoToUpload~
      uploadedPhotos: Array~UploadedPhoto~
      failedPhotos: Array~FailedPhoto~
      getUploadingPhotoCount(): Int
      getUploadedPhotoCount(): Int
      getFailedPhotoCount(): Int
    }  


    class UploadEngine {
      uploader: Uploader
      state: UploadState
      finished: SelectionPhoto[];

      
      onTickSubscribers: Array&lt;&lcub; callback: OnTickCallback &rcub;&gt;
      onQueueEmptySubscribers: Array&lt;&lcub; callback: OnQueueEmptyCallback &rcub;&gt;;
      constructor()
      onTick(callback: OnTickCallback): () => void
      notifyOnTick(): void
      onQueueEmpty(callback: OnQueueEmptyCallback): () => void
      notifyOnQueueEmpty(): void
      addToFinished(selectionPhoto: SelectionPhoto): void
      add(photo: PhotoToUpload): Promise~UploadedPhoto~
    }
    
    class OnTickCallback {
        type: () => void
    }
        
    class OnQueueEmptyCallback {
        type: () => void
    }
    
    class FotomUploader {
      provider: FotomApiProvider;
      isUploadPhotoOutsideCustomization: boolean;
      constructor(provider: FotomApiProvider, isUploadPhotoOutsideCustomization = false)
      upload(photoToUpload: PhotoToUpload): Promise~UploadedPhoto~
    }
    
    Uploader <|-- FotomUploader
    
    class UploadSuccessCallback {
      type: (uploadedPhoto: UploadedPhoto) => void 
    }

    
    UploadEngine --> OnTickCallback
    UploadEngine --> OnQueueEmptyCallback
    UploadEngine --> UploadSuccessCallback
    
    FotomUploader --> FotomApiProvider

```

## Mise à jour des models photos

```mermaid
classDiagram
    class Photo {
     id: string
    }
    
    class PhotoToUpload {
     
    }
    
    class GooglePhoto {
     url: string
    }
    
    class FilePhoto {
     file: File
    }
    
    class UploadedPhoto {
      hash: string
      width: Length
      height: Length
      lastModified: number
    }
    
    class FailedPhoto {
        photoToUpload: PhotoToUpload
        errors: Array~Error~
    }

    Photo <|-- PhotoToUpload 
    Photo <|-- FailedPhoto
    
    PhotoToUpload <|-- GooglePhoto 
    PhotoToUpload <|-- FilePhoto 
    Photo <|-- UploadedPhoto
```


# Etape 1 - Adaptation des entrées / sorties + nouveau models




Liste des modifications : 
 - `upload(selectionPhoto: SelectionPhoto, file?: File): Promise<SelectionPhoto>` 
     - `upload(photoToUpload: PhotoToUpload): Promise<UploadedPhoto>`
 - `UploadItem.selectionPhoto: SelectionPhoto` 
     - `UploadItem.photoToUpload: PhotoToUpload`
 - `OnQueueEmptyCallback = (selectionPhoto): void` 
     - `(): void` 
 - `notifyOnQueueEmpty(selectionPhotos): void`
     - `notifyOnQueueEmpty(): void` 
 - `UploadSucessCallback = (selectionPhoto): void` 
     - `(uploadedPhoto): void`
 - `doUpload(uploadItem): Promise<void>` 
     - `doUpload(uploadItem): Promise<UploadedPhoto>`
     - La mise à jour de la selection photo sera à faire dans le `.then` de la fonction `upload()` à partir de l'`UploadedPhoto` retournée

```mermaid
classDiagram
    class Photo {
     id: string
    }
    
    class PhotoToUpload {
     
    }
    
    class GooglePhoto {
     url: string
    }
    
    class FilePhoto {
     file: File
    }
    
    class UploadedPhoto {
      hash: string
      width: Length
      height: Length
      lastModified: number
    }

    Photo <|-- PhotoToUpload 
    PhotoToUpload <|-- GooglePhoto 
    PhotoToUpload <|-- FilePhoto 
    Photo <|-- UploadedPhoto
```

```mermaid
classDiagram
class UploadUtils {
        async uploadOrNot(uploadProvider: BaseUploadProvider | undefined, selectionPhoto: SelectionPhoto,dispatch: DispatchCustomizationAction)
        async handleFile(file: File, uploadProvider: BaseUploadProvider, dispatch?: DispatchCustomizationAction): Promise&lt;SelectionPhoto | void&gt;
    }

    UploadUtils ..> BaseUploadProvider
    UploadUtils ..> SelectionPhoto

    class BaseUploadProvider {
      queue: UploadItem[];
      finished: SelectionPhoto[];
      onTickSubscribers: Array&lt;&lcub; callback: OnTickCallback &rcub;&gt;
      onQueueEmptySubscribers: Array&lt;&lcub; callback: OnQueueEmptyCallback &rcub;&gt;;
      constructor()
      upload(photoToUpload: PhotoToUpload): Promise~UploadedPhoto~
      tick(): void
      doTick(): void
      start(uploadItem: UploadItem): void
      finish(uploadItem: UploadItem): void
      reject(uploadItem: UploadItem, error: Error): void
      doUpload(uploadItem: UploadItem): Promise~void~*
      count(status?: "queued" | "started" | "finished"): number
      onTick(callback: OnTickCallback): () => void
      notifyOnTick(): void
      onQueueEmpty(callback: OnQueueEmptyCallback): () => void
      notifyOnQueueEmpty(): void
      addToFinished(selectionPhoto: SelectionPhoto): void
    }
    
    class OnTickCallback {
        type: () => void
    }
        
    class OnQueueEmptyCallback {
        type: () => void
    }
    
    class FotomUploadProvider {
      provider: FotomApiProvider;
      isUploadPhotoOutsideCustomization: boolean;
      constructor(provider: FotomApiProvider, isUploadPhotoOutsideCustomization = false)
      doUpload(uploadItem: UploadItem): Promise~UploadedPhoto~
    }
    

    class UploadItem {
      key: string;
      reject: UploadErrorCallback;
      retry?: number;
      photoToUpload: PhotoToUpload;
      status: "queued" | "started" | "finished";
      success: UploadSuccessCallback;
    }
    
    class UploadSuccessCallback {
      type: (uploadedPhoto: UploadedPhoto) => void 
    }
    
    class SelectionPhoto {
      key: string;

      width: Length<Unit.PX>;
      height: Length<Unit.PX>;

      localRef?: string;
      lastModified?: number;

      provider: SelectionProvider;
      providerRef: string;
      providerThumbnailUrl?: string;
      providerPictureUrl?: string;

      fotomKey?: string;
      status?: "error" | "success" | "uploading";
    }
    
    class UploadError {
      retryable: boolean;
    }
    
    BaseUploadProvider <|-- FotomUploadProvider
    BaseUploadProvider --> UploadItem
    UploadItem --> UploadSuccessCallback
    BaseUploadProvider --> UploadError
    BaseUploadProvider --> OnTickCallback
    BaseUploadProvider --> OnQueueEmptyCallback
    BaseUploadProvider --> SelectionPhoto
    
    UploadError --|> Error
    FotomUploadProvider --> FotomApiProvider
    FotomUploadProvider --> UploadError
```
