# Upload Architecture and API Flow
This document explains the upload logic and different APIs used in the drive system and attachment service.
## Overview
The upload system consists of multiple layers working together to handle file selection, upload processing, state management, and display transformation.
## Architecture Layers
> **Important Note:** Both simple upload and multipart upload use the **exact same core logic**. The only difference is the delivery method - multipart upload is just a more efficient way to handle larger files using the same underlying patterns.
### 1. AttachmentService - File Selection Layer
The `AttachmentService` handles the initial file selection and preparation:
**Key Methods:**
- `takePhoto()` - Camera capture using `react-native-image-picker`
- `pickDocuments()` - File picker using `react-native-document-picker`
- `pickImages()` - Image-specific picker (shortcut method)
**Features:**
- Permission handling (Android camera permissions)
- File validation (10MB size limit)
- Multiple file type support
- Error handling and user cancellation
### 2. DriveApi - Upload Processing Layer
The `DriveApi` provides multiple upload strategies:
#### A. Simple Upload (`uploadFile`)
- Uses `FormData` with `multipart/form-data`
- Direct upload to `/file/upload` endpoint
- Suitable for smaller files
- Single API call
#### B. Multipart Upload (Advanced)
For larger files, the system uses a 3-step multipart upload process:
1. **Get Presigned URLs** (`getPresignedUrls`)
2. **Upload File Parts** (`uploadFilePart`)
3. **Finalize Upload** (`finalizeUpload`)
#### C. File Access (`getSignedUrl`)
- Generates temporary signed URLs for file access
- Used by `attachmentTransformer.ts` for displaying files
- Handles authentication and CORS issues
### 3. DriveStore - State Management Layer
The `DriveStore` manages upload state and integrates with the API:
**Features:**
- Progress tracking (`setUploadProgress`)
- Error handling
- State management (loading, success, error)
- Automatic file addition to store after successful upload
### 4. AttachmentTransformer - Display Layer
Handles transformation of uploaded files for display:
**Process:**
1. Determines media type from file extension
2. Gets signed URL using `DriveApi.getSignedUrl()`
3. Falls back to direct URL if signed URL fails
4. Returns display-ready `Attachment` object
## Upload Flow Diagram
```mermaid
graph TD
A[User Action<br/>Take Photo/Pick File] --> B[AttachmentService]
B --> C{Camera or<br/>File Picker}
C -->|Camera| D[launchCamera]
C -->|Files| E[DocumentPicker.pick]
D --> F[Create Attachment Object]
E --> F
F --> G[File Validation<br/>10MB limit]
G -->|Valid| H[DriveApi.getInstance]
G -->|Invalid| I[Return Error]
H --> J{Upload Strategy}
J -->|Small Files| K[Simple Upload<br/>uploadFile]
J -->|Large Files| L[Multipart Upload]
K --> M[FormData POST<br/>/file/upload<br/>Same DriveApi instance]
L --> N[Step 1: getPresignedUrls<br/>Same DriveApi instance]
N --> O[Step 2: uploadFilePart<br/>Same RNFS.readFile logic<br/>Same binary conversion]
O --> P[Step 3: finalizeUpload<br/>Same DriveApi instance]
M --> Q[DriveStore<br/>Same state management]
P --> Q
Q --> R[Same Progress Tracking<br/>Same Error Handling]
R --> S[Add to Files Array]
S --> T[AttachmentTransformer<br/>Same getSignedUrl logic]
T --> U[Same Signed URL Process]
U --> V[Return Display Format]
style H fill:#e1f5fe
style Q fill:#e1f5fe
style T fill:#e1f5fe
style O fill:#fff3e0
style M fill:#fff3e0
```
## API Endpoints
### Core Upload Endpoints
| Endpoint | Method | Purpose |
|----------|--------|---------|
| `/file/upload` | POST | Simple file upload |
| `/upload/get-presigned-urls` | GET | Get S3 presigned URLs for multipart |
| `/uppy/sign-part/{uploadId}/{partNumber}` | POST | Sign individual upload parts |
| `/upload/finalize` | POST | Complete multipart upload |
| `/file/signed-url` | GET | Get signed URL for file access |
### File Management Endpoints
| Endpoint | Method | Purpose |
|----------|--------|---------|
| `/file/find/{fileId}` | GET | Find file by ID |
| `/files` | GET | List files with pagination |
| `/file/{fileId}` | DELETE | Delete file |
| `/file/{fileId}/share` | POST | Share file with users |
| `/folder` | POST | Create folder |
| `/file/{fileId}/trash` | PATCH | Move to trash |
| `/file/{fileId}/restore` | PATCH | Restore from trash |
## Upload Strategies Comparison
### Simple Upload
```typescript
// Single API call
const formData = new FormData()
formData.append("file", params.file)
formData.append("name", params.name)
const response = await this.apisauce.post("/file/upload", formData)
```
**Pros:**
- Simple implementation
- Single API call
- Good for small files
**Cons:**
- Limited by server timeout
- No progress tracking
- Memory intensive for large files
### Multipart Upload (Same Core Logic)
Even in multipart upload, the system uses the **same fundamental logic** as simple upload:
```typescript
// Step 1: Get presigned URLs (same DriveApi instance)
const presignedUrls = await DriveApi.getInstance().getPresignedUrls({ name, size })
// Step 2: Upload parts directly to S3 (same file handling)
const parts = await Promise.all(
fileParts.map(async (part, index) => {
// Still uses same RNFS.readFile and binary conversion logic
const result = await this.uploadFilePart(signedUrls[index], part.uri)
return { ETag: result.etag, PartNumber: index + 1 }
})
)
// Step 3: Finalize upload (same DriveApi instance)
await DriveApi.getInstance().finalizeUpload({ fileKey, parts, fileId, ... })
```
**Key Point:** Multipart upload is just a **different delivery method** for the same file data. The core logic remains identical:
1. **Same DriveApi instance** - Uses the same singleton pattern
2. **Same file reading** - Uses `RNFS.readFile(fileUri, "base64")`
3. **Same binary conversion** - Converts base64 to binary the same way
4. **Same authentication** - Uses same headers (`current-org-id`, `x-device-id`)
5. **Same error handling** - Uses same `getGeneralApiProblem` pattern
6. **Same state management** - Goes through same DriveStore flow
**The only difference is WHERE the file goes:**
- **Simple Upload**: File → Server → S3
- **Multipart Upload**: File → S3 directly (via presigned URLs)
**Pros:**
- Handles large files efficiently
- Parallel upload of parts
- Better error recovery
- Progress tracking per part
- **Same familiar API patterns**
**Cons:**
- More API calls required
- Requires S3 integration
- **But uses identical core logic**
## Shared Logic Components
Both upload strategies share these identical components:
### 1. DriveApi Singleton Instance
```typescript
// Both strategies use the same instance
const driveApi = DriveApi.getInstance()
```
### 2. File Reading Logic
```typescript
// Both use identical RNFS file reading
const fileContent = await RNFS.readFile(fileUri, "base64")
const binaryString = atob(fileContent)
const bytes = new Uint8Array(binaryString.length)
for (let i = 0; i < binaryString.length; i++) {
bytes[i] = binaryString.charCodeAt(i)
}
```
### 3. Authentication Headers
```typescript
// Both use same headers automatically added by ApiBase
headers: {
"current-org-id": this.currentOrgId,
"x-device-id": this.deviceId
}
```
### 4. Error Handling Pattern
```typescript
// Both use identical error handling
const problem = getGeneralApiProblem(response)
if (problem) return problem
```
### 5. State Management Flow
```typescript
// Both go through same DriveStore flow
this.setUploadProgress(25, "Uploading...")
const result = await DriveApi.getInstance().uploadFile(params)
this.setUploadProgress(100, "Upload complete!")
```
### 6. File Display Logic
```typescript
// Both use same signed URL logic for display
const result = await driveApi.getSignedUrl(fileKey)
```
**The key insight:** Multipart upload is not a different system - it's the same system with a different delivery mechanism. All the core patterns, error handling, state management, and file processing remain identical.
## File Types and Validation
### Supported File Types
**Images:**
- JPG, JPEG, PNG, GIF, BMP, WebP
**Documents:**
- PDF, DOC, DOCX
**General:**
- Any file type (with 10MB limit)
### Validation Rules
```typescript
validateAttachment(attachment: Attachment): { valid: boolean; error?: string } {
// Check URI exists
if (!attachment.uri) {
return { valid: false, error: "Missing file URI" }
}
// Check name exists
if (!attachment.name) {
return { valid: false, error: "Missing file name" }
}
// Check file size (10MB limit)
const maxSize = 10 * 1024 * 1024 // 10MB
if (attachment.size && attachment.size > maxSize) {
return { valid: false, error: "File too large (max 10MB)" }
}
return { valid: true }
}
```
## Security Features
### Authentication Headers
- `current-org-id`: Organization context
- `x-device-id`: Device identification
### Access Control
- **Owner**: Full control (edit, delete, share)
- **Editor**: Can edit and share
- **Viewer**: Read-only access
### Signed URLs
- Temporary access with expiration
- Prevents unauthorized access
- Handles CORS issues in mobile apps
## Error Handling
### Common Error Scenarios
1. **Permission Denied**
```typescript
return { success: false, error: "Camera permission denied" }
```
2. **File Too Large**
```typescript
return { valid: false, error: "File too large (max 10MB)" }
```
3. **Upload Failed**
```typescript
return { kind: "unknown", temporary: true, error }
```
4. **Network Issues**
```typescript
return { kind: "unknown", temporary: true }
```
## State Management Flow
```mermaid
stateDiagram-v2
[*] --> Idle
Idle --> Selecting: User selects file
Selecting --> Validating: File selected
Validating --> Valid: File valid
Validating --> Invalid: File invalid
Invalid --> [*]: Show error
Valid --> Uploading: Start upload
Uploading --> Progress: Upload in progress
Progress --> Success: Upload complete
Progress --> Failed: Upload failed
Success --> [*]: Show success
Failed --> [*]: Show error
```
## Integration Points
### With Chat System
- Files uploaded can be attached to messages
- `conversationId` and `channelId` fields link files to conversations
- Real-time updates via WebSocket events
### With Project System
- Files can be organized by `projectId`
- Folder structure with `parentId` relationships
- Project-specific file sharing
### With User Management
- User-specific quotas and limits
- Organization-based access control
- Sharing permissions per user
## Performance Considerations
### File Size Limits
- **Simple Upload**: Limited by server timeout (~30 seconds)
- **Multipart Upload**: No practical limit (handles GB+ files)
### Memory Usage
- **Simple Upload**: Entire file loaded into memory
- **Multipart Upload**: Only current part in memory
### Network Efficiency
- **Simple Upload**: Single HTTP request
- **Multipart Upload**: Multiple parallel requests
## Best Practices
1. **Use Simple Upload** for files < 10MB
2. **Use Multipart Upload** for files > 10MB
3. **Always validate** files before upload
4. **Handle errors gracefully** with user-friendly messages
5. **Show progress** for better user experience
6. **Clean up** temporary files after upload
7. **Use signed URLs** for file access to avoid CORS issues
<style>
section#notes-recommendations,
section#notes-recommendations hr {
display: none !important;
}
</style>
Sign in with Wallet
Connect another wallet