# End-to-End ML Engineering: CI/CD, Docker, MLflow, FastAPI

* I designed and implemented a full-stack machine learning system for computer vision, integrating CI/CD pipelines, Docker containerization, MLflow experiment tracking, and FastAPI service deployment. The project demonstrates reproducible workflows, automated testing, and scalable deployment — transforming ML models into production-ready services.
* I don’t just build models — I build systems that make models usable, reproducible, and deployable.
* This project shows I can take a model from research to production, with all the engineering rigor clients need.

# :key: Highlights

✅ CI/CD: Automated testing and release pipelines with GitHub Actions.

✅ Docker: Lean runtime images + dev tooling split for reproducibility.

✅ MLflow: Experiment tracking, artifact logging, and reproducible runs.

✅ FastAPI: REST API exposing inference endpoints with Swagger docs.

✅ Testing Suite: Unit, integration, and registry consistency checks.

# :desktop_computer: Visuals to Include

Architecture diagram (Mermaid or exported PNG).

Screenshot of Swagger UI (/docs).

MLflow dashboard screenshot (metrics/artifacts).

CI/CD pipeline run screenshot (GitHub Actions green check).

These visuals make the project tangible for non-technical clients.

# :rocket: Quickstart

# 1. Prepare dataset (VOC → YOLO-seg)
make data

# 2. Start Docker services (app + MLflow)
make up
make mlflow-up

# 3. Run inference with MLflow tracking
make track-infer

# 4. Benchmark performance
make bench

# 5. Log dataset snapshot into MLflow
make snapshot

* Browse MLflow UI at http://localhost:5000 for metrics, configs, and artifacts.
* Push to main → CI automatically bumps patch version, tags repo, and publishes release artifacts (wheel + Docker image).

# :file_folder: Folder Structure

Root
├── configs/ # YAML configs for data, inference, and models
│ ├──data.yaml # Dataset splits + class list
│ ├──inference.yaml # Runtime + postprocess settings
│ └──model.yaml # Default model definition
├── docker/ # Dockerfiles + compose configs
│ ├──app.Dockerfile # Application container
│ ├──mlflow.Dockerfile # MLflow tracking server
│ └──compose.yaml # Docker Compose services
├── models/
│ ├──artifacts/ # Trained weights organized by model + version
│ └──yolo-ultra/1.0.0/best.pt
│ └──registry.json # Local model registry (semantic versioned)
├── scripts/ # Utility scripts
│ ├──prepare_data.py # Convert raw VOC → YOLO-seg format
│ ├── benchmark_inference.py # Benchmark latency/throughput/IoU
│ ├── train.py # Training wrapper if needed
│ └──demo.sh # End-to-end demo workflow
├── src/
│ └──ydd/ # Core library
│ ├── models/ # Model adapters (YOLOUltralytics, etc.)
│ ├── pipeline/ # Inference runner
│ └──tracking/ # MLflow client
├── Makefile # Common commands (data, bench, track-infer, snapshot)
└──README.md # Documentation

## :heavy_plus_sign: Adding a New Model

### Place weights

Put your trained weights in:

models/artifacts/<model-name>/<version>/best.pt

Example:

models/artifacts/yolo-large/1.0.0/best.pt

### Update registry

Add an entry in models/registry.json:

"yolo-large": {
  "1.0.0": {
    "weights_path": "models/artifacts/yolo-large/1.0.0/best.pt",
    "format": "pt",
    "task": "segment",
    "classes": ["good","LID","Silver_Line","Scratch","internal","background"],
    "notes": "YOLOv8l baseline trained on oscillator defects",
    "input_size": [640, 640],
    "conf_threshold": 0.25,
    "iou_threshold": 0.45
  }
}

### Update configs

Edit configs/model.yaml to include the new model:

models:
  - name: "yolo-ultra"
    version: "1.0.0"
    type: "ultralytics"
    weights_path: "models/artifacts/yolo-ultra/1.0.0/best.pt"
    ...
  - name: "yolo-large"
    version: "1.0.0"
    type: "ultralytics"
    weights_path: "models/artifacts/yolo-large/1.0.0/best.pt"
    ...

## :label: Releasing a New Version to GitHub

### Bump version

Use CLI or Makefile:

make bump-patch   # 1.0.0 → 1.0.1
make bump-minor   # 1.0.0 → 1.1.0
make bump-major   # 1.0.0 → 2.0.0

This updates both models/registry.json and configs/model.yaml.

### Push to GitHub

Commit and push changes:

git add models/registry.json configs/model.yaml
git commit -m "feat: bump YOLO Ultra to 1.0.1"
git push origin main

### CI/CD automation

#### GitHub Actions will

* Run tests + lint.
* Bump patch version automatically on successful builds.
* Tag repo (vX.Y.Z).
* Publish GitHub Release with artifacts (wheel + Docker tarball).

## :whale: Deploying with Docker

### Build images

docker compose -f docker/compose.yaml build

### Start services

docker compose -f docker/compose.yaml up -d

This launches:

* App container (inference pipeline).
* MLflow server (metrics + artifacts at http://localhost:5000).

### Run inference

make track-infer

Logs metrics, predictions, configs, and dataset snapshots into MLflow.

### Stop service

docker compose -f docker/compose.yaml down

flowchart TD
    A["User Input Images"] --> B["YOLO Adapter"]
    B --> C["Inference Runner"]
    C --> D["Postprocessing"]
    D --> E["Outputs (JSON, Visuals, Metrics)"]
    E --> F["MLflow Tracking"]

sequenceDiagram
    participant U as User
    participant I as Input Images
    participant M as Model Adapter (YOLOUltralytics)
    participant R as Inference Runner
    participant P as Postprocessing
    participant O as Outputs
    participant L as MLflow Tracking

    U->>I: Provide test/val images
    I->>M: Load model weights (best.pt)
    M->>R: Run predictions
    R->>P: Apply thresholds (conf, IoU)
    P->>O: Format masks, generate JSON + visuals
    O->>L: Log metrics, configs, dataset snapshot, artifacts
    L-->>U: Results available in MLflow UI