# Camera-Based Canny Edge Detection System Report

**Student Name:** YU-CHUN, LIN
**Student ID:** 71135843

## 1. Project Description
The goal of this project is to implement a real-time camera-based Canny edge detection system using OpenCV and C++. The main objectives include:
- Capturing live video feed from a camera using libcamera
- Processing multiple frames using the Canny edge detection algorithm
- Measuring and analyzing system performance metrics
- Comparing CPU time versus Wall time performance

Expected end results include processed frames showing edge detection results, performance metrics, and timing analysis for both CPU and Wall time measurements.

## 2. Experimental Setup
### Hardware Setup
- Camera system using libcamera
- Resolution: 640x480 pixels
- Frame rate: 30 fps

### Software Implementation
The system is implemented using C++ with the following key components:
- OpenCV library for image processing
- GStreamer pipeline for camera capture
- Custom Canny edge detection implementation

Key code components:

// GStreamer pipeline configuration
string pipeline = 
    "libcamerasrc ! "
    "video/x-raw,format=NV12,width=640,height=480,framerate=30/1 ! "
    "videoconvert ! "
    "appsink drop=true";

The methodology includes:
1. Camera initialization using GStreamer pipeline
2. Frame capture in a continuous loop
3. Conversion of frames to grayscale
4. Application of Canny edge detection
5. Performance measurement using high-resolution clock
6. Storage of processed frames

## 3. Results
### Image Processing Results
- The system successfully captures and processes up to 150 frames
- Each frame is converted to grayscale and processed using Canny edge detection
- Output images are saved in PGM format with sequential naming (frame001.pgm, frame002.pgm, etc.)

### Performance Metrics
The system measures three key timing metrics:
1. Wall Time: Total real-world time elapsed
2. CPU Time: Actual processor time used
3. Per-frame processing time and FPS (Frames Per Second)

### Timing Analysis
The code measures timing using both CPU clock and Wall clock:

auto total_start = high_resolution_clock::now();  // Wall time start
clock_t cpu_time = clock();                      // CPU time

### Performance Statistics

========================
Total frames processed: 150
Wall time (total): 11.857 seconds
CPU time: 20.450 seconds
Average time per frame: 0.079 seconds
Average FPS: 12.65

## 4. Problems and Discussion
### Challenges Faced
1. **Camera Integration**
- Challenge: Implementing reliable camera capture using libcamera
- Solution: Used GStreamer pipeline with appropriate configuration

2. **Performance Optimization**
- Challenge: Achieving real-time processing capabilities
- Solution: Implemented efficient frame handling and memory management

3. **Timing Measurement**
- Challenge: Accurate measurement of CPU vs Wall time
- Solution: Used high_resolution_clock for Wall time and clock() for CPU time

### CPU Time vs Wall Time
- CPU Time: Represents actual processor time used by the program
- Wall Time: Represents total elapsed real-world time
- Difference: Wall time includes system operations, I/O waiting, and other processes
- Wall time < (CPU time + System time): This indicates parallel processing

### Time Command Analysis
When running the executable with the bash time command:

$ time ./camera_canny <sigma> <tlow> <thigh>

The output shows:
- real: Total wall clock time (like watching a clock on the wall)
- user: CPU time spent in user mode
- sys: CPU time spent in kernel mode

### Potential Improvements
1. Implementation of parallel processing for faster frame processing
2. Memory optimization for handling longer recording sessions
3. Enhanced error handling for camera disconnection scenarios

## 5. Conclusion
The implemented camera-based Canny edge detection system successfully achieves real-time edge detection with performance monitoring capabilities. The system effectively demonstrates the relationship between CPU and Wall time in real-world applications, providing insights into system performance characteristics.

The timing analysis reveals the overhead associated with I/O operations and system calls, as evidenced by the difference between CPU and Wall time measurements. This project successfully implements both image processing functionality and performance measurement capabilities, providing a foundation for further optimization and enhancement.

### Running Instructions
1. Compile the program:

   g++ canny_util.c camera_canny.cpp -o camera_canny -I. `pkg-config --cflags --libs opencv4`

2. Run the program:

   ./camera_canny <sigma> <tlow> <thigh>

Example: `./camera_canny 1.0 0.5 0.75`

3. Press ESC to start processing
4. Program will automatically process 150 frames
5. Results will be saved as individual PGM files
6. To encode processed frames into video, run:

   ffmpeg -i frame%03d.pgm -pix_fmt yuvj420p frame_vid.h264