# **Image Processing On FPGA**

This is the official repository for the Image processing on FPGA project at IvLabs, VNIT Nagpur.
1. Project Overview
2. Tools and Technologies used
3. Components and Connections
4. RTL Design
5. Hardware and module testing
6. Demonstration
7. Github link
8. Mentors

# Project Overview

This project takes an image sent from a computer and displays it on a VGA monitor after applying filter(s) using an FPGA board (Basys 3).
So the FPGA works like a tiny, custom-built photo viewer.
Here’s the process flow:

1. Computer Sends Image Data
You run a Python program (or any tool) on your laptop that sends image data through a USB-to-UART cable.
Think of it as
“sending the picture one pixel at a time over a wire.”

2. FPGA Receives the Image Bytes
The FPGA receives these bytes using a UART receiver (like a postman collecting letters).
Each pixel of the image is sent as a set of 3 numbers:
Red intensity
Green intensity
Blue intensity
The FPGA groups these pieces into complete pixel values.

3. FPGA Stores the Image in Memory
Once a full pixel is received, the FPGA:
writes it into a memory block inside the board
keeps track of where the next pixel should go
stops writing once the whole frame is stored
So the FPGA ends up storing the entire picture inside itself.

4. FPGA Generates VGA Signals
The FPGA has a custom VGA controller that:
runs at precise timings
scans across the screen pixel by pixel
fetches the right pixel from memory
sends the Red, Green, and Blue values to the VGA port
This is what actually draws the image on the monitor.

5. Switch to Choose Mode & filters
You have a display_switch:
Switch OFF → Receive Mode
FPGA listens to UART and fills memory with the new image.
Switch ON → Display Mode
FPGA stops receiving and starts showing the stored image on the screen.
This prevents flickering or partially uploaded frames.
You also have varoius filter switches , one or more can be applied at the same time.

# Tools and Technologies Used

1. Basys 3 FPGA board by Digilent (xc7a35t-1)
2. Verilog Hardware Description language
3. Python libraries - OpenCV and PySerial
4. AMD Vivado Design Suite
5. Universal Asynchronous Receiver Transmitter (UART) Protocol
6. Video Graphics Array (VGA) video display standard

# Components and Connections


A USB cable connects the FPGA board to laptop for providing power and image data. VGA output of basys 3 connects with vga input of the VGA Monitor

# RTL Design

The memory available on basys 3 is not sufficient to store an entire VGA resolution image(640x480). So instead of using external memory we rather use a QVGA resolution image(320x240) and extrapolate to fill the entire VGA monitor screen.
We use python to send the image to the board.OpenCV is used to resizing to QVGA, RGB Sampling and PySerial for sending pixel values via UART.


In Verilog we designed the following modules
1.UART_Rx
2.Pixel_Compiler
3.DualPort_RAM
4.VGA_Controller
5.LineBuffer
6.Convolution
7.Top_Module


**UART_Rx** :This module receives the incoming data

**Pixel_Compiler**: This module receives bytes as sent by UART_Rx and compiles 3 consecutive bytes' 4 MSB in the sequence RGB using a finite state machine (FSM). So basically a particular pixels' RGB values are received serially but this module combine them into a 12 bit value(3x4)
Note: Usualy we use 8 bit images (RGB888) but since basys 3 provides 4 bit vga signals this module only takes most significant 4 bits (RGB444) instead of all 8 bits of a pixel.

**DualPort_RAM**: The pixel values needs to be written and read for real time image filtering. Hence this module writes into memory while receiving (Write Mode ;display_switch off)
Note: By default Vivado tries to implement this using LUT RAM which is not sufficient in case of Basys 3 and therefore implementation fails.Hence we need to force Vivado to implement this module (and also LineBuffers) using BRAM which is sufficient for storing an entire QVGA RGB444 image.
and can read once entire frame has been stored (Read mode; display_switch on).

**VGA_Controller**: To display through VGA we need control signals VSync and HSync. Though Pixel_Clock and x&y coordinates are not vga output signals we still need them and this modules provide these.The parameters frontporch, backporch,Hsync, Vsync are availabe on vesa.org . We have used the industry standard 640x480 @ 60 Hz refresh rate implying a 25MHz pixel clock.


**LineBuffer**: For convolution operations like blur filter we not only require the pixel to be transformed but also it's 8 neighbouring pixels. But we cannot read 9 pixels in the same pixel clock cycle. Hence even before we start kernel multiplication first 3 rows of pixels are stored in lineBuffer registers which work on first in first out (FIFO) principle. This ensures that no matter which pixel we are transforming we always have the window ready for matrix multiplication.

**Convolution**: As the name suggests this module performs convolution operation on a pixel (RGB in parallel) using the neighbouring values provided by linebuffers and provide the transformed pixel value (say blurred pixel).


**Top_module**: This module instantiates the rest submodules. Also it provides the VGA output signals vga_red, vga_green, vga_blue and provides address for memory read and write operation.

# Hardware and Module testing

1.Behavioural simulation of Convolution Module
* To Test the working of convolution module before implementing it and integrating it with the project we may perform behavioural simulation of the convolution module on grayscale images just for the sake of simplicity .
First using open CV resize any image to 320x240 and convert it to a grayscale image.Then convert it into a text file having hexadecimal values by converting the image pixels value into hexadecimal values strictly in the following form-
0
45
46
47
49
50
50
50
50
and so on.Note that any less number of pixels than 320*240 or any commas will result in simulation error.The input image and output image text files must be in the same simulation folder

The corresponding testbench will read values from this text file to perform convolution and write the hex values of the filtered image into another output text file.The test bench is given in github repository available at the end of this documentation.

Below are some behavioural simulation results--
Original Image

Blurred Image

Edge detection

Sharpen

2.Simultaneous application of various filters
In order to be able to apply various point opeartions on the digital image stored in memory the filter modules needs to be connected in series and each filter needs to be mapped with one of the 16 slide switches in the constraint file of the project.
Basys 3 comes with a built-in self test to check the working of various peripherals available on board right on start up. In order to check the working of slide switches just check if the corresponding led turns on and off with the help of the corresponding switch .

3.Data Transmission
In this project UART protocol has been used to send image data to the fpga board via a USB cable, the same cable which powers the board.
Inorder to check whether data transmission works , basys 3, again, comes with a built-in slef test on startup. Just open any serial monitor and select the appropriate COM port and 9600 baud rate and if the serial monitor displays a welcome message it implies the data transmision via UART is working. Also a receiver led glows as long as the data is transmiting.

4.VGA Output
To test the VGA output of the fgpga board simply connect the vga cable to the fpga board and the monitor and power up the board using a usb cable. Because of the built-in self test a pattern should appear on the screen , otherwise either the cable or vga output or both might be faulty.
Similarly the VGA_controller module could be tested by hardcoding a particular colour for each pixel as the vga output.

# Demonstration
https://drive.google.com/drive/folders/1-PaTR7W7rdAS2068yj5LmBEX0XvsdNMY

# GitHub Link
https://github.com/calmaditya19/image_processing_on_Basys3_FPGA
# Mentors

Kumar Ayush
Sanyam Gupta