# Introduction
- This project aims to reconstruct a Bayer pattern image into a full-color image with a resolution of 128 pixels in both height and width.
- The Bayer pattern, shown below, contains only red, green, or blue color information for each pixel.
- 
- This synthesis area cannot exceed 500 logic elements according to [requirement](https://github.com/PoChuan994/Image_Demosaicing/blob/main/2023_hw5.pdf), and implemented with only 282 logic elements which cost less than 1% area.
- Bilinear interpolation algorithm
- The bilinear interpolation algorithm is applied in this project to reconstruct the full-color image. The algorithm is illustrated below.
- For example, taking index 129 (the green pixel), the red and blue channel values of this pixel can be reconstructed from its surrounding pixels.
- \begin{align}
\text{Blue}(129) &= \left\lfloor \frac{\text{Blue}(128) + \text{Blue}(130)}{2} \right\rfloor \\
\text{Red}(129) &= \left\lfloor \frac{\text{Red}(1) + \text{Red}(257)}{2} \right\rfloor \\
\end{align}
- 
- The full requirements and additional infomation are available at [here](https://github.com/PoChuan994/Image_Demosaicing/blob/main/2023_hw5.pdf) and the implementation can be found in this [GitHub Repository](https://github.com/PoChuan994/Image_Demosaicing).
# Implementation
- To reduce the number of logic elements to less than 500, directly using a $127008 \ (126 \times 126 \times 8)$ register is not allowed. Therefore, the implementation algorithm must be redesigned.
<!-- ## I/O Table
| Sinal Name | I/O | width | Description |
| -------- | -------- | -------- | -------- |
| *clk* | I | 1 | This circuit is a synchronous design triggered at the **positive edge** of *clk*. |
| *reset* | I | 1 | **Active-high asynchronous** reset signal. |
| *data_in* | I | 8 | 8-bit unsigned Bayer pattern input data. |
| *wr_r* | O | 1 | Write/Read enable signal of the Red channel memory. When this signal is high, the value of *wdata_r* will be written to memory. When this signal is low, the value of addr_r in memory will -->
## Register, Wire declartion
- `state`, `next_state`: represent the current state and the next state of the finite state machine (FSM)..
- `center`: stores the coordinates of the current pixel.
- `counter`: tracks the number of pixels that have completed interpolation.
- `cx_add1, cx_minus1, cy_add1, cy_minus1`:
```verilog=
wire [6:0] cx_add1, cx_minus1, cy_add1, cy_minus1;
assign cx_add1 = center[6:0] + 7'd1;
assign cx_minus1 = center[6:0] - 7'd1;
assign cy_add1 = center[13:7] + 7'd1;
assign cy_minus1 = center[13:7] - 7'd1;
```
- Calculate the memory addresses of the neighboring pixels to be read.
- `cx_add1`: right neighboring pixel relative to the current `center` coordinate.
- `cx_minus1`: left neighboring pixel relative to the current `center` coordinate.
- `cy_add1`: above neighboring pixel relative to the current `center` coordinate.
- `cy_minus1`: below neighboring pixel relative to the current `center` coordinate.
- `r_tmp, g_tmp, b_tmp`: temporary registers to accumulate the intermediate values during the pixel color computation.
## Finite State Machine
```verilog=
parameter [3:0] GETDATA = 'd0,
SETTING = 'd1,
RESET = 'd2,
GREEN_BLOCK1_GETDATA = 'd3,
GREEN_BLOCK2_GETDATA = 'd4,
RED_BLOCK_GETDATA = 'd5,
BLUE_BLOCK_GETDATA = 'd6,
WRITE_MEMORY = 'd7,
RESULT = 'd8;
```
- `indicator` in the below FSM graph indicates `center[7], center[0]`.
```mermaid
graph LR;
A((GETDATA))--->|center<16383| A
A--->|center=16383| B((SETTING))
B---> C((RESET))
C--->|indicator=00| D((GREEN_<br/>BLOCK1_<br/>GETDATA))
C--->|indicator=11| E((GREEN_<br/>BLOCK2_<br/>GETDATA))
C--->|indicator=01| F((RED_<br/>BLOCK_<br/>GETDATA))
C--->|indicator=10| G((BLUE_<br/>BLOCK_<br/>GETDATA))
D--->|counter<=1| D
E--->|counter<=1| E
F--->|counter<=3| F
G--->|counter<=3| G
D--->|conter>1| H((WRITE_<br/>MEMORY))
E--->|conter>1| H
F--->|conter>3| H
G--->|conter>3| H
H--->I((RESULT))
```
## `GETDATA` state
```verilog=
GETDATA: begin
if (in_en) begin
center <= center + 1;
case ({center[7],center[0]})
// green case
'b00, 'b11: begin
wr_r <= 0;
wr_g <= 1;
wr_b <= 0;
addr_g <= center;
wdata_g <= data_in;
end
// red case
'b01: begin
wr_r <= 1;
wr_g <= 0;
wr_b <= 0;
addr_r <= center;
wdata_r <= data_in;
end
// blue case
'b10: begin
wr_r <= 0;
wr_g <= 0;
wr_b <= 1;
addr_b <= center;
wdata_b <= data_in;
end
endcase
end
end
```
- Increment the `counter` to update the pixel to be read at next clock.
- Determine the color of current pixel based on the its coordinate, `center[7]` and `center[0]`.
- `center[7]` indicates whether the current row is odd or even.
- `center[0]` indicates whether the current column is odd or even.
- Use the values of `center[7]` and `center[0]` to determine which color memory (red, green, or blue) the pixel data should be written to.
## `SETTING` state
```verilog=
SETTING: center <= 129;
```
- The first pixel to be interpolated is located at the first row and first column, (1, 1), corresponding to memory address 129.
## `RESET` state
```verilog=
RESET: begin
r_tmp <= 0;
g_tmp <= 0;
b_tmp <= 0;
counter <= 0;
wr_r <= 0;
wr_g <= 0;
wr_b <= 0;
end
```
- Reset all registers used during calculations, as well as the read/write signals, to prevent incorrect writes to memory.
## `GREEN_BLOCK1_GETDATA` state
```verilog=
GREEN_BLOCK1_GETDATA: begin
counter <= counter + 1;
// calculate red/blue memory address
if (counter<=1) begin
case (counter)
'd0: begin
addr_r <= {center[13:7], cx_minus1};
addr_b <= {cy_minus1, center[6:0]};
end
'd1: begin
addr_r <= {center[13:7], cx_add1};
addr_b <= {cy_add1, center[6:0]};
end
endcase
end
// bilinear interpolation
if (counter>=1) begin
r_tmp <= r_tmp + rdata_r;
b_tmp <= b_tmp + rdata_b;
end
end
```
- `GREEN_BLOCK1_GETDATA` represents the state where the required blue pixels are aligned in the same column and the required red pixels are aligned in the same row. In contrast, `GREEN_BLOCK2_GETDATA` aligns blue pixels in the same row and red pixels in the same column.
- The operation logic of the states `GREEN_BLOCK2_GETDATA`, `RED_BLOCK_GETDATA`, and `BLUE_BLOCK_GETDATA` are similar. Therefore, we use `GREEN_BLOCK1_GETDATA` as an example to explain the overall operation logic.
- When `counter` is `d0`, access the first pair of blue (above) and red (left) pixels. When `counter` is `d1`, access the second pair of blue (below) and red (right) pixels.
| cycle | counter | addr_r | addr_b | rdata_r/b available? | r_tmp / b_tmp update |
| :--------: | :--------: | :--------: | :--------: | :--------: | :--------: |
| 0 | 0 | Left coordinate | Qbove coordinate | $\times$ | $\times$ |
| 1 | 1 | Right coordinate | Below coordinate | First set of values | Accumulate first set of values |
| 2 | 2 | $\times$ | $\times$ | Second set of values | Accumulate second set of values |
## `WRITE_MEMORY` state
```verilog=
WRITE_MEMORY: begin
if (center[6:0]!=126) begin
center <= center + 1;
end else begin
center <= center + 3;
end
addr_r <= center;
addr_g <= center;
addr_b <= center;
case ({center[7], center[0]})
'b00, 'b11: begin
wr_r <= 1;
wr_g <= 0;
wr_b <= 1;
wdata_r <= r_tmp[8:1];
wdata_b <= b_tmp[8:1];
end
'b01: begin
wr_r <= 0;
wr_g <= 1;
wr_b <= 1;
wdata_g <= g_tmp[9:2];
wdata_b <= b_tmp[9:2];
end
'b10: begin
wr_r <= 1;
wr_g <= 1;
wr_b <= 0;
wdata_r <= r_tmp[9:2];
wdata_g <= g_tmp[9:2];
end
endcase
end
```
- Check if the x-index of the next pixel is not equal to 126. If it is not 126, it indicates that the current pixel is not the rightmost pixel in the row, so increment the `center` by 1. If it is 126, indicating the rightmost pixel in the row, increment `center` by 3 to move to the next row.
- Update the addresses for red, green, and blue memory (`addr_r`, `addr_g`, and `addr_b`) in lines 7–9.
- Determine which color channels should be written to based on the column (`center[7]`) and row (`center[0]`) indices::
- Enable or disable the write signals (`wr_r`, `wr_g`, `wr_b`).
- Assign the corresponding values to be written to memory (`wdata_r`, `wdata_g`, `wdata_b`).
## `RESULT` state
```verilog=
RESULT: begin
done <= 1;
end
```
- Set the `done` signal to 1 to indicate the operation is complete.
Sign in with Wallet
Connect another wallet