## Icebreaker 🧊
## Objectives🎯:
By the end of this lesson, you will:
1. Learn about the concepts of image sharpening.
2. Discuss various methods to mathematically enhance the contrast between a pixel and its neighboring pixels.
## Getting more information out of an image
Last time, we found out that the addition and multiplication approach was not a great strategy for reading the license plate. We also made changes to our addition code to account for the case when the value of the pixel exceeds 255.
This is how our image looked after handling this case:
It is better than the images we got when we didn't account for this case, but we still can't read the license plate.
**Discussion:** How else can we analyze the image to gather more information? Consider the drawbacks of our approach from yesterday (that is, only looking at one pixel at a time). Think about the relationships between the pixels.
<details>
<summary> <B>Think, then click</B> </summary>
Something that might help us is comparing the pixel to the pixels around it. That could help us determine which pixels are similar or different to those around them, and how different they are.
Other good answers:
- looking at groups of pixels
- looking for certain patterns in an image
</details>
<br>
**Discussion:** So, one such approach is comparing a pixel to the pixels directly next to it. What are some things we could do with that?
<details>
<summary> <B>Think, then click</B> </summary>
We can check whether pixels are similar or different from their neighbors. We can also change pixel values based on the values of their neighbors.
</details>
<br>
**Discussion:** If we set the value of the center pixel to the average value of the pixels surrounding it, what would this do?
<details>
<summary> <B>Think, then click!</B> </summary>
This makes the pixel's value closer to the value of its neighbors, or it makes the pixel more similar to its neighbors
</details>
<br>
### Exercise:
Let's practice this.
Here's a neighborhood of pixels:
In your breakout rooms, try to find the average of the pixels around (and including) the center pixel.
<details>
<summary> <B>Think, then click!</B> </summary>
(8 + 150 + 5 + 8 + 255 + 10 + 9 + 120 + 11) / 9 = 64
</details>
<br>
If we make the value of the pixel more similar to the value of its neighbors,the image would be blurry! Think about a simple case of an image with 2 pixels (let’s say their values are 0 and 255). If we take the average of the neighboring pixels, both pixels would end up with values of 127.
Also, think of this as what happens when you get further from an image. If we had 2 black and white pixels next to each other, and we squint or back away, they’ll end up looking gray.
Today, we are not going to focus on how to make images blurry, because that doesn't help us with our detective work. Does it make sense to blur the image in order to read the license plate?
**Discussion:** If our objective is't to make the pixels more similar to each other, what other approaches can we consider?
<details>
<summary> <B>Think, then click!</B> </summary>
we can make the value of the pixel more **different** to its neighbors!
</details>
<br>
**Discussion:** And if we do that, what would the image look like?
<details>
<summary> <B>Think, then click!</B> </summary>
It would make the image clearer. The differences in the values or color of the pixels are emphasized.
</details>
<br>
## Sharpening:
Sharpening helps to increase the contrast along the edges of objects, making them appear more distinct and clear. Sharpening works by increasing the difference between the pixels near the edges or boundaries of an image and the pixels in the surrounding areas.
**Discussion: What are some ways we could mathematically make a pixel more different from its neighbors?**
<details>
<summary> <B>Think, then click!</B> </summary>
One way we could do this is by subtracting the values of the surrounding pixels from the pixel we're looking at.
</details>
<br>
Let’s give this strategy a try to get a clearer image. Here’s a neighborhood of pixels, with i and j labeled:
To implement this we can start with our for loop from last time
```python=
result = np.zeros(img.shape)
for i in range(img.shape[0]):
for j in range(img.shape[1]):
# todo....
```
Okay, now we need to figure out how to access a pixel's neighborhood, or the pixels around it.
**Discuss: If we’re at row i and column j, what pixels do we need to look at in terms of i and j?**
<details>
<summary> <B>Think, then click!</B> </summary>
Rows/Columns labeled generically:
The pixels we need to look at are:
(i-1, j-1), (i-1, j), (i-1, j+1), (i, j-1), (i, j), (i, j+1), (i+1, j-1), (i+1, j), (i+1, j+1)
</details>
<br>
### Activity: Let’s code this!
Here is the Google Colab [link](https://colab.research.google.com/drive/1R4Bqr17th42dCS7fKqqry7FbxXJzNJ5v#scrollTo=Yl1Eux-vb6rB). As always, remember to make a copy!
Run this and see what it does!
```python=
def sharpen_img(img):
sharpened = np.zeros(img.shape)
for i in range(img.shape[0]):
for j in range(img.shape[1]):
total = 0
total -= img[i-1, j-1]
total -= img[i-1, j]
total -= img[i-1, j+1]
total -= img[i, j-1]
total += img[i, j]
total -= img[i, j+1]
total -= img[i+1, j-1]
total -= img[i+1, j]
total -= img[i+1, j+1]
sharpened[i,j] = total
return sharpened
```
**Discuss:** Guess what this error means
<details>
<summary> <B>Think, then click!</B> </summary>
We run into IndexError :(
What does this mean?
Basically, our picture only has 1608 pixels in a column, but we're trying to access the 1609th pixel (because our array starts at 0).
</details>
<br>
**Discuss: How can we prevent this error occurring while still having our code work for the pixels on the edges?**
<details>
<summary> <B>Think, then click!</B> </summary>
One way we could do this is choose to use some default color if we have an index that's not in the array. For instance, we could use black for any of these cases.
Another possible solution we could do is if we have an index that we can't use, use the closest valid index instead. So, for example, if we tried to access index -1, we could use 0 instead. Similarly, if we used 1608, we could use 1607 instead. There are also a bunch more ways to approach this (you'll learn if you take computer vision!).
A note: We also need to prevent our program from accessing negative numbered elements, because even though it won't error, it will give us a different pixel than the one we want. This is a python thing. It has something to do with how python treats negative array indexes.
</details>
<br>
Let's give implementing this solution a try. Let's try and write a "safe" function to get a pixel's value (that is, one that will never give us a value that will error.
**Exercise:** Write a function that will either return the value of the pixel at the specified index. If the specified index is not valid or out of bounds, return the value of a black pixel. Here's the function definition:
```python=
def safe_get(row, col, img):
## todo
```
<details>
<summary> <B>Solution</B> </summary>
Here's one way to do that:
```python=
def safe_get(row, col, img):
if row >= 0 and row < img.shape[0] and col >= 0 and col < img.shape[1]:
return img[row, col]
else:
return 0
```
</details>
<br>
Great! Now let's use our safe_get function any time we previously just looked up an array index...
```python=
def sharpen_img(img):
sharpened = np.zeros(img.shape)
for i in range(img.shape[0]):
for j in range(img.shape[1]):
total = 0
total -= safe_get(i-1, j-1, img)
total -= safe_get(i-1, j, img)
total -= safe_get(i-1, j+1, img)
total -= safe_get(i, j-1, img)
total += safe_get(i, j, img)
total -= safe_get(i, j+1, img)
total -= safe_get(i+1, j-1, img)
total -= safe_get(i+1, j, img)
total -= safe_get(i+1, j+1, img)
sharpened[i,j] = total
return sharpened
```
Let's go back to our neighborhood of pixels.
Let’s multiply the center pixel by 9 and subtract the value of every other pixel.
<details>
<summary> <B>Think, then click!</B> </summary>
128*9 - (111 + 108 + 114 + 110 + 123 + 103 + 102 + 122) = 259
</details>
<br>
Is there something wrong with this value?
It’s greater than 255, so as we found out last week, it’s gonna look a little weird if we don’t prevent it from going above 255
What other case do we want to prevent?
<details>
<summary> <B>Think, then click!</B> </summary>
Less than 0
</details>
<br>
What should we do if we get numbers that are greater than 255 or less than 0? Think back to what we did for values that were too bright.
<details>
<summary> <B>Think, then click!</B> </summary>
If a pixel's final value is greater than 255, set it equal to 255. If a pixel's final value is less than 0, set it equal to 0.
</details>
<br>
Okay, now let's try to account for this case in code:
```python=
if total > 255:
total = 255
elif total < 0:
total = 0
```
And update our sharpen_img function code:
```python=
def sharpen_img(img):
sharpened = np.zeros(img.shape)
for i in range(img.shape[0]):
for j in range(img.shape[1]):
total = 0
total -= safe_get(i-1, j-1, img)
total -= safe_get(i-1, j, img)
total -= safe_get(i-1, j+1, img)
total -= safe_get(i, j-1, img)
total += safe_get(i, j, img) * 9
total -= safe_get(i, j+1, img)
total -= safe_get(i+1, j-1, img)
total -= safe_get(i+1, j, img)
total -= safe_get(i+1, j+1, img)
# prevent values from being over 255 or less than 0
if total > 255:
total = 255
elif total < 0:
total = 0
sharpened[i,j] = total
return sharpened
```
Here is what we get after we run it:
Here’s the original for comparison:
What do we notice? It seems more defined overall
Do we think this strategy is sufficient?
What needs to change in order to decipher the license plate? We don't really need the same image but sharper, we want a more focused view of the license plate. Is there a way we could make the pixel values even more different than its surrounding pixels?
Sign in with Wallet
Connect another wallet