CS180 proj 2

The goal of this project is to use a variety of mathematical operations to edit images to achieve our desired effect. This can include sharpening or blurring images, merging images with different frequencies, or blending images along arbitrary borders.

Part 1: Fun with Frequencies

Finite Difference Operator

To begin, we first make use of the finite difference operators Dx and Dy to find partial derivatives of an image in the x and y direction. To do this, we concolve the image (in the form of an ndarray) with the finite difference operator. Note that if we are working with colored images, we might have to convolve channel by channel, depending on how the image is stored.

cameraman_og — The original `cameraman.png`

cameraman_x — The partial derivative of `cameraman.png` with respect to `Dx`

cameraman_y — The partial derivative of `cameraman.png` with respect to `Dy`

Now that we have both partial derivatives, we can use them to calculate the gradient magnitude of the image. After that, with some trial and error, we can find an arbitrary threshold to binarize the gradient magnitude, giving us the edges that are most prominent in the original image.

cameraman_grad — The gradient as calculated from the partial derivatives

binarized_gradient — The binarized result, with the threshold being 50

We see that the binarized result is technically a valid set of edges, but it is quite noisy. To improve upon this, we can use a smoothing operator to get rid of the noise in the original image first. This is where the Gaussian filter comes in.

Derivative of Gaussian (DoG) Filter

To smooth the image, we can convolve the original with a gaussian filter G , and repeat the procedure in the previous part. This should give us clearer edges in our binarized gradient image.

cameraman_blurred — The original `cameraman.png` convolved with a gaussian filter

If we follow the same procedure again to calculate the gaussian magnitude and the binazired gaussian, we see that the resulting image has much clearer edges and much less noise. The magnitude of the gaussian also dropped, since we have less pixels with extreme values after we smoothed the image.

An interesting property of this type of opertation is that since it is matrix algebra at its core, many of its steps are commutative and associative. What this means in context is that instead of convolving the original image with Dx and Dy and calculating the gradient with the two results, we can convolve them with the gaussian filter first, then use that to directly obtain the smoothed partial derivative. In many cases, this can save time since we're doing matrix operations on a much smaller scale.

gauss_dx — The gaussian filter convolved with the `Dx` finite difference operator

gauss_dy — The gaussian filter convolved with the `Dy` finite difference operator

dog_x — The partial derivative obtained from the above DoG. Note it's the same as the original partial derivative

dog_y — The partial derivative obtained from the above DoG. Note it's the same as the original partial derivative

dog_grad — The gradient magnitude from the partial derivatives above. Again, this is the same as the one from the original order of operations

bin_dog_grad — The binarized result of the gradient magnitude to the left

Part 2: Fun with Frequencies

Image Sharpening

Moving on to frequency manipulation, one way we can edit images through its frequencies is to "sharpen" it. By using a low pass Gaussian filter on an image, and subtracting that from the original, we have the image with all of its high frequencies retained. This usually looks like all the edges in an image. By finding the edges in an image, then adding more emphasis on those edges, we can create a result where the details and differences between colors are more pronounced. The magnitude of the sharpening effect is determined by alpha, which is a measure of just how much we're adding the edges back to the image.

taj_sharpened — The sharpened version of `taj.jpg`, with an alpha of 2.

Here are some other images I tried sharpening (zoom in to see detailed differences):

taj — The sharpened version of the image, with an alpha of 5

Hybrid Images

Another result of frequency manipulation is the creation of hybrid images. If we stack the low frequencies of one image with the high frequencies of another image, we can create a hybrid image. Since we are sensitive to different frequencies at different distances, this means we will be able to see the high frequency image more clearly when we are closer up, and the low frequency image when we are farther away.

derek_nutmeg — The hybrid image of the two, with Nutmeg being in the lower frequencies and Derek in the higher frequencies

happy_angry — The hybrid image of the two

cat_borzoi — The hybrid image of the two

Fourier transform of the original cat image

Fourier transform of the original borzoi image

Fourier transform of the gaussian filtered cat image

cat_borz — Fourier transform of the final hybrid image

Here's an example of a failed hybrid image. I believe the primary reason of the failure is the difference in size of the two images, as much of the detail was lost when the two images were aligned. This means the filter cannot capture enough information for the images to be distinguishable.

Multi-resolution Blending and the Oraple journey

Gaussian and Laplacian Stacks

Moving on from hybrid images, frequency manipulation can also be used to selectively merge different sections of images. This is done with multi-resolution blending, where we apply successive layers of gaussian and laplacian filters to the original image.

cat — The oraple padded with blank space

Learnings

Overall, this is a very fun and rewarding project. Seeing images being processed and blended/mixed/sharpened is incredibly interesting, and I got to create some funny results as well. I think the most important learning comes in the form of my deepened understanding of the relationship between images and frequencies. Before this, I always knew they are somehow related, but I never thought about the details. Now that I've done this, I can confidently say that I have a much better grasp on how images are stored, edited, and displayed.

CS 180 Project 2: Fun with Filters and Frequencies

Kevin Sheng

Project Overview