Title: Use image filters to emboss images in Python

[A picture of Terry Pratchett after applying an embossing filter in Python]

Image filters let you perform operations on the pixels in an image. In one kind of image filter, you have a two-dimensional array (list of lists) of numbers called the filter's kernel. For each pixel in the image, you center the kernel over that pixel. You then multiply the value of each pixel under the kernel by the corresponding kernel value. You add them up, divide by a kernel's weight value, and optionally add an offset to make the result look nicer. (For example, embossing filters tend to make the result very dark. You can add an offset of 127 to move the result to a mostly neutral value.) The result of those calculations gives you the new value for the center pixel.

To handle color, simply treat the red, green, and blue color components separately.

This example demonstrates two embossing filters and later examples will add more kinds of filters.

Note that the ImageFilter module can apply filters to images so you don't need to build your own filters if you don't feel like it. Building your own isn't too hard, though. It's also fun and interesting, hence this post.

Constructor

The Filter class represents filters. Here's the class's constructor.

class Filter: '''A public class to represent filters.''' def __init__(self, kernel, weight=None, offset=0): # Make sure the kernel has an odd number of rows and columns. if len(kernel) % 2 == 0: raise ValueError('The kernel must have an odd number of rows') if len(kernel[0]) % 2 == 0: raise ValueError('The kernel must have an odd number of columns') self.kernel = kernel if weight is None: self.normalize() else: self.weight = weight self.offset = offset self.num_rows = len(kernel) self.num_cols = len(kernel[0]) self.mid_y = self.num_rows // 2 self.mid_x = self.num_cols // 2

The constructor takes the filter's kernel array, weight, and offset as parameters. It first verifies that the kernel has an odd number of rows and columns. When we apply a kernel, we center it over a pixel and we can't do that if the kernel doesn't have a central value.

If the kernel has an odd number of dimensions, the constructor just saves the kernel, weight, and offset. If the weight parameter is missing, the constructor calls the normalize method described shortly to make the weight equal the sum of the kernel's values.

Helper Methods

The next piece of the class defines two short helper methods.

def get_total(self): '''Return the kernel's total weight.''' total = 0 for row in self.kernel: total += sum(row) return total def normalize(self): '''Set the filter's weight equal to the sum of the kernel's values.''' self.weight = self.get_total()

The get_total method returns the sum of the kernel's values.

The normalize method "normalizes" the filter by setting the kernel's weight equal to the sum of the kernel's values. (When you apply a normalized kernel to an image, the weight sort of averages the filter values so the result is a weighted average of the pixel values around the central pixel.)

apply

The most interesting piece of the Filter class is the apply method, which applies the filter to an image.

This method is mostly an exercise in bookkeeping, but there's one small catch: you can't directly modify the image. When you apply the filter to a pixel, you apply the kernel to the pixels around the target pixel. If you change the target pixel's value in the original image, then the new value will be used when you process the adjacent pixels. That doesn't work because we need to use the original pixel values when apply the kernel.

To handle that, the apply method makes a copy of the image to hold the result.

Here's the method's code.

def apply(self, image): '''Apply the filter to a PIL image.''' # Get the pixels. pixels = image.load() # Make a copy. result_image = image.copy() # Fill the image with a default color if desired. dr = ImageDraw.Draw(result_image) dr.rectangle((0, 0, result_image.width, result_image.height), fill='white') # Load the result pixels. result_pixels = result_image.load() # Find the number of rows and columns, and the middle row and column. num_rows = len(self.kernel) num_cols = len(self.kernel[0]) mid_y = num_rows // 2 mid_x = num_cols // 2 # Get bounds for pixels to which we can apply the kernel. min_y = mid_y max_y = image.height - mid_y min_x = mid_x max_x = image.width - mid_x # Apply the filter. for y in range(min_y, max_y): for x in range(min_x, max_x): # Apply the filters to pixel[x, y] # Process the pixels under the kernel. red = green = blue = 0 for dy in range(num_rows): pix_y = y + dy - mid_y for dx in range(num_cols): pix_x = x + dx - mid_x pix_red, pix_green, pix_blue = pixels[pix_x, pix_y] red += self.kernel[dy][dx] * pix_red green += self.kernel[dy][dx] * pix_green blue += self.kernel[dy][dx] * pix_blue # Divide by the weight, add the offset, and # make sure the result is between 0 and 255. red = int(self.offset + red / self.weight) red = max(min(red, 255), 0) green = int(self.offset + green / self.weight) green = max(min(green, 255), 0) blue = int(self.offset + blue / self.weight) blue = max(min(blue, 255), 0) # Set the new pixel's value. result_pixels[x, y] = (red, green, blue) # Return the result. return result_image

The method begins by loading the image so it can work with its pixels quickly. It then makes a copy of the image to hold the result, fills the copy with white, and loads that image so we can access its pixels quickly, too.

Next, the code finds the number of rows and columns in the kernel and calculates the middle row and column numbers.

It's not obvious how to handle pixels that lie too close to the edges of the image because, when you center the kernel over those pixels, some of the kernel's values fall outside the image. For example, if you center the kernel over the image's upper left pixel, the kernel extends above and to the left of the image. You can handle this in a few ways such as extending the edge pixel values as far as needed outside the image. No matter how you handle this, the edge pixels won't have perfectly "correct" values, so for this example we just won't calculate values for those pixels. To do that, the code calculates the minimum and maximum row and column values that we need to process.

Finally, the code applies the filter. The outer for y and for x loops traverse the image's rows and columns of pixels.

Inside those loops, for the pixel at position [x, y], the code applies the kernel. It first initializes variables red, green, and blue to 0. It then loops through the rows and columns in the kernel.

For each kernel position, the method gets the corresponding pixel's color components, multiplies them by their kernel values, and adds the results to the red, green, and blue totals.

After it adds up the pixel values times the kernel values, the code divides each color total by the filter's weight and adds the offset. It then saves those values in the result image's corresponding pixel.

After it has processed every pixel, the method returns the result image.

Filters

This example's Filter class uses the following factory methods to provide two embossing filters.

# Filters. @classmethod def embossing_filter1(cls): '''Create a standard embossing filter.''' return cls( [ [-1, 0, 0], [0, 0, 0], [0, 0, 1], ], weight=1, offset=127) @classmethod def embossing_filter2(cls): '''Create a standard embossing filter.''' return cls( [ [2, 0, 0], [0, -1, 0], [0, 0, -1], ], weight=1, offset=127)

The embossing_filter1 method returns a kernel with the following values.

-1 0 0 0 0 0 0 0 1

The sum of the kernel weights is zero so the kernel tends to drive roughly equal pixels values toward zero, which produces a black output pixel. The weight is 1, so the result value isn't scaled. Finally, the offset is 127 so the value is moved from something near black to a more neutral color near 127.

Because the kernel's non-zero values are in opposite corners, the result is an embossing filter. The picture at the top of this post shows the result if this filter. This filter has a negative value in its upper left corner and positive value in its lower right corner, so objects tend to look as if illuminated from the lower right as shown in the picture.

[Different embossing filters produce different results] The embossing_filter2 method returns a kernel with the following values.

2 0 0 0 -1 0 0 0 -1

Unlike the first filter, this new filter's positive and negative values are in the upper left and lower right, respectively, so the result makes objects appear illuminated from the upper left as shown in the picture on the right.

Main Program

Now that we have a Filter class, how do you use it? Here's how the example program applies the embossing_filter1 filter.

def mnu_emboss1(self): '''Apply an embossing filter.''' self.pil_image = Filter.embossing_filter1().apply(self.pil_image) self.show_result()

This code first calls the Filter.embossing_filter1 factory method to get the filter. It then calls that filter's apply method to apply it to the image loaded into the self.pil_image property. It then calls the following show_result method to display the result.

def show_result(self): '''Display the new image on the canvas.''' self.tk_image = ImageTk.PhotoImage(self.pil_image) self.canvas.delete(tk.ALL) self.canvas.create_image(0, 0, anchor=tk.NW, image=self.tk_image)

This method simply converts self.pil_image into a PhotoImage that tkinter can understand and displays it on the program's Canvas widget.

Conclusion

Download the example to see additional details and to experiment with it. For example, instead of making unprocessed edge pixels white, you could trim the result to remove those pixels.

Feel free to experiment with other kernels. For example, try some bigger kernels like this one:

0 0 0 0 2 0 0 0 2 0 0 0 0 0 0 0 -2 0 0 0 -2 0 0 0 0

Or try some non-square kernels like this one:

0 0 2 0 0 2 0 0 2 0 0 2 0 0 0 -2 0 0 -2 0 0 -2 0 0 -2 0 0

Or just make up your own kernels to see what they do. I'll describe some more filters in future posts

Note: Processing every pixel in an image can take a while, particularly if the kernel is large. For example, a 1,000×1,000 pixel image contains 1 million pixels. If you use a 5×5 kernel, you need to perform around 75 million calculations, 25 million for each of the red, green, and blue color components. That takes around 10 seconds on my computer.

The moral is to start with small images until you tweak your kernel to produce the result you want. Then you can process larger images with the perfected kernel.