Title: Apply a fisheye transformation to images in Python

[An image warped by a fisheye transformation]

PIL provides a few methods for transforming images geometrically. For example, it lets you resize, rotate, and crop images. It even lets you apply some kinds of transformation matrices. If you want to do something more complicated, you need to step outside of PIL's methods.

This example shows how you can use NumPy to apply a fisheye transformation to an image. You can use similar techniques to apply other unusual transformation s.

Using the Program

To use the program, open the File menu, select Open, and pick a file. The program scales the image so it fits on the window, but it works with the image at full scale.

Left-click on the image to set the fisheye's center point. Or left-click and drag to move the center point around. I've found dragging the mouse lets you fine tune the fisheye to get the best possible result.

When you like the result, use the File menu's Save As command to save the result.

Applying a Fisheye

The following apply_fisheye function shown in the following code does all the heavy lifting.

from PIL import Image import numpy as np from scipy.ndimage import map_coordinates def apply_fisheye(image, cx, cy, strength=1.5): '''Apply a radial fisheye distortion to an image.''' # strength: > 1.0 for fisheye, < 1.0 for pincushion. # Convert the image to a NumPy array. image_array = np.array(image) # Create a grid of target pixel coordinates (y, x). hgt, wid, channels = image_array.shape max_radius = np.sqrt((wid/2)**2 + (hgt/2)**2) y_indices, x_indices = np.indices((hgt, wid)) # Translate to put (cx, cy) at the origin (0, 0). x_shifted = x_indices - cx y_shifted = y_indices - cy # Convert to polar coordinates. r = np.sqrt(x_shifted**2 + y_shifted**2) theta = np.arctan2(y_shifted, x_shifted) # Normalize the radius to the 0.0 - 1.0 range. r_norm = r / max_radius # Apply the non-linear fisheye distortion formula. # Power-law formula: r_source = r_target ^ strength r_distorted_norm = np.power(r_norm, strength) r_distorted = r_distorted_norm * max_radius # Convert back to Cartesian coordinates. x_source = cx + r_distorted * np.cos(theta) y_source = cy + r_distorted * np.sin(theta) # Reconstruct the image channels using interpolation. output_array = np.zeros_like(image_array) for c in range(channels): output_array[:, :, c] = map_coordinates( image_array[:, :, c], [y_source, x_source], order=3, # Cubic spline interpolation for high quality mode='constant', # Fill black colors outside bounds cval=0, ) # Convert the result to a PIL image. return Image.fromarray(output_array)

The code first converts the image into a NumPy array. The np.array call converts the image into a three-dimensional array where the first dimension gives a pixel's Y coordinate, the second gives its X coordinate, and the third gives the pixel's red, green, or blue color component. (If the image has transparency, the third component will have a fourth alpha value.)

The code then gets the array's shape (height, width, and color depth) and calculates a radius to bound the effect.

The call to np.indices gets arrays holding the X and Y coordinates of the pixels in the array. For example, if a pixel has coordinates (x, y), then x_indices[x][y] is x and y_indices[x][y] is y. Yes, this is weird, but the function doesn't really move the pixels, it moves their positions. NumPy then uses the modified positions to rearrange the pixels.

Next, the code subtracts the fisheye's center position (cx, cy) from the indices to translate the image so the center position is at the origin (0, 0). Notice that this code and the next few lines are working with arrays not simple numbers. For example, x_shifted = x_indices - cx subtracts cx from every entry in the x_indices array and returns a new array holding the result.

The code then converts the pixel positions to polar coordinates. It squares each item in the arrays, adds the result arrays, and calculates the square roots of every item. The result is a new array r holding the distances from every pixel's shifted position to the origin.

The function uses np.arctan2 to get each pixel's angle and then normalizes the r values to determine the maximum area of effect.

The code then raises the normalized radii to the power of strength. Because the radii are normalized, they are smaller than 1. When raised to a power greater than 1, that makes them smaller. For example, 0.1 ^ 2 = 0.1 * 0.1 = 0.001.

Conversely, if you raise a value smaller than 1 to a power less than one, the result is bigger. For example, 0.1 ^ 0.5 = Sqrt(0.1) ≈ 0.316.

The result is the opposite of a fisheye effect. Points close to the origin are moved closer to it and points farther from the origin are moved farther away.

After distorting the radius values, the code multiplies them by max_radius to convert the normalized values back into reasonable coordinates.

The function then converts the coordinate values from polar coordinates back to Cartesian coordinates.

Remember that all of these operations are on the indices of the pixels x_indices and y_indices. Now the function needs to use the modified pixel positions to create the new image.

To do that, it first creates an array filled with zeros that's the same size as the image array.

Here's where it gets weird. For each channel (red, green, blue, and possibly alpha), the code calls scipy.ndimage.map_coordinates to map the transformed coordinates in y_source and x_source back to the original coordinates. For each new coordinate (x, y), map_coordinates figures out where that position is in the input array image_array. It then uses interpolation to find a value for the output position.

For example, suppose a particular (x, y) coordinate is (2.5, 10.2). Then the output value is a weighted average of the values around that position at positions (2, 10), (2, 11), (3, 10, and (3, 11).

It's the mapping back from the final coordinates to the original coordinates that creates the warping. The mathematical transformations moved points close to the origin (cx, cy) so they are closer to the origin. That means map_coordinates maps them closely together in the original image so many output pixels map to closely spaced input pixels, and that gives the image the fisheye effect.

Having built the output array holding the color coordinate values, the function calls Image.fromarray to convert the values into an image and returns the image.

Calling apply_fisheye

When you press the left mouse button down on the image, or when you press the button and move the mouse, the following code executes.

def mouse_down(self, event): '''Apply fisheye at this point.''' x = event.x / self.scale y = event.y / self.scale self.warped_image = apply_fisheye(self.original_image, x, y, self.strength_var.get()) self.display_image()

This code gets the mouse's current position and divides by the current scale factor to find the coordinates on the unscaled image. It then calls apply_fisheye passing it the original image, scaled mouse coordinates, and the strength value set by the program's Scale widget. It then calls self.dispay_image to show the result. That method isn't part of the image processing code so it isn't described here.

Conclusion

This example is actually pretty fun to use! Load a picture and drag the mouse around to see what kinds of effects you can produce.

My computer has a very slight lag when working with a 400×400 pixel image, but it's still quite usable. You may experience a larger lag with bigger images. You'll just have to give it a try and find out.

Download the example to experiment with it and to see additional details.

[An image warped by a fisheye transformation]