Title: Overlay a blended image on a background image with Python and PIL

[The program overlaying the eye of Sauron on top of a face]

The idea behind this program is fairly straightforward.

The program uses a grayscale mask image that is white in the middle and gradually shades to black at the edges.
It then pastes the foreground image onto the background image.
- Pixels corresponding to white mask pixels are copied from the foreground image.
- Pixels corresponding to black mask pixels keep the background pixel's value.
- Pixels corresponding to gray mask pixels get a weighted average of the background and foreground images.

Easy peasy!

At least the idea is easy. I'm not gonna lie, the implementation is fairly long (about 350 lines) and rather complicated. Because it's so long, I don't want to waste your time describing every single line of code. You can download the example program and look at the code yourself. Instead I'm going to focus on two of the more interesting parts of the program:

The user interface code that lets you move and resize the foreground image
The code that composes the images

User Interface

Unfortunately, this example's user interface is fairly complicated so I don't want to describe all of the code in detail. Instead I'll explain what it does and how it works. Then you can download the example to see the code for yourself.

Use the File menu's Load Background Image and Load Foreground Image commands to load the images. After you load the background image, the program displays it. After you load both images, the program draws the foreground image on top of the background image.

The program also draws drag handles around the foreground image so you can easily resize it. When you drag a handle, the handle on the opposite side of the image remains fixed and the image change size accordingly. (If that's not clear, give it a try. It's pretty intuitive to use.)

Click and drag on the body of the foreground image to move the image without resizing it.

As you resize the foreground image, the program ensures that it keeps its original aspect ratio so you don't stretch it out of shape.

(Code note: The program stores the drag handle positions in the drag_handles dictionary. It uses the names nw, n, ne, e, se, s, sw, and w as the keys into the dictionary and their values are the handle's locations on the background image.

The code that manages the drag handles and the code that preserves the image's aspect ratio are kind of interesting so I'll describe them a bit more shortly.

After you drag the foreground image into a position that you like, use the File menu's Save As command to save the composited result.

That's how you use the program. Now let's go back and talk about the code that manages the drag handles and that preserves the image's aspect ratio.

mouse_move_when_up

When you move the mouse without pressing its button, the following mouse_move_when_up event handler executes.

def mouse_move_when_up(self, event): '''Set the appropriate cursor.''' # See if the mouse is over a drag handle. self.over_handle = self.mouse_is_over(event.x, event.y) if self.over_handle is None: # Not over anything. Default cursor. self.canvas.config(cursor='') elif self.over_handle == 'body': # Over foreground image body. Size cursor. (Only in Windows?) self.canvas.config(cursor='size') else: # Over a drag handle. Tcorss cursor. self.canvas.config(cursor='tcross')

This code does three things. First, it calls mouse_is_over to see what the mouse is over. That method returns the name of a grab handle (nw, n, ne, etc.) if the mouse is over a grab handle. If the mouse is over the foreground image's body, it returns body. If the mouse isn't over any of those things, mouse_is_over returns None.

After calling mouse_is_over, the mouse_move_when_up event handler displays an appropriate cursor.

mouse_move_when_down

When you move the mouse while the mouse button is pressed, the following event handler updates the foreground image's size and position.

def mouse_move_when_down(self, event): '''Move or resize the foreground image.''' # If the mouse didn't start over something, do nothing. if self.over_handle is None: return # See what we started over. if self.over_handle == 'body': # We started over the foreground image. Move it. # See how far the mouse moved. dx = event.x - self.start_point[0] dy = event.y - self.start_point[1] # Update the bbox. self.fg_bbox = [ self.fg_bbox[0] + dx, self.fg_bbox[1] + dy, self.fg_bbox[2] + dx, self.fg_bbox[3] + dy, ] # Save the mouse's current position. self.start_point = (event.x, event.y) else: # It's a drag handle drag. # See if it's a corner drag. if self.over_handle in ['nw', 'ne', 'se', 'sw']: # It's a corner drag. Calculate the new width and height. new_wid = abs(self.opposite_point[0] - event.x) new_hgt = abs(self.opposite_point[1] - event.y) new_wid, new_hgt = adjust_dimensions(new_wid, new_hgt, self.fg_aspect_ratio) # Update the appropriate corner. if 'n' in self.over_handle: self.fg_bbox[1] = self.fg_bbox[3] - new_hgt else: self.fg_bbox[3] = self.fg_bbox[1] + new_hgt if 'w' in self.over_handle: self.fg_bbox[0] = self.fg_bbox[2] - new_wid else: self.fg_bbox[2] = self.fg_bbox[0] + new_wid elif self.over_handle == 'w' or self.over_handle == 'e': # It's an east/west side drag. # Calculate the new width and height. new_wid = abs(self.opposite_point[0] - event.x) if new_wid < 10: new_wid = 10 new_hgt = new_wid / self.fg_aspect_ratio # Update the corners. if self.over_handle == 'w': self.fg_bbox[0] = self.fg_bbox[2] - new_wid else: self.fg_bbox[2] = self.fg_bbox[0] + new_wid y = (self.fg_bbox[1] + self.fg_bbox[3]) / 2 self.fg_bbox[1] = y - new_hgt / 2 self.fg_bbox[3] = y + new_hgt / 2 else: # It's a north/south side drag. # Calculate the new width and height. new_hgt = abs(self.opposite_point[1] - event.y) if new_hgt < 10: new_hgt = 10 new_wid = new_hgt * self.fg_aspect_ratio # Update the corners. if self.over_handle == 'n': self.fg_bbox[1] = self.fg_bbox[3] - new_hgt else: self.fg_bbox[3] = self.fg_bbox[1] + new_hgt x = (self.fg_bbox[0] + self.fg_bbox[2]) / 2 self.fg_bbox[0] = x - new_wid / 2 self.fg_bbox[2] = x + new_wid / 2 self.set_drag_handles() self.show_composite()

The code first checks self.over_handle to see what was under the mouse when you pressed the mouse button. If you didn't press the mouse on a drag handle or the foreground image, the method simply returns. If the mouse was over something interesting, the code determines what that was.

If the mouse started over the foreground image's body, then you are trying to move it. In that case, the code calculates the distance the mouse has moved since the last time this method was called and updates the fg_bbox list that records the image's minimum and maximum X and Y coordinates. It then updates start_point to the mouse's current position so this method can perform the same update the next time you move the mouse.

If the mouse started over one of the drag handles, there are two cases: the mouse was over a corner handle or the mouse was over a side handle.

If the mouse was over a corner handle, the method calculates the vertical and horizontal distance between the mouse's current position and the location of the drag handle that is opposite the one where you pressed the mouse button. For example, if you pressed the mouse button down while over the nw drag handle, then opposite_point is the location of the se drag handle.

The code then calls adjust_dimensions (described shortly) to ensure the new width and height maintain the foreground image's original aspect ratio.

Depending on which handle you're dragging, the code then updates the X and Y coordinates of the foreground image's bounding box fg_bbox. For example, if you're dragging the sw handle, the code updates the maximum Y coordinate and the minimum X coordinate.

The last case occurs when you're dragging one of the side handles. That leads to two sub-cases: you're dragging an east/west handle or you're dragging a north/south handle.

If you're dragging the west or east handle, the code calculates the horizontal distance between the mouse and the opposite_point and uses it as the image's new width. It calculates the corresponding height to preserve the image's aspect ratio and then updates the appropriate bounding box entries.

The code for dragging the north or south handle works similarly.

After all of that work, the code calls set_drag_handles, which uses the bounding box to position the drag handles. The method finishes by calling show_composite to display the composited image.

adjust_dimensions

The adjust_dimensions method shown in the following code adjusts a width and height to maintain a given aspect ratio.

def adjust_dimensions(width, height, aspect_ratio): '''Adjust the width and height so they have the given aspect ratio.''' # Make sure we're not too small. if width < 10: width = 10 if height < 10: height = 10 # Adjust for the aspect ratio. if width / height > aspect_ratio: # Too short and wide. Increase the height. height = width / aspect_ratio else: # Too tall and thin. Inrcrease the width. width = height * aspect_ratio return width, height

First, the code ensures that the width and height are at least 10 so you don't make the image too small to be useful. It then compares the with-to-height ratio to the desired aspect ratio. If the new dimensions are too short and wide, the code increases the height. If the new dimensions are too tall and thin, the code increases the width.

Those are the most interesting pieces of the code that manages drag handles, moving the foreground image, and resizing that image. The next big piece of code draws the composited image.

Image Drawing

The following sub-sections describe the main parts of the program that draw the composited image.

show_composite

The following show_composite method composites the images and displays the result.

def show_composite(self): '''Display the image.''' # Get the result image. self.result_image = self.make_composite(True) if self.result_image is None: return # Display the result. self.result_photo_image = ImageTk.PhotoImage(self.result_image) self.canvas.delete(tk.ALL) self.canvas.create_image(0, 0, anchor=tk.NW, image=self.result_photo_image)

This code calls make_composite (described next) to combine the images. It then displays the result on the program's Canvas widget.

make_composite

The following code shows the make_composite, which does most of the interesting work.

def make_composite(self, draw_drag_handles): '''Build the composite image.''' # If we don't have a background image, do nothing. if self.bg_image is None: return None # Start with the background image. result_image = self.bg_image.copy() # See if we also have the foreground image. if self.fg_image is not None: # Enable the Save As command. self.file_menu.entryconfig('Save As...', state='normal') # Resize the foreground image. wid = round(self.fg_bbox[2] - self.fg_bbox[0]) hgt = round(self.fg_bbox[3] - self.fg_bbox[1]) fg_image = self.fg_image.resize((wid, hgt)) # Make the mask. mask = make_mask(wid, hgt, 2, 0.5) mask.save('mask.png') # Paste the foreground image onto the background. corner = [round(self.fg_bbox[0]), round(self.fg_bbox[1])] result_image.paste(fg_image, box=corner, mask=mask) # Make drag handles. if draw_drag_handles: dr = ImageDraw.Draw(result_image) for point in self.drag_handles.values(): draw_box(dr, point, self.radius, 'white', 'black') # Return the result. return result_image

After checking that we have a background image, the code copies it so we can mess modify it without messing up the original.

The code then checks whether we have a foreground image. If the user has not yet loaded a foreground image, the method doesn't modify the copied background image so it will return it without changes.

If we do have a foreground image loaded, the code enables the Save As command. It then resizes the foreground image and saves the result in variable fg_image.

The code then calls make_mask (described next) to make a mask image and uses it to paste the foreground image onto the background image.

Next, if the draw_drag_handles parameter is True, the code draws the drag handles. (When the program is displaying the images, it draws drag handles. When you save the result into a file, the result does not include the drag handles.)

The method finishes by returning the modified image.

make_mask

The final piece of code I want to talk about is the following make_mask method.

def make_mask(wid, hgt, inner_fraction): '''Make an elliptical grayscale mask fading from white to black.''' # Make the mask image 8-bit color starting black. mask = Image.new(size=(wid, hgt), mode='L', color='black') dr = ImageDraw.Draw(mask) # Fill the mask with ellipses. cx = wid / 2 # Center point. cy = hgt / 2 rx1 = cx # Largest ellipse radii. ry1 = cy rx2 = rx1 * inner_fraction # Smallest ellipse radii. ry2 = ry1 * inner_fraction num_steps = math.ceil(max(rx1 - rx2, ry1 - ry2)) dx = (rx2 - rx1) / num_steps # Amounts to change the radii for each step. dy = (ry2 - ry1) / num_steps dc = 255 / num_steps # Amount to change color for each step. rx = rx1 # The current ellipse radii. ry = ry1 color = 0 for step in range(num_steps): bbox = (cx - rx, cy - ry, cx + rx, cy + ry) dr.ellipse(bbox, fill=round(color)) rx += dx ry += dy color += dc # Return the result. return mask

This method creates a grayscale image that is initially filled with black. It then fills the image with increasingly smaller ellipses that have the same aspect ratio as the image.

It starts with the color 0 (black) and an ellipse with width and height equal to those of the image. It fills that ellipse and then decreases the ellipse's width and height slightly, makes the color a little brighter, and repeats.

The code repeats until the ellipse has width and height equal to inner_fraction times the mask's width and height. The inner_fraction value represents the fraction of the mask that should be white so the foreground image is fully opaque. For example, if the foreground image is 150 %times; 100 pixels and inner_fraction is 0.5 (which it is in this program), then the all-white ellipse is 75 %times; 50 pixels.

After it finishes drawing the increasingly white ellipses, the method returns the result.

Conclusion

I've left out a lot of code in this post. Download the example program to see additional details and to experiment with it.

The pictures below show a background image, foreground image, and a mask image created by the program. Notice how the mask image is black at the edges and shades to white in the middle. The mask image is the size of the resized foreground image when I dropped it onto the background.

I ran the program, placed the foreground image over the person's left eye, and saved the result. I then loaded that result and dropped another copy of the foreground image over the person's right eye. The following picture shows the final result. (Creepy enough for you? 😉) [The final blended composite image]

For more information image processing in Python, see my Manning liveProject Algorithm Projects with Python: Image Processing. It explains how to do things like:
• Rotation	• Scaling	• Stretching
• Brightness enhancement	• Contrast enhancement	• Cropping
• Remapping colors	• Image sharpening	• Embossing
• Color enhancement	• Sharpening	• Gray scale
• Black and white	• Sepia tone	• Other color scales