Awesome pictures fusing with a GIMP plugin [w/ code]

Switching, merging or swapping, call it what you like – it’s a pain to pull off. You need to spend a lot of time tuning the colors, blending the edges and smudging to get a decent result. So I wrote a plugin for the wonderful GIMP program that helps this process. The merge is done using a blending algorithm that blends in the colous from the original image into the pasted image.
I’ll write a little bit about coding GIMP plugins, which is very simple, and some about the algorithm and its sources.
Let’s see how it’s done

Writing a GIMP plugin

As I mentioned, writing a plugin for the GIMP is not very hard. I used the excellent (though outdated) guide for writing a plugin from GIMP.org.
There are 2 kinds of plugins: scripts, and “native” plugins. Script plugins are written in script-fu, the scripting language for GIMP, they are not compiled. On the opposite side, the “native” plugins are written either in C and compiled to executable, or in Python or Perl. I wrote my plugin in C based roughly on the plugin template, and compiled both on Linux and windows.

From the ground up

The first step I did was getting my hands on the pixel data of the selected layer. This is covered pretty nicely in the tutorial.
Once you get a GimpDrawble you just use gimp_pixel_rgn_init, and then gimp_pixel_rgn_get_row to get a whole row of pixel data (interleaved RGB/RGBA).
To access elements in the row of bytes you need to know how many channels are in the interleave.

gint         channels, x1, y1, x2, y2, x_off, y_off, m, n;
GimpPixelRgn rgn_in;
guchar       *row;
gimp_drawable_mask_bounds (drawable->drawable_id,
	                                   &x1, &y1,
	                                   &x2, &y2);
gimp_drawable_offsets(drawable->drawable_id,&x_off,&y_off);
m = y2 - y1; //height of relevant region
n = x2 - x1; //width of relevant region
channels = gimp_drawable_bpp (drawable->drawable_id);
gimp_pixel_rgn_init (&rgn_in,
			 drawable,
			 x1, y1,
			 n, m,
			 FALSE, FALSE);
row = g_new (guchar, channels * n);
for (int y = 0; y < m; y++) {
gimp_pixel_rgn_get_row (&rgn_in, 		row, 		0, y, n);
//manipulate pixels...
}

Working with layers masks is also simple. Masks are GimpDrawables as well.

gint32 mask_id = gimp_layer_get_mask(drawable->drawable_id);
GimpDrawable* mask_drawable = gimp_drawable_get(mask_id);

Progress reporting

During the run of a lengthy operation it’s recommended to notify the user of the progress. The easiest way to do it in a GIMP plugin is use the built-in progress bar. Very easy to use:

gimp_progress_init ("Clone...");
....
gimp_progress_set_text_printf("Create matrix %dx%d",mn,mn);
gimp_progress_update(0.33);
....
gimp_progress_update(1.0);

User inputs

User input for plugins can be easily done using GTK dialogs. The gimp plugin development tutorial is covering this briefly, but its fairly simple for someone who is already fimiliar with GTK. I copied off the tutorial, and changed the spinbutton to a checkbutton for my needs.

One tip: in the tutorial code the gimp_dialog_new function will not accept just an int for the 4th parameter (needs GtkDialogFlags), so I passed in GtkDialogFlags::GTK_DIALOG_MODAL.
Now that we got the bases covered, let’s move on to the algorithm.

The blending algorithm

The algorithm is based on a paper published in 03′ by Microsoft Research. The article presents a way to merge two images by inspecting their gradients, and using this information to “bleed” in colors from the background image into the pasted image. More generally, the colors from the outer image (the background) will seep into the pasted image, and they will do it faster if the pasted image is more smooth. This creates a very nice blending effect.
There’s some mathematical mumbo-jumbo about estimating and discretising the calculation of the gradient, and in the end it boils down to a simple sparse matrix multiplication. I used GMM++ library to do this in C environment. The library compiles from source (only .h files), so you only need to #include it and set the project paths correctly.

How to use the plugin

OK, so all this BS about GIMP, C and math gave me a headache, and we’re here to have some fun after all. So without further ado, let’s see how to use this plugin to do some nice tricks. Originally I used this plugin to merge faces together, but it actually can be used to merge anything.
For example, a polar bear in the desert:

This is how to do it:

1. get a background picture, preferably one with a uniform color part on which you’ll want to blend in the external object



2. get an image of an object, preferably surrounded by a uniform color “buffer” so it will blend nicely into the background



3. if needed, cut out a bounding rectangle of the object with a good margin on all sides, and place it where you want on the background



4. create a mask, and use lasso to create a tighter mask around the object



5. select the object layer, and fire the plugin



Some other stuff I did:



Video time…

Source of course

Source is as usual in our Google Code SVN repo.
Executable (win32) is available here.
Thank you for tuning in!
Roy.