I want to share a small piece of code to do Laplacian Blending using OpenCV. It’s one of the most basic and canonical methods of image blending, and is a must exercise for any computer graphics student.
Well basically it’s a matter of creating two Laplacian pyramids of both images, and a Gaussian pyramid of the mask.
Then we blend the pyramids into one, and collapse the resulting pyramid into the blended image.
#include "opencv2/opencv.hpp" using namespace cv; class LaplacianBlending { private: Mat_<Vec3f> left; Mat_<Vec3f> right; Mat_<float> blendMask; vector<Mat_<Vec3f> > leftLapPyr,rightLapPyr,resultLapPyr; Mat leftSmallestLevel, rightSmallestLevel, resultSmallestLevel; vector<Mat_<Vec3f> > maskGaussianPyramid; //masks are 3-channels for easier multiplication with RGB int levels; void buildPyramids() { buildLaplacianPyramid(left,leftLapPyr,leftSmallestLevel); buildLaplacianPyramid(right,rightLapPyr,rightSmallestLevel); buildGaussianPyramid(); } void buildGaussianPyramid() { assert(leftLapPyr.size()>0); maskGaussianPyramid.clear(); Mat currentImg; cvtColor(blendMask, currentImg, CV_GRAY2BGR); maskGaussianPyramid.push_back(currentImg); //highest level currentImg = blendMask; for (int l=1; l<levels+1; l++) { Mat _down; if (leftLapPyr.size() > l) { pyrDown(currentImg, _down, leftLapPyr[l].size()); } else { pyrDown(currentImg, _down, leftSmallestLevel.size()); //smallest level } Mat down; cvtColor(_down, down, CV_GRAY2BGR); maskGaussianPyramid.push_back(down); currentImg = _down; } } void buildLaplacianPyramid(const Mat& img, vector<Mat_<Vec3f> >& lapPyr, Mat& smallestLevel) { lapPyr.clear(); Mat currentImg = img; for (int l=0; l<levels; l++) { Mat down,up; pyrDown(currentImg, down); pyrUp(down, up, currentImg.size()); Mat lap = currentImg - up; lapPyr.push_back(lap); currentImg = down; } currentImg.copyTo(smallestLevel); } Mat_<Vec3f> reconstructImgFromLapPyramid() { Mat currentImg = resultSmallestLevel; for (int l=levels-1; l>=0; l--) { Mat up; pyrUp(currentImg, up, resultLapPyr[l].size()); currentImg = up + resultLapPyr[l]; } return currentImg; } void blendLapPyrs() { resultSmallestLevel = leftSmallestLevel.mul(maskGaussianPyramid.back()) + rightSmallestLevel.mul(Scalar(1.0,1.0,1.0) - maskGaussianPyramid.back()); for (int l=0; l<levels; l++) { Mat A = leftLapPyr[l].mul(maskGaussianPyramid[l]); Mat antiMask = Scalar(1.0,1.0,1.0) - maskGaussianPyramid[l]; Mat B = rightLapPyr[l].mul(antiMask); Mat_<Vec3f> blendedLevel = A + B; resultLapPyr.push_back(blendedLevel); } } public: LaplacianBlending(const Mat_<Vec3f>& _left, const Mat_<Vec3f>& _right, const Mat_<float>& _blendMask, int _levels): left(_left),right(_right),blendMask(_blendMask),levels(_levels) { assert(_left.size() == _right.size()); assert(_left.size() == _blendMask.size()); buildPyramids(); blendLapPyrs(); }; Mat_<Vec3f> blend() { return reconstructImgFromLapPyramid(); } }; Mat_<Vec3f> LaplacianBlend(const Mat_<Vec3f>& l, const Mat_<Vec3f>& r, const Mat_<float>& m) { LaplacianBlending lb(l,r,m,4); return lb.blend(); } int main(int argc, char** argv) { Mat l8u = imread("left.png"); Mat r8u = imread("right.png"); Mat_<Vec3f> l; l8u.convertTo(l,CV_32F,1.0/255.0); Mat_<Vec3f> r; r8u.convertTo(r,CV_32F,1.0/255.0); Mat_<float> m(l.rows,l.cols,0.0); m(Range::all(),Range(0,m.cols/2)) = 1.0; Mat_<Vec3f> blend = LaplacianBlend(l, r, m); imshow("blended",blend); waitKey(0); }
To use, simply call the function LaplacianBlend with your two images and your mask, and the result will be returned.
Here’s something I did with it:
Enjoy
Roy.
13 replies on “Just a simple Laplacian pyramid blender using OpenCV [w/code]”
Hi,
I tried using the code,
and i get an exception on this line:
cvtColor(blendMask, currentImg, CV_GRAY2BGR);
any idea what could cause the problem?
i’m kinda new to open cv
@D3vilJin
I can’t help without knowing what is exception..
Hi,
I’m currently experimenting with your code and I have a few questions if you could answer:
does your class accept different kinds of images like CV_8UC3 and CV_32FC3;
and if not, what are the relevant portions in the code where the image being a certain type is essential?
I will be grateful if you can help me with this.
@Paulo Almeida which is CV_32FC3 (CV_8UC3)
It actually only likes Mat_
You can make a simple conversion from CV_8UC3 to CV_32FC3 in the main function LaplacianBlend, or simply create another LaplacianBlend function that accepts Mat_
It would be nice if the whole thing will be templated, and then accept any kind of image without conversion
can you show me how I can call this in the main function? thanks.. I’am getting assertion failure.. So i was wondering, maybe there is a mistake on what I am doing.. Thanks and regards.
can you show how to use this in main, im getting assertion failure!
hope to hear something soon! thank you!
@jack
I’ve added a small main to the code excerpt.
Thank you! when i run the program the command line window pops up when the code is at pyrDown(currentImg, down), i guess it just doesn’t do anything with it and just exits.. could that be because I am using opencv 2.2 ?
Even i’am facing the same problem as jack is facing,i’am also using open CV 2.2.
I would be highly thankful if you could help me out with it.
@k_dee @jack
To get a better idea of the problem I would need some debug info.. like an exception thrown, etc.
Anyway I recommend updating to OpenCV 2.3+ (2.4 is latest at the moment)
R.
Hi,
I’ve been using your code and it works perfectly. I have a doubt, when I have an image where the part of interest is sorrounded by black pixels I got a frame of strong Laplacian lines that I wouldn’t like to have (blending doesn’t look well). How do I manage this situation ?. Thanks for your help.
I get this error, can you tell me how to fix it?
here is the line of your code cause the error
Mat_ l; l8u.convertTo(l,CV_32F,1.0/255.0);
Here is the error message:
OpenCV Error: Assertion failed (!fixedType() || ((Mat*)obj)->type() == mtype) in unknown function.
PS: I am using Visual Studio 2012 and OpenCV 2.4.4
Hi,
thanks for code sharing!
May you also supply the three images for experimenting?
Thanks!