背景建模

Overview

背景建模在视频监控等动态图像的检测上有很重要的位置.这篇文章将会逐渐学习一系列的背景建模方法.

Code

简单背景差分
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30

#include <iostream>
#include <opencv/cv.h>
#include <opencv/cv.hpp>
using namespace std;
using namespace cv;

int main(){
Mat back1, back2;
back1 = imread("/home/shensir/Documents/MyPrograming/Cpp/Clions/data/back1.jpg");
back2 = imread("/home/shensir/Documents/MyPrograming/Cpp/Clions/data/back2.jpg");

Mat Original_img = back1.clone();

cvtColor(back1, back1, COLOR_BGR2GRAY);
cvtColor(back2, back2, COLOR_BGR2GRAY);

resize(back1, back1, Size(), 0.1, 0.1);
resize(back2, back2, Size(), 0.1, 0.1);
imshow("back1", back1);
imshow("back2", back2);

// 直接相减
Mat result = back2 - back1;
imshow("result", result);

waitKey(0);
return 0;
}

输出:

可以看到,直接相减再thresh一下,效果还是比较好的。但是在实际上,这种方法要求比较高,它要求背景图像是静止的,任何微小的背景变动都会对检测造成影响:

In this approach, we assume that the background is static. If some parts of our background start moving, then those parts will start getting detected as new objects. So, even if the movements are minor, say a waving flag, it will cause problems in our detection algorithm. This approach is also sensitive to changes in illumination, and it cannot handle any camera movement. Needless to say, it’s a delicate approach! We need something that can handle all these things in the real world.

帧差值法

帧差值法是利用连续帧之间的差异来获取物体的移动信息。

像这样, 重叠的部分是不会被检测到,只有边缘部分会被检测到,但是很明显,这要求物体在转化为灰度图时灰度级是近似均匀的,不然…即使重叠也还是会有差值,不能对冲掉。

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90

#include <iostream>
#include <opencv/cv.h>
#include <opencv/cv.hpp>

Mat frameDiff(Mat prevFrame, Mat curFrame, Mat nextFrame){
Mat diffFrames1, diffFrames2, output;
// 计算当前帧和下一帧的绝对差值
absdiff(nextFrame, curFrame, diffFrames1);
// 计算当前帧和前一帧的绝对差值
absdiff(curFrame, prevFrame, diffFrames2);

// 对以上两个不同的图像进行按位“与”操作
bitwise_and(diffFrames1, diffFrames2, output);

// 为了使得边缘更加明显,进行dilate操作
dilate(output, output, Mat(), Point(-1,-1), 1);

return output;
}


// 从摄像头获取帧
Mat getFrame(VideoCapture cap, float scalingFactor){
// 浮动比例因子设为0.5
Mat frame, output;

// 捕获当前帧
cap >> frame;
// 调整大小
resize(frame, frame, Size(), scalingFactor, scalingFactor, INTER_AREA);
// 转换为灰度
cvtColor(frame, output, CV_BGR2GRAY);

return output;
}




int main(){
Mat frame, prevFrame, curFrame, nextFrame;
char ch;

// 创建捕获对象t
// 0->输入变量表示数据源来自摄像头
VideoCapture cap(0);

// 如果摄像头无法打开,停止执行
if(!cap.isOpened()){
return -1;
}

// 创建GUI窗口
// namedWindow("Frame");

// 调整摄像头输入帧大小的缩放因子
float scalingFactor = 0.75;

prevFrame = getFrame(cap, scalingFactor);
curFrame = getFrame(cap, scalingFactor);
nextFrame = getFrame(cap, scalingFactor);

// 循环知道用户按下Esc键
while(true){
// 显示对象移动
imshow("Object Movement", frameDiff(prevFrame, curFrame, nextFrame));

// 更新变量并抓取下一帧
prevFrame = curFrame;
curFrame = nextFrame;
nextFrame = getFrame(cap, scalingFactor);

// 获取键盘输入,并检测用户是否按下Esc键
// 27->Esc 按钮的ASCII码
ch = waitKey(30);
if(ch==27)
break;

}

cap.release();
destroyAllWindows();

return 0;

}



在移动时摄像头情况:

混合高斯法

OpenCV By Example上的MOG没法使用,查了查cv::bgsegm::BackgroundSubtractorMOG也无法使用…查看文档好像又是cv::cuda::BackgroundSubtractorMOG,仍然没办法使用,可能是编译的时候没有添加支持的问题。于是就看了看官方给的demo,用的MOG2.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152

#include <iostream>
#include <opencv/cv.h>
#include <opencv/cv.hpp>

// Global variables
Mat frame; //current frame
Mat fgMaskMOG2; //fg mask fg mask generated by MOG2 method
Ptr<BackgroundSubtractor> pMOG2; //MOG2 Background subtractor
char keyboard; //input from keyboard
void help();
void processVideo(char* videoFilename);
void processImages(char* firstFrameFilename);
void help()
{
cout
<< "--------------------------------------------------------------------------" << endl
<< "This program shows how to use background subtraction methods provided by " << endl
<< " OpenCV. You can process both videos (-vid) and images (-img)." << endl
<< endl
<< "Usage:" << endl
<< "./bg_sub {-vid <video filename>|-img <image filename>}" << endl
<< "for example: ./bg_sub -vid video.avi" << endl
<< "or: ./bg_sub -img /data/images/1.png" << endl
<< "--------------------------------------------------------------------------" << endl
<< endl;
}
int main(int argc, char* argv[])
{
//print help information
help();
//check for the input parameter correctness
if(argc != 3) {
cerr <<"Incorret input list" << endl;
cerr <<"exiting..." << endl;
return EXIT_FAILURE;
}
//create GUI windows
namedWindow("Frame");
namedWindow("FG Mask MOG 2");
//create Background Subtractor objects
pMOG2 = createBackgroundSubtractorMOG2(); //MOG2 approach
if(strcmp(argv[1], "-vid") == 0) {
//input data coming from a video
processVideo(argv[2]);
}
else if(strcmp(argv[1], "-img") == 0) {
//input data coming from a sequence of images
processImages(argv[2]);
}
else {
//error in reading input parameters
cerr <<"Please, check the input parameters." << endl;
cerr <<"Exiting..." << endl;
return EXIT_FAILURE;
}
//destroy GUI windows
destroyAllWindows();
return EXIT_SUCCESS;
}
void processVideo(char* videoFilename) {
//create the capture object
VideoCapture capture(videoFilename);
if(!capture.isOpened()){
//error in opening the video input
cerr << "Unable to open video file: " << videoFilename << endl;
exit(EXIT_FAILURE);
}
//read input data. ESC or 'q' for quitting
keyboard = 0;
while( keyboard != 'q' && keyboard != 27 ){
//read the current frame
if(!capture.read(frame)) {
cerr << "Unable to read next frame." << endl;
cerr << "Exiting..." << endl;
exit(EXIT_FAILURE);
}
//update the background model
pMOG2->apply(frame, fgMaskMOG2);
erode(fgMaskMOG2, fgMaskMOG2, Mat(), Point(-1, -1), 1);
//get the frame number and write it on the current frame
stringstream ss;
rectangle(frame, cv::Point(10, 2), cv::Point(100,20),
cv::Scalar(255,255,255), -1);
ss << capture.get(CAP_PROP_POS_FRAMES);
string frameNumberString = ss.str();
putText(frame, frameNumberString.c_str(), cv::Point(15, 15),
FONT_HERSHEY_SIMPLEX, 0.5 , cv::Scalar(0,0,0));
//show the current frame and the fg masks
imshow("Frame", frame);
imshow("FG Mask MOG 2", fgMaskMOG2);
//get the input from the keyboard
keyboard = (char)waitKey( 30 );
}
//delete capture object
capture.release();
}
void processImages(char* fistFrameFilename) {
//read the first file of the sequence
frame = imread(fistFrameFilename);
if(frame.empty()){
//error in opening the first image
cerr << "Unable to open first image frame: " << fistFrameFilename << endl;
exit(EXIT_FAILURE);
}
//current image filename
string fn(fistFrameFilename);
//read input data. ESC or 'q' for quitting
keyboard = 0;
while( keyboard != 'q' && keyboard != 27 ){
//update the background model
pMOG2->apply(frame, fgMaskMOG2);
//get the frame number and write it on the current frame
size_t index = fn.find_last_of("/");
if(index == string::npos) {
index = fn.find_last_of("\\");
}
size_t index2 = fn.find_last_of(".");
string prefix = fn.substr(0,index+1);
string suffix = fn.substr(index2);
string frameNumberString = fn.substr(index+1, index2-index-1);
istringstream iss(frameNumberString);
int frameNumber = 0;
iss >> frameNumber;
rectangle(frame, cv::Point(10, 2), cv::Point(100,20),
cv::Scalar(255,255,255), -1);
putText(frame, frameNumberString.c_str(), cv::Point(15, 15),
FONT_HERSHEY_SIMPLEX, 0.5 , cv::Scalar(0,0,0));
//show the current frame and the fg masks
imshow("Frame", frame);
imshow("FG Mask MOG 2", fgMaskMOG2);
//get the input from the keyboard
keyboard = (char)waitKey( 30 );
//search for the next image in the sequence
ostringstream oss;
oss << (frameNumber + 1);
string nextFrameNumberString = oss.str();
string nextFrameFilename = prefix + nextFrameNumberString + suffix;
//read the next frame
frame = imread(nextFrameFilename);
if(frame.empty()){
//error in opening the next image in the sequence
cerr << "Unable to open image frame: " << nextFrameFilename << endl;
exit(EXIT_FAILURE);
}
//update the path of the current frame
fn.assign(nextFrameFilename);
}
}



终端运行:

cmake .
make
./Clions -vid /home/shensir/Documents/MyPrograming/Cpp/Clions/data/BGC.avi

输出:

上面的demo也可以对一系列的连续的图片进行检测,只需要将所有图片按顺序命名为1.xxx, 2.xxx这样,然后运行的时候只提供第一帧的即1.xxx并使用-img即可。关于其中的字符串的操作,我们还原如下:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37

#include <iostream>
#include <sstream>
using namespace std;

int main(){
string fn("/data/1.jpg");
size_t index = fn.find_last_of("/");
cout<<"Index: "<<index<<endl;
string prefix = fn.substr(0, index+1);
cout<<"Prefix: "<<prefix<<endl;

size_t index2 = fn.find_last_of(".");
cout<<"Index2: "<<index2<<endl;
string suffix = fn.substr(index2);
cout<<"suffix: "<<suffix<<endl;

string frameNumberString = fn.substr(index+1, index2-index-1);
cout<<"frameNUmberString: "<<frameNumberString<<endl;

istringstream iss(frameNumberString);

int frameNumber = 0;
iss >> frameNumber;
cout<<"frameNumber: "<<frameNumber<<endl;

ostringstream oss;
oss << (frameNumber + 1);
string nextFrameNumberString = oss.str();
string nextFrameFilename = prefix + nextFrameNumberString + suffix;

cout<<"nextFrameFilename: "<<nextFrameFilename<<endl;
return 0;
}



输出:

Index: 5
Prefix: /data/
Index2: 7
suffix: .jpg
frameNUmberString: 1
frameNumber: 1
nextFrameFilename: /data/2.jpg

开始的时候对istringstream的用法不太懂,查到下面一个例子,虽不懂原理,但是它的用途还是可以了解到:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21

#include <iostream> // std::cout
#include <sstream> // std::istringstream
#include <string> // std::string

int main () {

std::string stringvalues = "125 320 512 750 333";
std::istringstream iss (stringvalues);

for (int n=0; n<5; n++)
{
int val;
iss >> val;
std::cout << val*2 << '\n';
}

return 0;
}


输出:

250
640
1024
1500
666

Refrence

OpenCV By Example

本文标题:背景建模

文章作者:不秩稚童

发布时间:2017年05月19日 - 00:54:03

最后更新:2017年05月21日 - 09:22:13

原始链接:http://datahonor.com/2017/05/19/背景建模/

许可协议: 署名-非商业性使用-禁止演绎 4.0 国际 转载请保留原文链接及作者。

击蒙御寇