Machine Vision Box (Composed by Wangkang)

Computer Vision, Intelligent Algorithm

How to use HaarTraining in OpenCV

Filed under Computer Vision by Wang Kang on 12-08-2009

Today I find a useful Manual about how to use HaarTraing, everybody who use OpenCV can find a sample ‘find face’ program, this program use this method, but we can only find the face trained xml in the opencv data folder,now, I’ll show you how to(like training hand dataset):

We can not follow exact ways how OpenCV executed haartrainig for face detection because they do not provide us several information such as what images they used for training. This report gives you a step-by-step procedure which you can exactly follow.

My working environment is Visual Studio + cygwin on Windows XP, or on Linux. I use several commands.

  • Data Prepartion

  • Positive (Face) Images

I downloaded from The UMIST Face Database.
FYI: There is a database list on Face Recognition Homepage – Databases.

  • Negative (Background) Images

Kuranov et. al. [3] states as they used 3000 negative images.
Fortunately, I found http://face.urtho.net/ (Negatives sets, Set 1 – Various negatives ). But, this collection was used for eye detection, and includes some faces in some pictures. Therefore, I deleted all suspicious images which looked to include faces. About 3000 images were remained, thus they should be enough.
The collection is available. See Download section.

  • Create Samples (Increase Images)

We can create traning samples and testing samples with createsamples software.
createsamples

createsamples software has mainly four functionalities.

I describe these functionalities here because the Tutorial [1] does not explain them clearly (but please see the Tutorial [1] also for further options).

1. Create training samples from one image applying distortions.
This is launched when options, -img, -bg, and -vec were specified.

  • -img [a positive image]

  • -bg [a description file of negatives]

  • -vec [name of the output file containing the generated samples]

For example,
$ createsamples -img face.png -num 10 -bg negatives.dat -vec samples.vec -maxxangle 0.6 -maxyangle 0 -maxzangle 0.3 -maxidev 100 -bgcolor 0 -bgthresh 0 -w 20 -h 20

2. Create training samples from some images without applying distortions
You may think this function is a simple file format conversion function.
This is launched when options, -info, and -vec were specified.

  • -info [a description file of positives]

  • -vec [name of the output file containing the generated samples]

In this case, the option -num is used only to reduce the number of samples to generate, this does not generate many samples from one image applying distortions.
For example,
$ createsamples -info samples.dat -vec samples.vec -w 20 -h 20

3. Create test samples and their ground truth from single image applying distortions
This is triggered when options, -img, -bg, and -info were specified.

  • -img [a positive image]

  • -bg [a description file of negatives]

  • -info [a generated description file for the generated test iamges]

In this case, -w and -h are used to determine the minimal size of positives to embed in the test images.
$ createsamples -img face.png -num 10 -bg negatives.dat -info test.dat -maxxangle 0.6 -maxyangle 0 -maxzangle 0.3 -maxidev 100 -bgcolor 0 -bgthresh 0 This generates tons of jpg files such as

The output file name format is as <number>_<x>_<y>_<width>_<height>.jpg, where x, y, width and height are the coordinates of placed object bounding rectangle.

4. Show images within a vec file
This is triggered when only an option, -vec, was specified (no -info, -img, -bg). For example,
$ createsamples -vec samples.vec -w 20 -h 20 Create Description File

The format of a description file which is used by createsamples is [1]
[filename]
[filename]
[filename]
.
. such as
img/img1.jpg
img/img2.jpg or if it is used for testing,
[filename] [# of objects] [[x y width height] [... for 2nd object] …]
[filename] [# of objects] [[x y width height] [... for 2nd object] …]
[filename] [# of objects] [[x y width height] [... for 2nd object] …]
.
. such as
img/img1.jpg 1 140 100 45 45
img/img2.jpg 2 100 200 50 50 50 30 25 25 The first format can be easily created with the find command as
$ cd [your working directory]
$ find [image dir] -name ‘*.[image ext]‘ > [description file] such as
$ find ../../data/negatives/ -name ‘*.jpg’ > negatives.dat
$ find ../../data/umist_cropped/ -name ‘*.pgm’ > positives.dat EXTRA: random seed

The createsamples software applys the same sequence of distortions for each image. We may want to apply the different sequence of distortions for each image because, otherwise, our resulting detection may work only for specific distortions.
This can be done by modifying createsamples slightly as:
Add below in the top
#include<time.h> Add below in the main function
srand(time(NULL)); The modified source code is available at svn:createsamples.cpp

Create Training Samples

Kuranov et. al. [3] mentions as they used 5000 positive frontal face patterns and 3000 negatives for training, and 5000 positive frontal face patterns were derived from 1000 original faces.
However, you may have noticed that none of 4 functionalities of createsample software provide us the functionality to generate 5000 positives from 1000 images at burst. We have to use the 1st functionality of the createsample to generate 5 (or some) positives form 1 image, repeat the procedures 1000 (or some) times, and finally merge the generated output vec files.
I wrote a program, mergevec.cpp, to merge vec files.
I also wrote a script, createtrainsamples.pl, to repeat the procedures 1000 (or some) times. Please modify the path to createsamples and its option parameters directly in the file. I specified 7000 instead of 5000 because the Tutorial [1] states as “the reasonable number of positive samples is 7000.”
The input format of createtrainsamples.pl is
$ perl createtrainsamples.pl <positives.dat> <negatives.dat> <vec_out_dir> And, the input format of mergevec is
$ mergevec <vec_collection_file_name> <output_vec_file_name> Example)
$ perl createtrainsamples.pl positives.dat negatives.dat samples
$ find samples/ -name ‘*.vec’ > samples.dat # to create a collection file for vec files
$ mergevec samples.dat samples.vec
$ createsamples -vec samples.vec -show -w 20 -h 20 # Extra: If you want show Create Testing Samples

Testing samples are images which include positives in negative images and locations of positives are known in the images. We can use the 3rd functionality of createsamples to do it. But, we can specify only one image using it, thus, creating a script to repeat the procedure would help us. The script is available at svn:createtestsamples.pl. Please modify the path to createsamples and its option parameters directly in the file.
The input format of the createtestsamples.pl is as
$ perl createtestsamples.pl <positives.dat> <negatives.dat> <output_dir> This generates lots of jpg files and info.dat. The jpg file name format is as <number>_<x>_<y>_<width>_<height>.jpg, where x, y, width and height are the coordinates of placed object bounding rectangle.
Example)
$ perl createtestsamples.pl positives.dat negatives.dat tests
$ find tests/ -name ‘info.dat’ -exec cat \{\} \; | tee tests.dat # merge info files Training

Training is done by the haartraining software.
haartraining

Kuranov et. al. [3] states as 20×20 of sample size achieved the highest hit rate. Furthermore, they states as “For 18×18 four split nodes performed best, while for 20×20 two nodes were slightly better. The difference between weak tree classifiers with 2, 3 or 4 split nodes is smaller than their superiority with respect to stumps.”
Furthermore, there was a description as “20 stages were trained. Assuming that my test set is representative for the learning task, I can expect a false alarm rate about and a hit rate about .”
Therefore, I used 20×20 of sample size with nsplit = 2, nstages = 20, minhitrate = 0.9999, maxfalselarm = 0.5 such as
$ haartraining -data trainout -vec samples.vec -bg negatives.dat -nstages 20 -nsplits 2 -minhitrate 0.999 -maxfalsealarm 0.5 -npos 7000 -neg 2984 -w 20 -h 20 -mem 512 -mode ALL The “-mode ALL” uses Extended Sets of Haar-like Features [2]. Default is BASIC and it uses only upright features, while ALL uses the full set of upright and 45 degree rotated feature set [1].
The “-mem 512″ is the available memory in MB for precalculation [1]. Default is 200MB, so increase if more memory is available. We should not use all system RAM because otherwise it will result in a considerable training slow down.
#If you use OpenMP (multi-processing) supporting compilers such as Interl C++ compiler and MS Visual Studio 8.0, the haartraining is automatically built with the OpenMP. This is effective when your computer has multi-CPUs.
#One training took two days.
Generate xml

The haartraing software did not generate cascade.xml which is used for the the facedetect software (you should also have expected to obtaine cascade.xml).
Although the Tutorial [1] does not mention how to generate cascade.xml, OpenCV has a software to convert the haartraining output dir tree into xml file in the OpenCV/samples/c/convert_cascade.c (in the install directory).
The input format is as
$ convert_cascade –size=”<sample_width>x<sampe_height>” <haartraining_ouput_dir> <ouput_file> Example)
$ convert_cascade –size=”20×20″ trainout cascade.xml Testing

under construction. I am not achieving sufficient results….
performance

We will evaluate the performance of the generated classifier using the performance software.
The input format of the software is
$ performance -data <haartraining_output_dir> -info <description_file_for_test_samples> such as
$ performance -data trainout -info tests.dat The output format is as
+================================+======+======+== ====+
| File Name | Hits |Missed| False|
+================================+======+======+== ====+
‘Hits’ shows the # of correctly found objects, and ‘Missed’ shows the # of missed objects (there must exist, but were not found), and ‘False’ shows the # of false alarm (there must not exist, but were found).
facedetect

Fun with a USB camera or some image files.
$ facedetect –cascade=<xml_file> [<imagefile_or_videofile_or_cameraid>] Discussion

The generated classifier did not work well. Specializing classifiers for only the frontal faces may achieve the better result (UMIST images include 90 degree faces for each person.)
Download

The files are available at http://opensvn.csie.org/sonots/SciSo…/haartraining/ .

  • HaarTraining Source Code, Binary, Makefile (VC++ Project File)

  • data The collected Image Datasets

  • result Generated Files

This is a svn repository, thus you can download files at burst if you have a svn client. (I will not explain how to install or use the svn client here.) Downloading Image Datasets may take forever…
The list of my additional utilities (they are in HaarTraining/ directory):

The following additional utilities can be obtained from OpenCV/samples/c in your OpenCV install directory (they are also in HaarTraining/ directory).

References

  • [3] Alexander Kuranov, Rainer Lienhart, and Vadim Pisarevsky. An Empirical Analysis of Boosting Algorithms for Rapid Objects With an Extended Set of Haar-like Features. Intel Technical Report MRL-TR-July02-01, 2002. ( http://www.lienhart.de/Publications/DAGM2003.pdf )

Comments:

One Response


  1. hello, love from finland. your post looks great. Mind if i quote it in my blog?

Leave a Reply

Subscribe to RSS Feed Rss