Skip to content

Latest commit

 

History

History
146 lines (105 loc) · 7.23 KB

Documentation.md

File metadata and controls

146 lines (105 loc) · 7.23 KB

FERNLV Documentation

Requirements

To run this project, you need to install:

  1. OpenCV: pip install opencv-python
  2. Numpy: pip install numpy
  3. Tensorflow: please refers to link
  4. PyQt5: pip install PyQt5

(:grey_exclamation: Cautions: I haven't check the compatibility of all version from the library. I used numpy==1.15.4, PyQt5==5.11.3, tensorflow==1.11.0, and opencv==3.4.1. Please use python>3.4)

Before Usage

Please download all of haarcascade.xml from this link to provide the module. Put all of them on the Assets folder*

All of the pickle file on the project is based on my example. It detect and recognize several character from MCU (11 character: Tony Stark (not in ironman suit), Loki, Thanos, Bucky Barnes, Wanda Maximoff, Thor, Steve Rogers (not in Captain America suit), Natasha Romanoff, Dr Strange, T'Challa (not in black panther suit), Mantis). If the face detected but the face is not in the list, it will give unknown label.

After you installed all the requirement and download the XML file of haarcascade, you can use the project as well as on the example folder.

Workflow

As a part of this documentation, I would write the workflow I've made so far and illustrate it.

EigenFace

A.Training Phase

Training Phase

steps:

  1. Face Dateset: it consist of folder with this configuration. For every sub folder, please use the name of the class:

     -> Root_folder
     -> class_1
     -> class_2
     -> class_3

  2. Face Transform : change image to a vector. Later the vector will combine into one matrix, as dataset matrix

  3. Mean Extraction: Extract mean from the matrix

  4. Subtracted with Mean : to create covariance matrix, the dataset matrix will be subtracted with its mean

  5. Covariance Transform: dot product the subtracted matrix with its transpose

  6. Eigen Decomposition: get eigenvector and eigenvalue from covariance matrix

  7. Get Eigenface: Take k-value from the eigenvalue as eigenface

  8. Create Weight with Eigenface: It is done by dot prodcut the eigenface with the subtracted matrix

  9. Save it to Pickle: Later we save the weight, eigenface and the mean

Code

  1. Split the data using split_data_train_test_val. More on link

    split_data_train_test_val(image_dir: str, base_dir: str = 'base_dir',
                      testing_percentage=0,
                      validation_percentage=10,
                      size=32, save=True,
                      cropped=False)
    • image_dir: a root folder consist of category-named directories. The root directory should looks like this:
    -> root_dir:
     --> class_1
     --> class_2
     --> ...
    • base_dir: the directory to placed the train, validation and testing.
    • testing_percentage: From the available data, it will used n-percent for testing. USE INTEGER
    • validation_percentage:From the available data, it will used n-percent for validation. USE INTEGER
    • size: The size to resize the image. Suggested 32, so the image will resize into (32, 32)
    • save: Save option. If True, it will produce pickle files for train, validation and testing (if available)
    • cropped: if the dataset already in desired condition, put this into False. Otherwise it will cropped faces from the image using default face cascade classifier. Use it with cautions
    • return: train_data, train_labels, val_data, val_labels, test_data, test_labels

    Example:

    train_data, train_labels, val_data, val_labels, test_data, test_label = eu.split_data_train_test_val('Dataset', base_dir='base_dir')

  2. Training. Use create_eigen_face_vector. More on link

    create_eigen_face_vector(data, normalization=True, max_component=25, save=True)
    • data: data that consists of image vector in each row
    • normalization: the option for normalization
    • max_component: number of component to be used for eigenface
    • save: the option for saving the eigenface
    • return: eigen_face, weight, average

    Example:

    eigen_face, weight, average = eu.create_eigen_face_vector(train_data)

B. Usage Workflow (based on this example)

Usage Workflow

Steps:

  1. Start the Webcam: start with available channel. Default is 0. You can choose yours by defined the number on the blank box
  2. Check if the webcam is works or not. If not, the process would not be execute. Otherwise the the looping wil start while yielding the frame
  3. Face Detection: Using haarcascade face classifier from OpenCV and crop it.
  4. Face Recognition: From the cropped face, the face recognition would try to recognize whose the face belongs to.
  5. Show: show the frame with detected and recognized face
Code

From the example, I used from FERNLV.FaceRecognition import EigenFaceRecognition

    __init__(self, eigenface_path=None, train_path=None, val_path=None, face_cascade='default')

The code above used to make EigenFaceRecognition instance.

  • eigenface_path: path of eigenface pickle

  • train_path: path of train data pickle

  • val_path: path of validation data pickle

  • face_cascade: face_cascade: name of haarcascade that available from OpenCV which is:

    • 'default': default frontal face
    • 'alt': alternative of frontal face
    • 'alt2': another alternative for frontal face
    • 'altree': another alternative version with tree
    • 'profile': Only detect the profile face
    • 'LBP': XMl made with Local Binary Pattern
    • 'cudaDef': A Cuda version of default

To predict the face from the given image, use predicts method.

    predicts(self, image)
  • image: an image. it should be Numpy.ndarray
  • return: an image and the list of the cropped face

Example

    import cv2
    from FERNLV.FaceRecognition import EigenFaceRecognition as eu
    
    image = cv2.imread(...)
    efr = eu(eigenface_path='eigenface.pickle', train_path='train_data.pickle', val_path='val_data.pickle', face_cascade='default')
    detected_face, crops = efr.predicts(image)

References

The explanation about eigenface can be reach from: