Skip to content

Commit

Permalink
Merge pull request #11 from luisdemarchi/develop
Browse files Browse the repository at this point in the history 
Develop
  • Loading branch information
@luisdemarchi
luisdemarchi authored Jul 1, 2019
2 parents 2bae346 + 974f13b commit 59e4c12
Show file tree
Hide file tree
Showing 6 changed files with 191 additions and 112 deletions.
78 changes: 71 additions & 7 deletions README.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,10 @@ This plugin implementation of the method described in [2013 by Markuš et al](ht

PS: For older smartphones, it is ideal that each frame analyzed has a maximum height and width of 60 pixels and the processing loop runs every 100 milliseconds or more. In the sample project this was implemented.

<!-- blank line -->
----
<!-- blank line -->

# Installation
```
cordova plugins add cordova-plugin-facedetection-lite
Expand All @@ -32,11 +36,6 @@ facedetection.initFaceDetection(5, "./facefinder", function (result) {
});
```
<info>**Warning**: Until the current version, the parameters are being ignored on some platforms, being ixed default value in the code.</info>
{: .alert .alert-warning}

<!-- blank line -->
----
<!-- blank line -->

### detections(rgba, width, height, minSizeFace, maxSizeFace, iouthreshold, resultCallback)

Expand All @@ -48,8 +47,6 @@ facedetection.initFaceDetection(5, "./facefinder", function (result) {
* `iouthreshold` - Maximum size of selected faces
* `resultCallback` - Callback function

<info>**Warning**: Until the current version, only the first 3 parameters are implemented and the rest of the parameters are being ignored on some of the platforms, being fixed default value in the code.</info>

##### Code:
```javascript
facedetection.detections(rgba, cameraWidth, cameraHeight, cameraWidth * 0.2, cameraWidth * 1.2, 0.1, function (dets) {
Expand All @@ -65,8 +62,15 @@ facedetection.detections(rgba, cameraWidth, cameraHeight, cameraWidth * 0.2, cam
}
});
```
<info>**Warning**: Until the current version, only the first 3 parameters are implemented and the rest of the parameters are being ignored on some of the platforms, being fixed default value in the code.</info>

<!-- blank line -->
----
<!-- blank line -->

# Sample App

## Camera
TODO

<table>
Expand All @@ -82,6 +86,62 @@ TODO
</tr>
</table>

## Image
```javascript
function sampleImageDetector() {
var width = 319;
var height = 480;

var canvasPreview = document.createElement('canvas');
canvasPreview.width = width;
canvasPreview.height = height;

var canvasPreviewCtx = canvasPreview.getContext('2d');

var url = 'https://upload.wikimedia.org/wikipedia/commons/thumb/6/65/Will_Smith_by_Gage_Skidmore.jpg/319px-Will_Smith_by_Gage_Skidmore.jpg';
var baseImage = new Image();
baseImage.src = url + '?' + new Date().getTime();
baseImage.setAttribute('crossOrigin', '');
baseImage.onload = function () {
canvasPreviewCtx.drawImage(baseImage, 0, 0);
var rgba = canvasPreviewCtx.getImageData(0, 0, width, height).data;

facedetection.initFaceDetection(0, "./facefinder", function (result) {
facedetection.detections(rgba, width, height, width * 0.2, width * 1.2, 0.1,
function (dets) {
var greaterFace = Math.max.apply(
Math,
dets.map(function (o) {
return o[2];
})
);

for (i = 0; i < dets.length; ++i) {
var box = dets[i];
if (box[2] != greaterFace || box[3] === undefined) {
continue;
}

if (box !== undefined) {
canvasPreviewCtx.beginPath();
canvasPreviewCtx.arc(box[1], box[0], box[2] / 2, 0, 2 * Math.PI, false);
canvasPreviewCtx.lineWidth = 3;
canvasPreviewCtx.strokeStyle = 'red';
canvasPreviewCtx.stroke();
}
}
document.body.appendChild(canvasPreview);
});
});

};
}
```

<!-- blank line -->
----
<!-- blank line -->

# Task List
- [x] Basic structure of the plugin;
- [x] Add PicoJS library to the Browser
Expand All @@ -90,6 +150,10 @@ TODO
- [ ] Process dynamic path to training file
- [ ] Process dynamic parameters when calling each function

<!-- blank line -->
----
<!-- blank line -->

# Development
If you intend to do some improvement in the project, follow some instructions, such as compiling library in the C language.

Expand Down
Binary file removed images/demo-browser.gif
View file
Binary file not shown.
Binary file removed images/demo-ios.gif
View file
Binary file not shown.
2 changes: 1 addition & 1 deletion package.json
View file
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
{
"name": "cordova-plugin-facedetection-lite",
"version": "0.3.3",
"version": "0.3.4",
"description": "A face detection offline with a few lines of code. Was designed to run on old smartphones.",
"cordova": {
"id": "cordova-plugin-facedetection-lite",
Expand Down
2 changes: 1 addition & 1 deletion plugin.xml
View file
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
<?xml version='1.0' encoding='utf-8'?>
<plugin id="cordova-plugin-facedetection-lite" version="0.3.3"
<plugin id="cordova-plugin-facedetection-lite" version="0.3.4"
xmlns="http://apache.org/cordova/ns/plugins/1.0"
xmlns:android="http://schemas.android.com/apk/res/android">
<name>FaceDetection-Lite</name>
Expand Down
221 changes: 118 additions & 103 deletions src/ios/FaceDetection.m
View file
Original file line number Diff line number Diff line change
Expand Up @@ -6,10 +6,10 @@
@interface FaceDetection : CDVPlugin {
bool initialized;
int faceFrameMemory;
UInt8* faceFinder;
NSData *faceFinder;
int maxslotsize;
int maxndets;
Float32* memory;
Float32* memoryDets;
UInt32* counts;
}

Expand Down Expand Up @@ -56,111 +56,126 @@ - (void)initFaceDetection:(CDVInvokedUrlCommand*)command{
[self.commandDelegate sendPluginResult:[CDVPluginResult resultWithStatus:CDVCommandStatus_OK] callbackId:command.callbackId];
return;
}


NSURL *URL = [NSURL URLWithString:@"https://raw.githubusercontent.com/nenadmarkus/pico/c2e81f9d23cc11d1a612fd21e4f9de0921a5d0d9/rnt/cascades/facefinder"];
NSURLSession *session = [NSURLSession sessionWithConfiguration:
[NSURLSessionConfiguration defaultSessionConfiguration]];
NSURLSessionTask *task = [session downloadTaskWithURL:URL
completionHandler:^(NSURL *location, NSURLResponse *response, NSError *error) {
if ( !error){
NSData* file = [NSData dataWithContentsOfURL: location];
UInt8* faceFinderTemp = malloc(sizeof(UInt8) * file.length);
[file getBytes:faceFinderTemp length:file.length];
self->faceFinder = faceFinderTemp;

self->maxndets = 20;

// self->detections = malloc(sizeof(Float32) * 4 * self->maxndets);

int nmemslots = 5;
self->maxslotsize = 256;

self->memory = malloc(sizeof(Float32*) * 4*nmemslots*self->maxslotsize);


[self.commandDelegate sendPluginResult:[CDVPluginResult resultWithStatus:CDVCommandStatus_OK] callbackId:command.callbackId];
}
}];
[task resume];
initialized = true;
}
// NSURL *URL = [NSURL URLWithString:@"https://raw.githubusercontent.com/nenadmarkus/pico/c2e81f9d23cc11d1a612fd21e4f9de0921a5d0d9/rnt/cascades/facefinder"];
// NSURLSession *session = [NSURLSession sessionWithConfiguration:
// [NSURLSessionConfiguration defaultSessionConfiguration]];
// NSURLSessionTask *task = [session downloadTaskWithURL:URL
// completionHandler:^(NSURL *location, NSURLResponse *response, NSError *error) {
// if ( !error){
// NSData* file = [NSData dataWithContentsOfURL: location];
// UInt8* faceFinderTemp = malloc(sizeof(UInt8) * file.length);
// [file getBytes:faceFinderTemp length:file.length];
// self->faceFinder = file;
// }
// }];
// [task resume];


NSString* currentFile = [self.commandDelegate pathForResource:@"facefinder"];
NSData *fileData = [NSData dataWithContentsOfFile:currentFile];

- (void)detections:(CDVInvokedUrlCommand*)command{

if (self->maxndets > 0){
NSDictionary* data = [command argumentAtIndex:0];
NSDictionary* rgbaDict = [data objectForKey:@"rgba"];

int height = [[data objectForKey:@"height"] intValue];
int width = [[data objectForKey:@"width"] intValue];

long totalPixels = height*width;
UInt8 rawData[totalPixels];
for(int i=0; i < totalPixels; ++i) {
// gray = red(20%) + green(70%) + blue(10%)
int red = 0.2126 * [[rgbaDict valueForKey:[NSString stringWithFormat:@"%d", 4*i]] intValue];
int green = 0.7152 * [[rgbaDict valueForKey:[NSString stringWithFormat:@"%d", 4*i+1]] intValue];
int blue = 0.0722 * [[rgbaDict valueForKey:[NSString stringWithFormat:@"%d", 4*i+2]] intValue];
rawData[i] = red + green + blue;
}

float dets[4 * self->maxndets];


int ndets;
ndets = find_objects(
&dets, self->maxndets,
self->faceFinder, 0.0,
rawData, height, width, width,
1.1, 0.1, 10, 100
);

int nmemslots = 5;
int* slot[1];
slot[0] = 0;

UInt32* counts = malloc(sizeof(UInt32) * nmemslots);
for(int i=0; i < nmemslots; ++i) {
counts[i] = 0;
}

ndets = update_memory(
slot,
&self->memory, counts, nmemslots, self->maxslotsize,
&dets, ndets, self->maxndets
);
ndets = cluster_detections(
&dets, ndets
);
NSMutableArray* detectionArray = [NSMutableArray arrayWithCapacity:ndets];
for (int i = 0; i < ndets; i++) {
NSNumber *y = [NSNumber numberWithFloat:dets[4*i]];
NSNumber *x = [NSNumber numberWithFloat:dets[4*i+1]];
NSNumber *size = [NSNumber numberWithFloat:dets[4*i+2]];
NSNumber *score = [NSNumber numberWithFloat:dets[4*i+4]];

[detectionArray addObject:@[y, x, size, score]];
}

// free(dets);
// free(rawData);
// free(slot);
counts = NULL;
free(counts);
for(int i=0; i < totalPixels; ++i) {
rawData[i] = 0;
}
data = nil;
rgbaDict = nil;
CDVPluginResult* pluginResult = [CDVPluginResult resultWithStatus:CDVCommandStatus_OK messageAsArray:detectionArray];
[self.commandDelegate sendPluginResult:pluginResult callbackId:command.callbackId];
detectionArray = nil;

}else{
CDVPluginResult* pluginResult = [CDVPluginResult resultWithStatus:CDVCommandStatus_OK messageAsArray:@[]];
[self.commandDelegate sendPluginResult:pluginResult callbackId:command.callbackId];
}
self->faceFinder = fileData;
self->maxndets = 20;
int nmemslots = 5;
self->maxslotsize = 256;
self->memoryDets = malloc(sizeof(Float32*) * 4*nmemslots*self->maxslotsize);

initialized = true;

[self.commandDelegate sendPluginResult:[CDVPluginResult resultWithStatus:CDVCommandStatus_OK] callbackId:command.callbackId];
}
- (void)detections:(CDVInvokedUrlCommand*)command
{
[self.commandDelegate runInBackground:^{
if (self->maxndets > 0){
NSDictionary* data;
NSDictionary* rgbaDict;
int height;
int width;

data = [command argumentAtIndex:0];
rgbaDict = [data objectForKey:@"rgba"];

height = [[data objectForKey:@"height"] intValue];
width = [[data objectForKey:@"width"] intValue];


long totalPixels = height*width;
UInt8 rawData[totalPixels];
for(int i=0; i < totalPixels; ++i) {
// gray = red(20%) + green(70%) + blue(10%)
int red = 0.2126 * [[rgbaDict valueForKey:[NSString stringWithFormat:@"%d", 4*i]] intValue];
int green = 0.7152 * [[rgbaDict valueForKey:[NSString stringWithFormat:@"%d", 4*i+1]] intValue];
int blue = 0.0722 * [[rgbaDict valueForKey:[NSString stringWithFormat:@"%d", 4*i+2]] intValue];
rawData[i] = red + green + blue;
}

float dets[4 * self->maxndets];

int ndets;
float minSizeFace = width * 0.3;
float maxSizeFace = width * 0.9;
ndets = find_objects(
&dets, self->maxndets,
[self->faceFinder bytes], 0.0,
rawData, height, width, width,
1.1, 0.12, minSizeFace, maxSizeFace
);

int nmemslots = 5;
int* slot[1];
slot[0] = 0;
UInt32* counts[nmemslots];

ndets = update_memory(
slot,
&self->memoryDets, counts, nmemslots, self->maxslotsize,
&dets, ndets, self->maxndets
);
ndets = cluster_detections(
&dets, ndets
);
NSMutableArray* detectionArray = [NSMutableArray new];
for (int i = 0; i < ndets; i++) {
NSNumber *y = [NSNumber numberWithFloat:dets[4*i]];
NSNumber *x = [NSNumber numberWithFloat:dets[4*i+1]];
NSNumber *size = [NSNumber numberWithFloat:dets[4*i+2]];
NSNumber *score = [NSNumber numberWithFloat:dets[4*i+4]];

if (!isnormal(y.floatValue) || !isnormal(x.floatValue) || !isnormal(size.floatValue)){
continue;
}else if (isnan(score.floatValue)){
self->maxndets = 0;
self->initialized = false;
#warning TODO - pass original parameters
[self initFaceDetection:command];
CDVPluginResult* pluginResult = [CDVPluginResult resultWithStatus:CDVCommandStatus_OK messageAsArray:@[]];
[self.commandDelegate sendPluginResult:pluginResult callbackId:command.callbackId];
return;
}

[detectionArray addObject:@[y, x, size, score]];
}

@try {
CDVPluginResult* pluginResult = [CDVPluginResult resultWithStatus:CDVCommandStatus_OK messageAsArray:detectionArray];
[self.commandDelegate sendPluginResult:pluginResult callbackId:command.callbackId];
}
@catch (NSException *exception) {
CDVPluginResult* pluginResult = [CDVPluginResult resultWithStatus:CDVCommandStatus_OK messageAsArray:@[]];
[self.commandDelegate sendPluginResult:pluginResult callbackId:command.callbackId];
}

data = nil;
rgbaDict = nil;
detectionArray = nil;
}else{
CDVPluginResult* pluginResult = [CDVPluginResult resultWithStatus:CDVCommandStatus_OK messageAsArray:@[]];
[self.commandDelegate sendPluginResult:pluginResult callbackId:command.callbackId];
}
}];
}
@end

0 comments on commit 59e4c12

Please sign in to comment.