diff --git a/.github/VideoThumbnail.png b/.github/VideoThumbnail.png new file mode 100644 index 0000000..e841cc0 Binary files /dev/null and b/.github/VideoThumbnail.png differ diff --git a/.gitignore b/.gitignore new file mode 100644 index 0000000..13e31d0 --- /dev/null +++ b/.gitignore @@ -0,0 +1,77 @@ +# Xcode +# +# gitignore contributors: remember to update Global/Xcode.gitignore, Objective-C.gitignore & Swift.gitignore + +## Build generated +build/ +DerivedData/ + +## Various settings +*.pbxuser +!default.pbxuser +*.mode1v3 +!default.mode1v3 +*.mode2v3 +!default.mode2v3 +*.perspectivev3 +!default.perspectivev3 +xcuserdata/ + +## Other +*.moved-aside +*.xccheckout +*.xcscmblueprint + +## Obj-C/Swift specific +*.hmap +*.ipa +*.dSYM.zip +*.dSYM + +## Playgrounds +timeline.xctimeline +playground.xcworkspace + +# Swift Package Manager +# +# Add this line if you want to avoid checking in source code from Swift Package Manager dependencies. +# Packages/ +# Package.pins +.build/ + +# CocoaPods +# +# We recommend against adding the Pods directory to your .gitignore. However +# you should judge for yourself, the pros and cons are mentioned at: +# https://guides.cocoapods.org/using/using-cocoapods.html#should-i-check-the-pods-directory-into-source-control +# +# Pods/ + +# Carthage +# +# Add this line if you want to avoid checking in source code from Carthage dependencies. +# Carthage/Checkouts + +Carthage/Build + +# fastlane +# +# It is recommended to not store the screenshots in the git repo. Instead, use fastlane to re-generate the +# screenshots whenever they are needed. +# For more information about the recommended setup visit: +# https://docs.fastlane.tools/best-practices/source-control/#source-control + +fastlane/report.xml +fastlane/Preview.html +fastlane/screenshots +fastlane/test_output + +**/.DS_Store +**/*.pyc +Data-Model/Training/input/augmented +Data-Model/Training/input/normalized +Data-Model/Training/input/training-data/**/*.jpg +Data-Model/Training/input/training-data/**/*.JPG +Data-Model/Training/input/training-data/**/*.jpeg +Data-Model/Training/input/training-data/**/*.JPEG +Data-Model/Training/output/tf_files \ No newline at end of file diff --git a/Data-Model/EmojiSketches.mlmodel b/Data-Model/EmojiSketches.mlmodel new file mode 100644 index 0000000..20d748c Binary files /dev/null and b/Data-Model/EmojiSketches.mlmodel differ diff --git a/Data-Model/README.md b/Data-Model/README.md new file mode 100644 index 0000000..99b8609 --- /dev/null +++ b/Data-Model/README.md @@ -0,0 +1,32 @@ +# Data Model + +The Core ML model was built using transfer learning. To perform this task, I used + +- A data set of hand-drawn emojis I created +- TensorFlow and Docker +- A pre-trained [MobileNet](https://arxiv.org/abs/1704.04861) snapshot +- Data augmentation + +## Classes + +The data model can recognize seven emojis: + +- 😊 `smile` +- 😂 `laugh` +- ☀️ `sun` +- ☁️ `cloud` +- ❤️ `heart` +- ✔️ `checkmark` +- 🥐 `croissant` + +## Training with your own data set + +To build your own data set, you can use the `SampleCollection` iOS app. Once you want to train your model, export the saved samples using iTunes or Xcode and put them in the `Training/input/training-data` folder. + +Open a shell in the `Training` folder. + +Run the `prepare.sh` script to download the dependencies and prepare the images. If you do not want to perform data augmentation (which requires high CPU resources), you can edit the `input/normalize` script to use the `input/training-data` folder instead of `input/augmented`. + +Once the `input/normalized` folder is filled with the normalized images, you can run the `retrain.sh` script to trai create the Core ML model. This can take around half an hour, depending on your computer's capabilities. + +The `EmojiSketches.coreml` model file in this directory will be updated when training has completed. diff --git a/Data-Model/Training/Dockerfile b/Data-Model/Training/Dockerfile new file mode 100644 index 0000000..540d6cc --- /dev/null +++ b/Data-Model/Training/Dockerfile @@ -0,0 +1,20 @@ +FROM gcr.io/tensorflow/tensorflow:1.6.0 + +ENV IMAGE_SIZE 224 +ENV OUTPUT_GRAPH tf_files/retrained_graph.pb +ENV OUTPUT_LABELS tf_files/retrained_labels.txt +ENV ARCHITECTURE mobilenet_1.0_${IMAGE_SIZE} +ENV TRAINING_STEPS 1000 + +VOLUME /output +VOLUME /input + +RUN curl -O https://raw.githubusercontent.com/tensorflow/tensorflow/master/tensorflow/examples/image_retraining/retrain.py + +ENTRYPOINT python -m retrain \ + --how_many_training_steps="${TRAINING_STEPS}" \ + --model_dir=/output/tf_files/models/ \ + --output_graph=/output/"${OUTPUT_GRAPH}" \ + --output_labels=/output/"${OUTPUT_LABELS}" \ + --architecture="${ARCHITECTURE}" \ + --image_dir=/input/ \ No newline at end of file diff --git a/Data-Model/Training/export.py b/Data-Model/Training/export.py new file mode 100644 index 0000000..e3ce3bc --- /dev/null +++ b/Data-Model/Training/export.py @@ -0,0 +1,17 @@ +# +# MLMOJI +# +# This file is part of Alexis Aubry's WWDC18 scolarship submission open source project. +# +# Copyright (c) 2018 Alexis Aubry. Available under the terms of the MIT License. +# + +import tfcoreml as tf_converter + +tf_converter.convert(tf_model_path = 'output/tf_files/retrained_graph.pb', + mlmodel_path = '../EmojiSketches.mlmodel', + output_feature_names = ['final_result:0'], + class_labels = 'output/tf_files/retrained_labels.txt', + input_name_shape_dict = {'input:0' : [1, 224, 224, 3]}, + image_input_names = ['input:0']) + diff --git a/Data-Model/Training/input/augment.sh b/Data-Model/Training/input/augment.sh new file mode 100644 index 0000000..d94dbe3 --- /dev/null +++ b/Data-Model/Training/input/augment.sh @@ -0,0 +1,30 @@ +#!/bin/bash + +# +# MLMOJI +# +# This file is part of Alexis Aubry's WWDC18 scolarship submission open source project. +# +# Copyright © 2018 Alexis Aubry. Available under the terms of the MIT License. +# + +# Prepare directory +rm -rf augmented +cp -R training-data augmented + +# Generate Data +python augmentor/main.py augmented/ fliph +python augmentor/main.py augmented/ noise_0.01 +python augmentor/main.py augmented/ fliph,noise_0.01 +python augmentor/main.py augmented/ fliph,rot_-30 +python augmentor/main.py augmented/ fliph,rot_30 +python augmentor/main.py augmented/ rot_15,trans_20_20 +python augmentor/main.py augmented/ rot_33,trans_-20_50 +python augmentor/main.py augmented/ trans_0_20,zoom_100_50_300_300 +python augmentor/main.py augmented/ fliph,trans_50_20,zoom_60_50_200_200 +python augmentor/main.py augmented/ rot_-15,zoom_75_50_300_300 +python augmentor/main.py augmented/ rot_30 +python augmentor/main.py augmented/ blur_4.0 +python augmentor/main.py augmented/ fliph,blur_4.0 +python augmentor/main.py augmented/ fliph,rot_30,blur_4.0 +python augmentor/main.py augmented/ zoom_50_50_250_250 diff --git a/Data-Model/Training/input/augmentor/README.md b/Data-Model/Training/input/augmentor/README.md new file mode 100755 index 0000000..0cb93a3 --- /dev/null +++ b/Data-Model/Training/input/augmentor/README.md @@ -0,0 +1,123 @@ +## Image Augmentor + +> Original repository: [codebox/image_augmentor](https://github/com/codebox/image_augmentor) + +This is a simple data augmentation tool for image files, intended for use with machine learning data sets. +The tool scans a directory containing image files, and generates new images by performing a specified set of +augmentation operations on each file that it finds. This process multiplies the number of training examples that can +be used when developing a neural network, and should significantly improve the resulting network's performance, +particularly when the number of training examples is relatively small. + +Run the utility from the command-line as follows: + + python main.py ... + +The `` argument should be the path to a directory containing the image files to be augmented. +The utility will search the directory recursively for files with any of the following extensions: +`jpg, jpeg, bmp, png`. + +The `transform` arguments determine what types of augmentation operations will be performed, +using the codes listed in the table below: + +|Code|Description|Example Values| +|---|---|------| +|`fliph`|Horizontal Flip|`fliph`| +|`flipv`|Vertical Flip|`flipv`| +|`noise`|Adds random noise to the image|`noise_0.01`,`noise_0.5`| +|`rot`|Rotates the image by the specified amount|`rot_90`,`rot_-45`| +|`trans`|Shifts the pixels of the image by the specified amounts in the x and y directions|`trans_20_10`,`trans_-10_0`| +|`zoom`|Zooms into the specified region of the image, performing stretching/shrinking as necessary|`zoom_0_0_20_20`,`zoom_-10_-20_10_10`| +|`blur`|Blurs the image by the specified amount|`blur_1.5`| + + +Each transform argument results in one additional output image being generated for each input image. +An argument may consist of one or more augmentation operations. Multiple operations within a single argument +must be separated by commas, and the order in which the operations are performed will match the order in which they +are specified within the argument. + +### Examples +Produce 2 output images for each input image, one of which is flipped horizontally, and one of which is flipped vertically: + + python main.py ./my_images fliph flipv + +Produce 1 output image for each input image, by first rotating the image by 90° and then flipping it horizontally: + + python main.py ./my_images rot_90,fliph + +### Operations + +#### Horizontal Flip +Mirrors the image around a vertical line running through its center + + python main.py ./my_images fliph + +Original Image +           +Flipped Image + +#### Vertical Flip +Mirrors the image around a horizontal line running through its center + + python main.py ./my_images flipv + +Original Image +           +Flipped Image + +#### Noise +Adds random noise to the image. The amount of noise to be added is specified by a floating-point numeric value that is included +in the transform argument, the numeric value must be greater than 0. + + python main.py ./my_images noise_0.01 noise_0.02 noise_0.05 + +Original Image +           +Noisy Image +Noisy Image +Noisy Image + +#### Rotate +Rotates the image. The angle of rotation is specified by a integer value that is included in the transform argument + + python main.py ./my_images rot_90 rot_180 rot_-90 + +Original Image +           +Rotated Image +Rotated Image +Rotated Image + +#### Translate +Performs a translation on the image. The size of the translation in the x and y directions are specified by integer values that +are included in the transform argument + + python main.py ./my_images trans_20_20 trans_0_100 + +Original Image +           +Translated Image +Translated Image + +#### Zoom/Stretch +Zooms in (or out) to a particular area of the image. The top-left and bottom-right coordinates of the target region are +specified by integer values included in the transform argument. By specifying a target region with an aspect ratio that +differs from that of the source image, stretching transformations can be performed. + + python main.py ./my_images zoom_150_0_300_150 zoom_0_50_300_150 zoom_200_0_300_300 + +Original Image +           +Zoomed Image +Stretched Image +Stretched Image + +#### Blur +Blurs the image. The amount of blurring is specified by a floating-point value included in the transform argument. + + python main.py ./my_images blur_1.0 blur_2.0 blur_4.0 + +Original Image +           +Blurred Image +Blurred Image +Blurred Image diff --git a/Data-Model/Training/input/augmentor/counter.py b/Data-Model/Training/input/augmentor/counter.py new file mode 100755 index 0000000..01ec532 --- /dev/null +++ b/Data-Model/Training/input/augmentor/counter.py @@ -0,0 +1,29 @@ +from multiprocessing.dummy import Lock + +class Counter: + def __init__(self): + self.lock = Lock() + self._processed = 0 + self._error = 0 + self._skipped_no_match = 0 + self._skipped_augmented = 0 + + def processed(self): + with self.lock: + self._processed += 1 + + def error(self): + with self.lock: + self._error += 1 + + def skipped_no_match(self): + with self.lock: + self._skipped_no_match += 1 + + def skipped_augmented(self): + with self.lock: + self._skipped_augmented += 1 + + def get(self): + with self.lock: + return {'processed' : self._processed, 'error' : self._error, 'skipped_no_match' : self._skipped_no_match, 'skipped_augmented' : self._skipped_augmented} diff --git a/Data-Model/Training/input/augmentor/main.py b/Data-Model/Training/input/augmentor/main.py new file mode 100755 index 0000000..e694476 --- /dev/null +++ b/Data-Model/Training/input/augmentor/main.py @@ -0,0 +1,105 @@ +import sys, os, re, traceback +from os.path import isfile +from multiprocessing.dummy import Pool, cpu_count +from counter import Counter +from ops.rotate import Rotate +from ops.fliph import FlipH +from ops.flipv import FlipV +from ops.zoom import Zoom +from ops.blur import Blur +from ops.noise import Noise +from ops.translate import Translate +from skimage.io import imread, imsave + +EXTENSIONS = ['png', 'jpg', 'jpeg', 'bmp'] +WORKER_COUNT = max(cpu_count() - 1, 1) +OPERATIONS = [Rotate, FlipH, FlipV, Translate, Noise, Zoom, Blur] + +''' +Augmented files will have names matching the regex below, eg + + original__rot90__crop1__flipv.jpg + +''' +AUGMENTED_FILE_REGEX = re.compile('^.*(__.+)+\\.[^\\.]+$') +EXTENSION_REGEX = re.compile('|'.join(map(lambda n : '.*\\.' + n + '$', EXTENSIONS))) + +thread_pool = None +counter = None + +def build_augmented_file_name(original_name, ops): + root, ext = os.path.splitext(original_name) + result = root + for op in ops: + result += '__' + op.code + return result + ext + +def work(d, f, op_lists): + try: + in_path = os.path.join(d,f) + for op_list in op_lists: + out_file_name = build_augmented_file_name(f, op_list) + if isfile(os.path.join(d,out_file_name)): + continue + img = imread(in_path) + for op in op_list: + img = op.process(img) + imsave(os.path.join(d, out_file_name), img) + + counter.processed() + except: + traceback.print_exc(file=sys.stdout) + +def process(dir, file, op_lists): + thread_pool.apply_async(work, (dir, file, op_lists)) + +if __name__ == '__main__': + if len(sys.argv) < 3: + print 'Usage: {} ( ...)'.format(sys.argv[0]) + sys.exit(1) + + image_dir = sys.argv[1] + if not os.path.isdir(image_dir): + print 'Invalid image directory: {}'.format(image_dir) + sys.exit(2) + + op_codes = sys.argv[2:] + op_lists = [] + for op_code_list in op_codes: + op_list = [] + for op_code in op_code_list.split(','): + op = None + for op in OPERATIONS: + op = op.match_code(op_code) + if op: + op_list.append(op) + break + + if not op: + print 'Unknown operation {}'.format(op_code) + sys.exit(3) + op_lists.append(op_list) + + counter = Counter() + thread_pool = Pool(WORKER_COUNT) + print 'Thread pool initialised with {} worker{}'.format(WORKER_COUNT, '' if WORKER_COUNT == 1 else 's') + + matches = [] + for dir_info in os.walk(image_dir): + dir_name, _, file_names = dir_info + print 'Processing {}...'.format(dir_name) + + for file_name in file_names: + if EXTENSION_REGEX.match(file_name): + if AUGMENTED_FILE_REGEX.match(file_name): + counter.skipped_augmented() + else: + process(dir_name, file_name, op_lists) + else: + counter.skipped_no_match() + + print "Waiting for workers to complete..." + thread_pool.close() + thread_pool.join() + + print counter.get() diff --git a/Data-Model/Training/input/augmentor/ops/__init__.py b/Data-Model/Training/input/augmentor/ops/__init__.py new file mode 100755 index 0000000..e69de29 diff --git a/Data-Model/Training/input/augmentor/ops/blur.py b/Data-Model/Training/input/augmentor/ops/blur.py new file mode 100755 index 0000000..d6469ff --- /dev/null +++ b/Data-Model/Training/input/augmentor/ops/blur.py @@ -0,0 +1,22 @@ +from skimage.filters import gaussian +from skimage.exposure import rescale_intensity +import re + +CODE = 'blur' +REGEX = re.compile(r"^" + CODE + "_(?P[.0-9]+)") + +class Blur: + def __init__(self, sigma): + self.code = CODE + str(sigma) + self.sigma = sigma + + def process(self, img): + is_colour = len(img.shape)==3 + return rescale_intensity(gaussian(img, sigma=self.sigma, multichannel=is_colour)) + + @staticmethod + def match_code(code): + match = REGEX.match(code) + if match: + d = match.groupdict() + return Blur(float(d['sigma'])) diff --git a/Data-Model/Training/input/augmentor/ops/fliph.py b/Data-Model/Training/input/augmentor/ops/fliph.py new file mode 100755 index 0000000..6b8c156 --- /dev/null +++ b/Data-Model/Training/input/augmentor/ops/fliph.py @@ -0,0 +1,15 @@ +import numpy as np + +CODE = 'fliph' + +class FlipH: + def __init__(self): + self.code = CODE + + def process(self, img): + return np.fliplr(img) + + @staticmethod + def match_code(code): + if code == CODE: + return FlipH() \ No newline at end of file diff --git a/Data-Model/Training/input/augmentor/ops/flipv.py b/Data-Model/Training/input/augmentor/ops/flipv.py new file mode 100755 index 0000000..50f5ecd --- /dev/null +++ b/Data-Model/Training/input/augmentor/ops/flipv.py @@ -0,0 +1,15 @@ +import numpy as np + +CODE = 'flipv' + +class FlipV: + def __init__(self): + self.code = CODE + + def process(self, img): + return np.flipud(img) + + @staticmethod + def match_code(code): + if code == CODE: + return FlipV() \ No newline at end of file diff --git a/Data-Model/Training/input/augmentor/ops/noise.py b/Data-Model/Training/input/augmentor/ops/noise.py new file mode 100755 index 0000000..abf0a55 --- /dev/null +++ b/Data-Model/Training/input/augmentor/ops/noise.py @@ -0,0 +1,20 @@ +from skimage.util import random_noise +import re + +CODE = 'noise' +REGEX = re.compile(r"^" + CODE + "_(?P[.0-9]+)") + +class Noise: + def __init__(self, var): + self.code = CODE + str(var) + self.var = var + + def process(self, img): + return random_noise(img, mode='gaussian', var=self.var) + + @staticmethod + def match_code(code): + match = REGEX.match(code) + if match: + d = match.groupdict() + return Noise(float(d['var'])) diff --git a/Data-Model/Training/input/augmentor/ops/rotate.py b/Data-Model/Training/input/augmentor/ops/rotate.py new file mode 100755 index 0000000..a8886bc --- /dev/null +++ b/Data-Model/Training/input/augmentor/ops/rotate.py @@ -0,0 +1,20 @@ +from skimage import transform +import re + +PREFIX = 'rot' +REGEX = re.compile(r"^" + PREFIX + "_(?P-?[0-9]+)") + +class Rotate: + def __init__(self, angle): + self.angle = angle + self.code = PREFIX + str(angle) + + def process(self, img): + return transform.rotate(img, -self.angle, cval=1) + + @staticmethod + def match_code(code): + match = REGEX.match(code) + if match: + d = match.groupdict() + return Rotate(int(d['angle'])) diff --git a/Data-Model/Training/input/augmentor/ops/translate.py b/Data-Model/Training/input/augmentor/ops/translate.py new file mode 100755 index 0000000..6d4d26a --- /dev/null +++ b/Data-Model/Training/input/augmentor/ops/translate.py @@ -0,0 +1,22 @@ +from skimage.transform import AffineTransform +from skimage import transform as tf +import re + +CODE = 'trans' +REGEX = re.compile(r"^" + CODE + "_(?P[-0-9]+)_(?P[-0-9]+)") + +class Translate: + def __init__(self, x_trans, y_trans): + self.code = CODE + str(x_trans) + '_' + str(y_trans) + self.x_trans = x_trans + self.y_trans = y_trans + + def process(self, img): + return tf.warp(img, AffineTransform(translation=(-self.x_trans, -self.y_trans)), cval=1) + + @staticmethod + def match_code(code): + match = REGEX.match(code) + if match: + d = match.groupdict() + return Translate(int(d['x_trans']), int(d['y_trans'])) diff --git a/Data-Model/Training/input/augmentor/ops/zoom.py b/Data-Model/Training/input/augmentor/ops/zoom.py new file mode 100755 index 0000000..ade41d3 --- /dev/null +++ b/Data-Model/Training/input/augmentor/ops/zoom.py @@ -0,0 +1,46 @@ +from skimage import transform +import numpy as np +import re + +PREFIX = 'zoom' +REGEX = re.compile(r"^" + PREFIX + "_(?P[-0-9]+)_(?P[-0-9]+)_(?P[-0-9]+)_(?P[-0-9]+)") +PAD_VALUE = 0 + +class Zoom: + def __init__(self, p1x, p1y, p2x, p2y): + self.p1x = p1x + self.p1y = p1y + self.p2x = p2x + self.p2y = p2y + self.code = PREFIX + str(p1x) + '_' + str(p1y) + '_' + str(p2x) + '_' + str(p2y) + + def process(self, img): + h = len(img) + w = len(img[0]) + + crop_p1x = max(self.p1x, 0) + crop_p1y = max(self.p1y, 0) + crop_p2x = min(self.p2x, w) + crop_p2y = min(self.p2y, h) + + cropped_img = img[crop_p1y:crop_p2y, crop_p1x:crop_p2x] + + x_pad_before = -min(0, self.p1x) + x_pad_after = max(0, self.p2x-w) + y_pad_before = -min(0, self.p1y) + y_pad_after = max(0, self.p2y-h) + + padding = [(y_pad_before, y_pad_after), (x_pad_before, x_pad_after)] + is_colour = len(img.shape) == 3 + if is_colour: + padding.append((0,0)) # colour images have an extra dimension + + padded_img = np.pad(cropped_img, padding, 'constant') + return transform.resize(padded_img, (h,w)) + + @staticmethod + def match_code(code): + match = REGEX.match(code) + if match: + d = match.groupdict() + return Zoom(int(d['p1x']), int(d['p1y']), int(d['p2x']), int(d['p2y'])) diff --git a/Data-Model/Training/input/normalize.sh b/Data-Model/Training/input/normalize.sh new file mode 100755 index 0000000..1a80fd8 --- /dev/null +++ b/Data-Model/Training/input/normalize.sh @@ -0,0 +1,28 @@ +#!/bin/bash +set -e + +# +# MLMOJI +# +# This file is part of Alexis Aubry's WWDC18 scolarship submission open source project. +# +# Copyright © 2018 Alexis Aubry. Available under the terms of the MIT License. +# + +RAW_DIR=$(pwd)/augmented +NORMALIZED_DIR=$(pwd)/normalized +MOBILENET_IMAGE_SIZE=224 + +# Preparation +rm -rf $NORMALIZED_DIR +cp -R $RAW_DIR $NORMALIZED_DIR + +# Rezsizing +cd $NORMALIZED_DIR +sips -Z $MOBILENET_IMAGE_SIZE heart/*.jpg +sips -Z $MOBILENET_IMAGE_SIZE cloud/*.jpg +sips -Z $MOBILENET_IMAGE_SIZE sun/*.jpg +sips -Z $MOBILENET_IMAGE_SIZE smile/*.jpg +sips -Z $MOBILENET_IMAGE_SIZE laugh/*.jpg +sips -Z $MOBILENET_IMAGE_SIZE croissant/*.jpg +sips -Z $MOBILENET_IMAGE_SIZE checkmark/*.jpg diff --git a/Data-Model/Training/input/training-data/checkmark/.gitkeep b/Data-Model/Training/input/training-data/checkmark/.gitkeep new file mode 100644 index 0000000..e69de29 diff --git a/Data-Model/Training/input/training-data/cloud/.gitkeep b/Data-Model/Training/input/training-data/cloud/.gitkeep new file mode 100644 index 0000000..e69de29 diff --git a/Data-Model/Training/input/training-data/croissant/.gitkeep b/Data-Model/Training/input/training-data/croissant/.gitkeep new file mode 100644 index 0000000..e69de29 diff --git a/Data-Model/Training/input/training-data/heart/.gitkeep b/Data-Model/Training/input/training-data/heart/.gitkeep new file mode 100644 index 0000000..e69de29 diff --git a/Data-Model/Training/input/training-data/laugh/.gitkeep b/Data-Model/Training/input/training-data/laugh/.gitkeep new file mode 100644 index 0000000..e69de29 diff --git a/Data-Model/Training/input/training-data/smile/.gitkeep b/Data-Model/Training/input/training-data/smile/.gitkeep new file mode 100644 index 0000000..e69de29 diff --git a/Data-Model/Training/input/training-data/sun/.gitkeep b/Data-Model/Training/input/training-data/sun/.gitkeep new file mode 100644 index 0000000..e69de29 diff --git a/Data-Model/Training/output/.gitkeep b/Data-Model/Training/output/.gitkeep new file mode 100644 index 0000000..e69de29 diff --git a/Data-Model/Training/prepare.sh b/Data-Model/Training/prepare.sh new file mode 100755 index 0000000..01d7380 --- /dev/null +++ b/Data-Model/Training/prepare.sh @@ -0,0 +1,28 @@ +#!/bin/bash +set -e + +# +# MLMOJI +# +# This file is part of Alexis Aubry's WWDC18 scolarship submission open source project. +# +# Copyright © 2018 Alexis Aubry. Available under the terms of the MIT License. +# + +# Install dependencies +echo "👉 Installing dependencies..." +pip install -U tfcoreml six numpy scipy scikit-image opencv-python imgaug + +# Normalize images +echo "👉 Normalizing images..." +cd input +sh ./normalize.sh + +# Create output directory +echo "👉 Creating output directory..." +cd .. +rm -rf output && mkdir output + +# Building TensorFlow image +echo "👉 Building Docker image" +docker build -t train-classifier . diff --git a/Data-Model/Training/retrain.sh b/Data-Model/Training/retrain.sh new file mode 100755 index 0000000..9c65f21 --- /dev/null +++ b/Data-Model/Training/retrain.sh @@ -0,0 +1,18 @@ +#!/bin/bash +set -e + +# +# MLMOJI +# +# This file is part of Alexis Aubry's WWDC18 scolarship submission open source project. +# +# Copyright © 2018 Alexis Aubry. Available under the terms of the MIT License. +# + +# Train classifier +echo "👉 Retraining model..." +docker run -it -v $(pwd)/output:/output \-v $(pwd)/input/normalized:/input train-classifier + +# Export MLModel +echo "👉 Exporting model..." +python "export.py" diff --git a/Data-Model/Utilities/ModelTesting/ModelTesting.xcodeproj/project.pbxproj b/Data-Model/Utilities/ModelTesting/ModelTesting.xcodeproj/project.pbxproj new file mode 100644 index 0000000..68a8dab --- /dev/null +++ b/Data-Model/Utilities/ModelTesting/ModelTesting.xcodeproj/project.pbxproj @@ -0,0 +1,408 @@ +// !$*UTF8*$! +{ + archiveVersion = 1; + classes = { + }; + objectVersion = 48; + objects = { + +/* Begin PBXBuildFile section */ + 552FFB702071FBD600B0E10E /* Paper.swift in Sources */ = {isa = PBXBuildFile; fileRef = 552FFB5D2071FBD600B0E10E /* Paper.swift */; }; + 552FFB712071FBD600B0E10E /* BitmapPaperExporter.swift in Sources */ = {isa = PBXBuildFile; fileRef = 552FFB5F2071FBD600B0E10E /* BitmapPaperExporter.swift */; }; + 552FFB722071FBD600B0E10E /* PaperExporter.swift in Sources */ = {isa = PBXBuildFile; fileRef = 552FFB602071FBD600B0E10E /* PaperExporter.swift */; }; + 552FFB732071FBD600B0E10E /* CVPixelBufferExporter.swift in Sources */ = {isa = PBXBuildFile; fileRef = 552FFB612071FBD600B0E10E /* CVPixelBufferExporter.swift */; }; + 552FFB762071FBD600B0E10E /* Classes.swift in Sources */ = {isa = PBXBuildFile; fileRef = 552FFB662071FBD600B0E10E /* Classes.swift */; }; + 552FFB772071FBD600B0E10E /* PredictionSessionDelegate.swift in Sources */ = {isa = PBXBuildFile; fileRef = 552FFB672071FBD600B0E10E /* PredictionSessionDelegate.swift */; }; + 552FFB792071FBD600B0E10E /* PredictionSession.swift in Sources */ = {isa = PBXBuildFile; fileRef = 552FFB692071FBD600B0E10E /* PredictionSession.swift */; }; + 552FFB802071FC6600B0E10E /* PaperContainerView.swift in Sources */ = {isa = PBXBuildFile; fileRef = 552FFB7F2071FC6600B0E10E /* PaperContainerView.swift */; }; + 55319E4A206681B100B866DD /* EmojiSketches.mlmodel in Sources */ = {isa = PBXBuildFile; fileRef = 55319E49206681B100B866DD /* EmojiSketches.mlmodel */; }; + 556DED2820652EA700094928 /* AppDelegate.swift in Sources */ = {isa = PBXBuildFile; fileRef = 556DED2720652EA700094928 /* AppDelegate.swift */; }; + 556DED2A20652EA700094928 /* ViewController.swift in Sources */ = {isa = PBXBuildFile; fileRef = 556DED2920652EA700094928 /* ViewController.swift */; }; + 556DED2D20652EA700094928 /* Main.storyboard in Resources */ = {isa = PBXBuildFile; fileRef = 556DED2B20652EA700094928 /* Main.storyboard */; }; + 556DED2F20652EA700094928 /* Assets.xcassets in Resources */ = {isa = PBXBuildFile; fileRef = 556DED2E20652EA700094928 /* Assets.xcassets */; }; + 556DED3220652EA700094928 /* LaunchScreen.storyboard in Resources */ = {isa = PBXBuildFile; fileRef = 556DED3020652EA700094928 /* LaunchScreen.storyboard */; }; +/* End PBXBuildFile section */ + +/* Begin PBXFileReference section */ + 552FFB5D2071FBD600B0E10E /* Paper.swift */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.swift; path = Paper.swift; sourceTree = ""; }; + 552FFB5F2071FBD600B0E10E /* BitmapPaperExporter.swift */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.swift; path = BitmapPaperExporter.swift; sourceTree = ""; }; + 552FFB602071FBD600B0E10E /* PaperExporter.swift */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.swift; path = PaperExporter.swift; sourceTree = ""; }; + 552FFB612071FBD600B0E10E /* CVPixelBufferExporter.swift */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.swift; path = CVPixelBufferExporter.swift; sourceTree = ""; }; + 552FFB662071FBD600B0E10E /* Classes.swift */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.swift; path = Classes.swift; sourceTree = ""; }; + 552FFB672071FBD600B0E10E /* PredictionSessionDelegate.swift */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.swift; path = PredictionSessionDelegate.swift; sourceTree = ""; }; + 552FFB692071FBD600B0E10E /* PredictionSession.swift */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.swift; path = PredictionSession.swift; sourceTree = ""; }; + 552FFB7F2071FC6600B0E10E /* PaperContainerView.swift */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.swift; path = PaperContainerView.swift; sourceTree = ""; }; + 55319E49206681B100B866DD /* EmojiSketches.mlmodel */ = {isa = PBXFileReference; lastKnownFileType = file.mlmodel; name = EmojiSketches.mlmodel; path = ../../../EmojiSketches.mlmodel; sourceTree = ""; }; + 556DED2420652EA700094928 /* ModelTesting.app */ = {isa = PBXFileReference; explicitFileType = wrapper.application; includeInIndex = 0; path = ModelTesting.app; sourceTree = BUILT_PRODUCTS_DIR; }; + 556DED2720652EA700094928 /* AppDelegate.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = AppDelegate.swift; sourceTree = ""; }; + 556DED2920652EA700094928 /* ViewController.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = ViewController.swift; sourceTree = ""; }; + 556DED2C20652EA700094928 /* Base */ = {isa = PBXFileReference; lastKnownFileType = file.storyboard; name = Base; path = Base.lproj/Main.storyboard; sourceTree = ""; }; + 556DED2E20652EA700094928 /* Assets.xcassets */ = {isa = PBXFileReference; lastKnownFileType = folder.assetcatalog; path = Assets.xcassets; sourceTree = ""; }; + 556DED3120652EA700094928 /* Base */ = {isa = PBXFileReference; lastKnownFileType = file.storyboard; name = Base; path = Base.lproj/LaunchScreen.storyboard; sourceTree = ""; }; + 556DED3320652EA700094928 /* Info.plist */ = {isa = PBXFileReference; lastKnownFileType = text.plist.xml; path = Info.plist; sourceTree = ""; }; +/* End PBXFileReference section */ + +/* Begin PBXFrameworksBuildPhase section */ + 556DED2120652EA700094928 /* Frameworks */ = { + isa = PBXFrameworksBuildPhase; + buildActionMask = 2147483647; + files = ( + ); + runOnlyForDeploymentPostprocessing = 0; + }; +/* End PBXFrameworksBuildPhase section */ + +/* Begin PBXGroup section */ + 552FFB5B2071FBD600B0E10E /* Sources */ = { + isa = PBXGroup; + children = ( + 552FFB5C2071FBD600B0E10E /* Paper */, + 552FFB652071FBD600B0E10E /* Prediction */, + ); + name = Sources; + path = ../../../../Playground/MLMOJI.playgroundbook/Contents/Sources; + sourceTree = ""; + }; + 552FFB5C2071FBD600B0E10E /* Paper */ = { + isa = PBXGroup; + children = ( + 552FFB5D2071FBD600B0E10E /* Paper.swift */, + 552FFB5E2071FBD600B0E10E /* Exporters */, + ); + path = Paper; + sourceTree = ""; + }; + 552FFB5E2071FBD600B0E10E /* Exporters */ = { + isa = PBXGroup; + children = ( + 552FFB5F2071FBD600B0E10E /* BitmapPaperExporter.swift */, + 552FFB602071FBD600B0E10E /* PaperExporter.swift */, + 552FFB612071FBD600B0E10E /* CVPixelBufferExporter.swift */, + ); + path = Exporters; + sourceTree = ""; + }; + 552FFB652071FBD600B0E10E /* Prediction */ = { + isa = PBXGroup; + children = ( + 552FFB662071FBD600B0E10E /* Classes.swift */, + 552FFB672071FBD600B0E10E /* PredictionSessionDelegate.swift */, + 552FFB692071FBD600B0E10E /* PredictionSession.swift */, + ); + path = Prediction; + sourceTree = ""; + }; + 55319E3F206593A600B866DD /* Supporting */ = { + isa = PBXGroup; + children = ( + 552FFB7F2071FC6600B0E10E /* PaperContainerView.swift */, + 556DED2720652EA700094928 /* AppDelegate.swift */, + 556DED2E20652EA700094928 /* Assets.xcassets */, + 556DED2B20652EA700094928 /* Main.storyboard */, + 556DED3020652EA700094928 /* LaunchScreen.storyboard */, + 556DED3320652EA700094928 /* Info.plist */, + ); + name = Supporting; + sourceTree = ""; + }; + 556DED1B20652EA700094928 = { + isa = PBXGroup; + children = ( + 556DED2620652EA700094928 /* ModelTesting */, + 556DED2520652EA700094928 /* Products */, + ); + sourceTree = ""; + }; + 556DED2520652EA700094928 /* Products */ = { + isa = PBXGroup; + children = ( + 556DED2420652EA700094928 /* ModelTesting.app */, + ); + name = Products; + sourceTree = ""; + }; + 556DED2620652EA700094928 /* ModelTesting */ = { + isa = PBXGroup; + children = ( + 55319E49206681B100B866DD /* EmojiSketches.mlmodel */, + 556DED2920652EA700094928 /* ViewController.swift */, + 552FFB5B2071FBD600B0E10E /* Sources */, + 55319E3F206593A600B866DD /* Supporting */, + ); + path = ModelTesting; + sourceTree = ""; + }; +/* End PBXGroup section */ + +/* Begin PBXNativeTarget section */ + 556DED2320652EA700094928 /* ModelTesting */ = { + isa = PBXNativeTarget; + buildConfigurationList = 556DED3620652EA700094928 /* Build configuration list for PBXNativeTarget "ModelTesting" */; + buildPhases = ( + 556DED2020652EA700094928 /* Sources */, + 556DED2120652EA700094928 /* Frameworks */, + 556DED2220652EA700094928 /* Resources */, + ); + buildRules = ( + ); + dependencies = ( + ); + name = ModelTesting; + productName = TestDrawing; + productReference = 556DED2420652EA700094928 /* ModelTesting.app */; + productType = "com.apple.product-type.application"; + }; +/* End PBXNativeTarget section */ + +/* Begin PBXProject section */ + 556DED1C20652EA700094928 /* Project object */ = { + isa = PBXProject; + attributes = { + LastSwiftUpdateCheck = 0920; + LastUpgradeCheck = 0930; + ORGANIZATIONNAME = "Alexis Aubry"; + TargetAttributes = { + 556DED2320652EA700094928 = { + CreatedOnToolsVersion = 9.2; + ProvisioningStyle = Automatic; + }; + }; + }; + buildConfigurationList = 556DED1F20652EA700094928 /* Build configuration list for PBXProject "ModelTesting" */; + compatibilityVersion = "Xcode 8.0"; + developmentRegion = en; + hasScannedForEncodings = 0; + knownRegions = ( + en, + Base, + ); + mainGroup = 556DED1B20652EA700094928; + productRefGroup = 556DED2520652EA700094928 /* Products */; + projectDirPath = ""; + projectRoot = ""; + targets = ( + 556DED2320652EA700094928 /* ModelTesting */, + ); + }; +/* End PBXProject section */ + +/* Begin PBXResourcesBuildPhase section */ + 556DED2220652EA700094928 /* Resources */ = { + isa = PBXResourcesBuildPhase; + buildActionMask = 2147483647; + files = ( + 556DED3220652EA700094928 /* LaunchScreen.storyboard in Resources */, + 556DED2F20652EA700094928 /* Assets.xcassets in Resources */, + 556DED2D20652EA700094928 /* Main.storyboard in Resources */, + ); + runOnlyForDeploymentPostprocessing = 0; + }; +/* End PBXResourcesBuildPhase section */ + +/* Begin PBXSourcesBuildPhase section */ + 556DED2020652EA700094928 /* Sources */ = { + isa = PBXSourcesBuildPhase; + buildActionMask = 2147483647; + files = ( + 552FFB732071FBD600B0E10E /* CVPixelBufferExporter.swift in Sources */, + 552FFB792071FBD600B0E10E /* PredictionSession.swift in Sources */, + 556DED2A20652EA700094928 /* ViewController.swift in Sources */, + 552FFB722071FBD600B0E10E /* PaperExporter.swift in Sources */, + 55319E4A206681B100B866DD /* EmojiSketches.mlmodel in Sources */, + 552FFB802071FC6600B0E10E /* PaperContainerView.swift in Sources */, + 552FFB702071FBD600B0E10E /* Paper.swift in Sources */, + 552FFB712071FBD600B0E10E /* BitmapPaperExporter.swift in Sources */, + 552FFB762071FBD600B0E10E /* Classes.swift in Sources */, + 552FFB772071FBD600B0E10E /* PredictionSessionDelegate.swift in Sources */, + 556DED2820652EA700094928 /* AppDelegate.swift in Sources */, + ); + runOnlyForDeploymentPostprocessing = 0; + }; +/* End PBXSourcesBuildPhase section */ + +/* Begin PBXVariantGroup section */ + 556DED2B20652EA700094928 /* Main.storyboard */ = { + isa = PBXVariantGroup; + children = ( + 556DED2C20652EA700094928 /* Base */, + ); + name = Main.storyboard; + sourceTree = ""; + }; + 556DED3020652EA700094928 /* LaunchScreen.storyboard */ = { + isa = PBXVariantGroup; + children = ( + 556DED3120652EA700094928 /* Base */, + ); + name = LaunchScreen.storyboard; + sourceTree = ""; + }; +/* End PBXVariantGroup section */ + +/* Begin XCBuildConfiguration section */ + 556DED3420652EA700094928 /* Debug */ = { + isa = XCBuildConfiguration; + buildSettings = { + ALWAYS_SEARCH_USER_PATHS = NO; + CLANG_ANALYZER_NONNULL = YES; + CLANG_ANALYZER_NUMBER_OBJECT_CONVERSION = YES_AGGRESSIVE; + CLANG_CXX_LANGUAGE_STANDARD = "gnu++14"; + CLANG_CXX_LIBRARY = "libc++"; + CLANG_ENABLE_MODULES = YES; + CLANG_ENABLE_OBJC_ARC = YES; + CLANG_WARN_BLOCK_CAPTURE_AUTORELEASING = YES; + CLANG_WARN_BOOL_CONVERSION = YES; + CLANG_WARN_COMMA = YES; + CLANG_WARN_CONSTANT_CONVERSION = YES; + CLANG_WARN_DEPRECATED_OBJC_IMPLEMENTATIONS = YES; + CLANG_WARN_DIRECT_OBJC_ISA_USAGE = YES_ERROR; + CLANG_WARN_DOCUMENTATION_COMMENTS = YES; + CLANG_WARN_EMPTY_BODY = YES; + CLANG_WARN_ENUM_CONVERSION = YES; + CLANG_WARN_INFINITE_RECURSION = YES; + CLANG_WARN_INT_CONVERSION = YES; + CLANG_WARN_NON_LITERAL_NULL_CONVERSION = YES; + CLANG_WARN_OBJC_IMPLICIT_RETAIN_SELF = YES; + CLANG_WARN_OBJC_LITERAL_CONVERSION = YES; + CLANG_WARN_OBJC_ROOT_CLASS = YES_ERROR; + CLANG_WARN_RANGE_LOOP_ANALYSIS = YES; + CLANG_WARN_STRICT_PROTOTYPES = YES; + CLANG_WARN_SUSPICIOUS_MOVE = YES; + CLANG_WARN_UNGUARDED_AVAILABILITY = YES_AGGRESSIVE; + CLANG_WARN_UNREACHABLE_CODE = YES; + CLANG_WARN__DUPLICATE_METHOD_MATCH = YES; + CODE_SIGN_IDENTITY = "iPhone Developer"; + COPY_PHASE_STRIP = NO; + DEBUG_INFORMATION_FORMAT = dwarf; + ENABLE_STRICT_OBJC_MSGSEND = YES; + ENABLE_TESTABILITY = YES; + GCC_C_LANGUAGE_STANDARD = gnu11; + GCC_DYNAMIC_NO_PIC = NO; + GCC_NO_COMMON_BLOCKS = YES; + GCC_OPTIMIZATION_LEVEL = 0; + GCC_PREPROCESSOR_DEFINITIONS = ( + "DEBUG=1", + "$(inherited)", + ); + GCC_WARN_64_TO_32_BIT_CONVERSION = YES; + GCC_WARN_ABOUT_RETURN_TYPE = YES_ERROR; + GCC_WARN_UNDECLARED_SELECTOR = YES; + GCC_WARN_UNINITIALIZED_AUTOS = YES_AGGRESSIVE; + GCC_WARN_UNUSED_FUNCTION = YES; + GCC_WARN_UNUSED_VARIABLE = YES; + IPHONEOS_DEPLOYMENT_TARGET = 11.2; + MTL_ENABLE_DEBUG_INFO = YES; + ONLY_ACTIVE_ARCH = YES; + SDKROOT = iphoneos; + SWIFT_ACTIVE_COMPILATION_CONDITIONS = DEBUG; + SWIFT_OPTIMIZATION_LEVEL = "-Onone"; + }; + name = Debug; + }; + 556DED3520652EA700094928 /* Release */ = { + isa = XCBuildConfiguration; + buildSettings = { + ALWAYS_SEARCH_USER_PATHS = NO; + CLANG_ANALYZER_NONNULL = YES; + CLANG_ANALYZER_NUMBER_OBJECT_CONVERSION = YES_AGGRESSIVE; + CLANG_CXX_LANGUAGE_STANDARD = "gnu++14"; + CLANG_CXX_LIBRARY = "libc++"; + CLANG_ENABLE_MODULES = YES; + CLANG_ENABLE_OBJC_ARC = YES; + CLANG_WARN_BLOCK_CAPTURE_AUTORELEASING = YES; + CLANG_WARN_BOOL_CONVERSION = YES; + CLANG_WARN_COMMA = YES; + CLANG_WARN_CONSTANT_CONVERSION = YES; + CLANG_WARN_DEPRECATED_OBJC_IMPLEMENTATIONS = YES; + CLANG_WARN_DIRECT_OBJC_ISA_USAGE = YES_ERROR; + CLANG_WARN_DOCUMENTATION_COMMENTS = YES; + CLANG_WARN_EMPTY_BODY = YES; + CLANG_WARN_ENUM_CONVERSION = YES; + CLANG_WARN_INFINITE_RECURSION = YES; + CLANG_WARN_INT_CONVERSION = YES; + CLANG_WARN_NON_LITERAL_NULL_CONVERSION = YES; + CLANG_WARN_OBJC_IMPLICIT_RETAIN_SELF = YES; + CLANG_WARN_OBJC_LITERAL_CONVERSION = YES; + CLANG_WARN_OBJC_ROOT_CLASS = YES_ERROR; + CLANG_WARN_RANGE_LOOP_ANALYSIS = YES; + CLANG_WARN_STRICT_PROTOTYPES = YES; + CLANG_WARN_SUSPICIOUS_MOVE = YES; + CLANG_WARN_UNGUARDED_AVAILABILITY = YES_AGGRESSIVE; + CLANG_WARN_UNREACHABLE_CODE = YES; + CLANG_WARN__DUPLICATE_METHOD_MATCH = YES; + CODE_SIGN_IDENTITY = "iPhone Developer"; + COPY_PHASE_STRIP = NO; + DEBUG_INFORMATION_FORMAT = "dwarf-with-dsym"; + ENABLE_NS_ASSERTIONS = NO; + ENABLE_STRICT_OBJC_MSGSEND = YES; + GCC_C_LANGUAGE_STANDARD = gnu11; + GCC_NO_COMMON_BLOCKS = YES; + GCC_WARN_64_TO_32_BIT_CONVERSION = YES; + GCC_WARN_ABOUT_RETURN_TYPE = YES_ERROR; + GCC_WARN_UNDECLARED_SELECTOR = YES; + GCC_WARN_UNINITIALIZED_AUTOS = YES_AGGRESSIVE; + GCC_WARN_UNUSED_FUNCTION = YES; + GCC_WARN_UNUSED_VARIABLE = YES; + IPHONEOS_DEPLOYMENT_TARGET = 11.2; + MTL_ENABLE_DEBUG_INFO = NO; + SDKROOT = iphoneos; + SWIFT_OPTIMIZATION_LEVEL = "-Owholemodule"; + VALIDATE_PRODUCT = YES; + }; + name = Release; + }; + 556DED3720652EA700094928 /* Debug */ = { + isa = XCBuildConfiguration; + buildSettings = { + ASSETCATALOG_COMPILER_APPICON_NAME = AppIcon; + CODE_SIGN_STYLE = Automatic; + DEVELOPMENT_TEAM = 75E8K39Z38; + INFOPLIST_FILE = "$(SRCROOT)/ModelTesting/Info.plist"; + LD_RUNPATH_SEARCH_PATHS = "$(inherited) @executable_path/Frameworks"; + PRODUCT_BUNDLE_IDENTIFIER = fr.alexaubry.wwdc18.mlmoji.ModelTraining; + PRODUCT_NAME = "$(TARGET_NAME)"; + SWIFT_VERSION = 4.0; + TARGETED_DEVICE_FAMILY = "1,2"; + }; + name = Debug; + }; + 556DED3820652EA700094928 /* Release */ = { + isa = XCBuildConfiguration; + buildSettings = { + ASSETCATALOG_COMPILER_APPICON_NAME = AppIcon; + CODE_SIGN_STYLE = Automatic; + DEVELOPMENT_TEAM = 75E8K39Z38; + INFOPLIST_FILE = "$(SRCROOT)/ModelTesting/Info.plist"; + LD_RUNPATH_SEARCH_PATHS = "$(inherited) @executable_path/Frameworks"; + PRODUCT_BUNDLE_IDENTIFIER = fr.alexaubry.wwdc18.mlmoji.ModelTraining; + PRODUCT_NAME = "$(TARGET_NAME)"; + SWIFT_VERSION = 4.0; + TARGETED_DEVICE_FAMILY = "1,2"; + }; + name = Release; + }; +/* End XCBuildConfiguration section */ + +/* Begin XCConfigurationList section */ + 556DED1F20652EA700094928 /* Build configuration list for PBXProject "ModelTesting" */ = { + isa = XCConfigurationList; + buildConfigurations = ( + 556DED3420652EA700094928 /* Debug */, + 556DED3520652EA700094928 /* Release */, + ); + defaultConfigurationIsVisible = 0; + defaultConfigurationName = Release; + }; + 556DED3620652EA700094928 /* Build configuration list for PBXNativeTarget "ModelTesting" */ = { + isa = XCConfigurationList; + buildConfigurations = ( + 556DED3720652EA700094928 /* Debug */, + 556DED3820652EA700094928 /* Release */, + ); + defaultConfigurationIsVisible = 0; + defaultConfigurationName = Release; + }; +/* End XCConfigurationList section */ + }; + rootObject = 556DED1C20652EA700094928 /* Project object */; +} diff --git a/Data-Model/Utilities/ModelTesting/ModelTesting.xcodeproj/project.xcworkspace/contents.xcworkspacedata b/Data-Model/Utilities/ModelTesting/ModelTesting.xcodeproj/project.xcworkspace/contents.xcworkspacedata new file mode 100644 index 0000000..ad9035e --- /dev/null +++ b/Data-Model/Utilities/ModelTesting/ModelTesting.xcodeproj/project.xcworkspace/contents.xcworkspacedata @@ -0,0 +1,7 @@ + + + + + diff --git a/Data-Model/Utilities/ModelTesting/ModelTesting.xcodeproj/project.xcworkspace/xcshareddata/IDEWorkspaceChecks.plist b/Data-Model/Utilities/ModelTesting/ModelTesting.xcodeproj/project.xcworkspace/xcshareddata/IDEWorkspaceChecks.plist new file mode 100644 index 0000000..18d9810 --- /dev/null +++ b/Data-Model/Utilities/ModelTesting/ModelTesting.xcodeproj/project.xcworkspace/xcshareddata/IDEWorkspaceChecks.plist @@ -0,0 +1,8 @@ + + + + + IDEDidComputeMac32BitWarning + + + diff --git a/Data-Model/Utilities/ModelTesting/ModelTesting/AppDelegate.swift b/Data-Model/Utilities/ModelTesting/ModelTesting/AppDelegate.swift new file mode 100644 index 0000000..0adfcc1 --- /dev/null +++ b/Data-Model/Utilities/ModelTesting/ModelTesting/AppDelegate.swift @@ -0,0 +1,13 @@ +// +// MLMOJI +// +// This file is part of Alexis Aubry's WWDC18 scolarship submission open source project. +// Copyright © 2018 Alexis Aubry. Available under the terms of the MIT License. +// + +import UIKit + +@UIApplicationMain +class AppDelegate: UIResponder, UIApplicationDelegate { + var window: UIWindow? +} diff --git a/Data-Model/Utilities/ModelTesting/ModelTesting/Assets.xcassets/AppIcon.appiconset/Contents.json b/Data-Model/Utilities/ModelTesting/ModelTesting/Assets.xcassets/AppIcon.appiconset/Contents.json new file mode 100644 index 0000000..1d060ed --- /dev/null +++ b/Data-Model/Utilities/ModelTesting/ModelTesting/Assets.xcassets/AppIcon.appiconset/Contents.json @@ -0,0 +1,93 @@ +{ + "images" : [ + { + "idiom" : "iphone", + "size" : "20x20", + "scale" : "2x" + }, + { + "idiom" : "iphone", + "size" : "20x20", + "scale" : "3x" + }, + { + "idiom" : "iphone", + "size" : "29x29", + "scale" : "2x" + }, + { + "idiom" : "iphone", + "size" : "29x29", + "scale" : "3x" + }, + { + "idiom" : "iphone", + "size" : "40x40", + "scale" : "2x" + }, + { + "idiom" : "iphone", + "size" : "40x40", + "scale" : "3x" + }, + { + "idiom" : "iphone", + "size" : "60x60", + "scale" : "2x" + }, + { + "idiom" : "iphone", + "size" : "60x60", + "scale" : "3x" + }, + { + "idiom" : "ipad", + "size" : "20x20", + "scale" : "1x" + }, + { + "idiom" : "ipad", + "size" : "20x20", + "scale" : "2x" + }, + { + "idiom" : "ipad", + "size" : "29x29", + "scale" : "1x" + }, + { + "idiom" : "ipad", + "size" : "29x29", + "scale" : "2x" + }, + { + "idiom" : "ipad", + "size" : "40x40", + "scale" : "1x" + }, + { + "idiom" : "ipad", + "size" : "40x40", + "scale" : "2x" + }, + { + "idiom" : "ipad", + "size" : "76x76", + "scale" : "1x" + }, + { + "idiom" : "ipad", + "size" : "76x76", + "scale" : "2x" + }, + { + "idiom" : "ipad", + "size" : "83.5x83.5", + "scale" : "2x" + } + ], + "info" : { + "version" : 1, + "author" : "xcode" + } +} \ No newline at end of file diff --git a/Data-Model/Utilities/ModelTesting/ModelTesting/Base.lproj/LaunchScreen.storyboard b/Data-Model/Utilities/ModelTesting/ModelTesting/Base.lproj/LaunchScreen.storyboard new file mode 100644 index 0000000..f83f6fd --- /dev/null +++ b/Data-Model/Utilities/ModelTesting/ModelTesting/Base.lproj/LaunchScreen.storyboard @@ -0,0 +1,25 @@ + + + + + + + + + + + + + + + + + + + + + + + + + diff --git a/Data-Model/Utilities/ModelTesting/ModelTesting/Base.lproj/Main.storyboard b/Data-Model/Utilities/ModelTesting/ModelTesting/Base.lproj/Main.storyboard new file mode 100644 index 0000000..03c13c2 --- /dev/null +++ b/Data-Model/Utilities/ModelTesting/ModelTesting/Base.lproj/Main.storyboard @@ -0,0 +1,24 @@ + + + + + + + + + + + + + + + + + + + + + + + + diff --git a/Data-Model/Utilities/ModelTesting/ModelTesting/Info.plist b/Data-Model/Utilities/ModelTesting/ModelTesting/Info.plist new file mode 100644 index 0000000..cf834bf --- /dev/null +++ b/Data-Model/Utilities/ModelTesting/ModelTesting/Info.plist @@ -0,0 +1,49 @@ + + + + + CFBundleDevelopmentRegion + $(DEVELOPMENT_LANGUAGE) + CFBundleDisplayName + ModelTesting + CFBundleExecutable + $(EXECUTABLE_NAME) + CFBundleIdentifier + $(PRODUCT_BUNDLE_IDENTIFIER) + CFBundleInfoDictionaryVersion + 6.0 + CFBundleName + $(PRODUCT_NAME) + CFBundlePackageType + APPL + CFBundleShortVersionString + 1.0 + CFBundleVersion + 1 + LSRequiresIPhoneOS + + UIFileSharingEnabled + + UILaunchStoryboardName + LaunchScreen + UIMainStoryboardFile + Main + UIRequiredDeviceCapabilities + + armv7 + + UISupportedInterfaceOrientations + + UIInterfaceOrientationPortrait + UIInterfaceOrientationLandscapeLeft + UIInterfaceOrientationLandscapeRight + + UISupportedInterfaceOrientations~ipad + + UIInterfaceOrientationPortrait + UIInterfaceOrientationPortraitUpsideDown + UIInterfaceOrientationLandscapeLeft + UIInterfaceOrientationLandscapeRight + + + diff --git a/Data-Model/Utilities/ModelTesting/ModelTesting/PaperContainerView.swift b/Data-Model/Utilities/ModelTesting/ModelTesting/PaperContainerView.swift new file mode 100644 index 0000000..9a21a3b --- /dev/null +++ b/Data-Model/Utilities/ModelTesting/ModelTesting/PaperContainerView.swift @@ -0,0 +1,44 @@ +// +// MLMOJI +// +// This file is part of Alexis Aubry's WWDC18 scolarship submission open source project. +// Copyright © 2018 Alexis Aubry. Available under the terms of the MIT License. +// + +import UIKit + +class PaperContainerView: UIView { + + let paper: Paper + + init(paper: Paper) { + self.paper = paper + super.init(frame: .zero) + createConstraints() + } + + required init?(coder aDecoder: NSCoder) { + self.paper = Paper() + super.init(coder: aDecoder) + createConstraints() + } + + private func createConstraints() { + addSubview(paper) + paper.translatesAutoresizingMaskIntoConstraints = false + paper.centerXAnchor.constraint(equalTo: centerXAnchor).isActive = true + paper.centerYAnchor.constraint(equalTo: centerYAnchor).isActive = true + paper.widthAnchor.constraint(equalToConstant: 300).isActive = true + paper.heightAnchor.constraint(equalToConstant: 300).isActive = true + + setContentCompressionResistancePriority(.defaultHigh, for: .horizontal) + setContentCompressionResistancePriority(.defaultHigh, for: .vertical) + setContentHuggingPriority(.defaultLow, for: .horizontal) + setContentHuggingPriority(.defaultLow, for: .vertical) + } + + override var intrinsicContentSize: CGSize { + return CGSize(width: 300, height: 300) + } + +} diff --git a/Data-Model/Utilities/ModelTesting/ModelTesting/ViewController.swift b/Data-Model/Utilities/ModelTesting/ModelTesting/ViewController.swift new file mode 100644 index 0000000..1cca147 --- /dev/null +++ b/Data-Model/Utilities/ModelTesting/ModelTesting/ViewController.swift @@ -0,0 +1,247 @@ +// +// MLMOJI +// +// This file is part of Alexis Aubry's WWDC18 scolarship submission open source project. +// Copyright © 2018 Alexis Aubry. Available under the terms of the MIT License. +// + +import UIKit + +/** + * The view controller that records the samples for data labels. + */ + +class ViewController: UIViewController { + + /** + * Possible states for sample collection. + */ + + enum State { + case idle + case drawing + case recognized(String) + } + + /// The state of sample collection. + private var state: State = .idle { + didSet { + refreshInterfaceForUpdatedState() + } + } + + private var session = PredictionSession() + + + // MARK: - Views + + private let currentTitleLabel = UILabel() + private let paper = Paper() + private let submitButton = UIButton() + private let stackView = UIStackView() + + // MARK: - UI Setup + + override func viewDidLoad() { + super.viewDidLoad() + session.delegate = self + + createViews() + createContraints() + view.backgroundColor = UIColor.groupTableViewBackground + } + + /** + * Creates the views to display on the data collection screen. + */ + + private func createViews() { + + currentTitleLabel.adjustsFontSizeToFitWidth = true + currentTitleLabel.textAlignment = .center + currentTitleLabel.font = UIFont.systemFont(ofSize: 30) + currentTitleLabel.numberOfLines = 0 + + paper.backgroundColor = .white + paper.isUserInteractionEnabled = false + paper.delegate = self + + let paperContainer = PaperContainerView(paper: paper) + + submitButton.setTitleColor(.white, for: .normal) + submitButton.setTitle("Save", for: .normal) + submitButton.titleLabel?.font = UIFont.systemFont(ofSize: 17, weight: .semibold) + + let buttonBackgroundColor = UIGraphicsImageRenderer(size: CGSize(width: 1, height: 1)).image { context in + UIColor(red: 0, green: 122/255, blue: 1, alpha: 1).setFill() + context.fill(CGRect(x: 0, y: 0, width: 1, height: 1)) + } + + submitButton.setBackgroundImage(buttonBackgroundColor, for: .normal) + submitButton.setContentHuggingPriority(.defaultLow, for: .horizontal) + submitButton.layer.cornerRadius = 12 + submitButton.clipsToBounds = true + + submitButton.addTarget(self, action: #selector(submitSketch), for: .touchUpInside) + + stackView.alignment = .fill + stackView.axis = .vertical + stackView.addArrangedSubview(currentTitleLabel) + stackView.addArrangedSubview(paperContainer) + stackView.addArrangedSubview(submitButton) + + view.addSubview(stackView) + + configureViews(isRegular: traitCollection.horizontalSizeClass == .regular) + refreshInterfaceForUpdatedState() + + } + + /** + * Creates the Auto Layout constraints for the view. + */ + + private func createContraints() { + stackView.translatesAutoresizingMaskIntoConstraints = false + stackView.centerXAnchor.constraint(equalTo: view.centerXAnchor).isActive = true + stackView.centerYAnchor.constraint(equalTo: view.centerYAnchor).isActive = true + + submitButton.heightAnchor.constraint(equalToConstant: 55).isActive = true + } + + /** + * Responds to size class changes. + */ + + override func traitCollectionDidChange(_ previousTraitCollection: UITraitCollection?) { + super.traitCollectionDidChange(previousTraitCollection) + configureViews(isRegular: traitCollection.horizontalSizeClass == .regular) + } + + /** + * Configures the view for adaptive presentation. + */ + + private func configureViews(isRegular: Bool) { + stackView.spacing = isRegular ? 64 : 32 + } + + // MARK: - Session + + + /** + * Refreshes the UI when the state changes. + */ + + private func refreshInterfaceForUpdatedState() { + + switch state { + case .idle: + submitButton.setTitle("Recognize", for: .normal) + currentTitleLabel.isHidden = true + submitButton.isEnabled = false + paper.isUserInteractionEnabled = true + case .drawing: + submitButton.isEnabled = true + case .recognized(let labels): + paper.isUserInteractionEnabled = false + submitButton.setTitle("Restart", for: .normal) + currentTitleLabel.text = labels + currentTitleLabel.isHidden = false + } + + } + + /** + * Attempts to save the skecth from the paper into the session. + */ + + @objc private func submitSketch() { + + switch state { + case .idle: + return + case .recognized: + paper.clear() + state = .idle + return + case .drawing: + break + } + + let exporter = CVPixelBufferExporter(size: session.inputSize, scale: 1) + + do { + let buffer = try paper.export(with: exporter) + session.requestPrediction(for: buffer) + } catch { + handleError((error as NSError).localizedDescription) + } + + } + +} + +// MARK: - PredictionSessionDelegate + PaperDelegate + +extension ViewController: PredictionSessionDelegate { + + func predictionSession(_ session: PredictionSession, didUpdatePrediction prediction: EmojiSketchesOutput) { + + let top3: [String] = prediction.final_result__0.sorted { a, b in + return a.value > b.value + }.prefix(3).flatMap { + guard let predictionClass = Class(rawValue: $0.key) else { + handleError("Invalid class name '\($0.key)'") + return nil + } + + return "\(predictionClass.emojiValue) = \($0.value)" + } + + state = .recognized(top3.joined(separator: "\n")) + + } + + func predictionSession(_ session: PredictionSession, didFailToProvidePredictionWith error: NSError) { + handleError(error.localizedDescription) + } + +} + +extension ViewController: PaperDelegate { + + func paperDidStartDrawingStroke(_ paper: Paper) { + // no-op + } + + func paperDidFinishDrawingStroke(_ paper: Paper) { + guard case .idle = state else { + return + } + + state = .drawing + } + +} + +// MARK: - Error Handling + +extension ViewController { + + func handleError(_ description: String) { + + let alert = UIAlertController(title: "Could not recognize emoji", + message: description, + preferredStyle: .alert) + + let cancelAction = UIAlertAction(title: "Cancel", style: .cancel, handler: { _ in + self.paper.clear() + }) + + alert.addAction(cancelAction) + present(alert, animated: true) + + } + +} diff --git a/Data-Model/Utilities/SampleCollecting/SampleCollecting.xcodeproj/project.pbxproj b/Data-Model/Utilities/SampleCollecting/SampleCollecting.xcodeproj/project.pbxproj new file mode 100644 index 0000000..47098aa --- /dev/null +++ b/Data-Model/Utilities/SampleCollecting/SampleCollecting.xcodeproj/project.pbxproj @@ -0,0 +1,416 @@ +// !$*UTF8*$! +{ + archiveVersion = 1; + classes = { + }; + objectVersion = 48; + objects = { + +/* Begin PBXBuildFile section */ + 5508E2E0206567DA00EBEB83 /* Session.swift in Sources */ = {isa = PBXBuildFile; fileRef = 5508E2DF206567DA00EBEB83 /* Session.swift */; }; + 5508E2E22065873A00EBEB83 /* Storage.swift in Sources */ = {isa = PBXBuildFile; fileRef = 5508E2E12065873A00EBEB83 /* Storage.swift */; }; + 552FFBBA2072003F00B0E10E /* Paper.swift in Sources */ = {isa = PBXBuildFile; fileRef = 552FFBA72072003F00B0E10E /* Paper.swift */; }; + 552FFBBB2072003F00B0E10E /* BitmapPaperExporter.swift in Sources */ = {isa = PBXBuildFile; fileRef = 552FFBA92072003F00B0E10E /* BitmapPaperExporter.swift */; }; + 552FFBBC2072003F00B0E10E /* PaperExporter.swift in Sources */ = {isa = PBXBuildFile; fileRef = 552FFBAA2072003F00B0E10E /* PaperExporter.swift */; }; + 552FFBBD2072003F00B0E10E /* CVPixelBufferExporter.swift in Sources */ = {isa = PBXBuildFile; fileRef = 552FFBAB2072003F00B0E10E /* CVPixelBufferExporter.swift */; }; + 552FFBC02072003F00B0E10E /* Classes.swift in Sources */ = {isa = PBXBuildFile; fileRef = 552FFBB02072003F00B0E10E /* Classes.swift */; }; + 552FFBC12072003F00B0E10E /* PredictionSessionDelegate.swift in Sources */ = {isa = PBXBuildFile; fileRef = 552FFBB12072003F00B0E10E /* PredictionSessionDelegate.swift */; }; + 552FFBC22072003F00B0E10E /* EmojiSketches.swift in Sources */ = {isa = PBXBuildFile; fileRef = 552FFBB22072003F00B0E10E /* EmojiSketches.swift */; }; + 552FFBC32072003F00B0E10E /* PredictionSession.swift in Sources */ = {isa = PBXBuildFile; fileRef = 552FFBB32072003F00B0E10E /* PredictionSession.swift */; }; + 55319E41206629AC00B866DD /* PaperContainerView.swift in Sources */ = {isa = PBXBuildFile; fileRef = 55319E40206629AC00B866DD /* PaperContainerView.swift */; }; + 556DED2820652EA700094928 /* AppDelegate.swift in Sources */ = {isa = PBXBuildFile; fileRef = 556DED2720652EA700094928 /* AppDelegate.swift */; }; + 556DED2A20652EA700094928 /* ViewController.swift in Sources */ = {isa = PBXBuildFile; fileRef = 556DED2920652EA700094928 /* ViewController.swift */; }; + 556DED2D20652EA700094928 /* Main.storyboard in Resources */ = {isa = PBXBuildFile; fileRef = 556DED2B20652EA700094928 /* Main.storyboard */; }; + 556DED2F20652EA700094928 /* Assets.xcassets in Resources */ = {isa = PBXBuildFile; fileRef = 556DED2E20652EA700094928 /* Assets.xcassets */; }; + 556DED3220652EA700094928 /* LaunchScreen.storyboard in Resources */ = {isa = PBXBuildFile; fileRef = 556DED3020652EA700094928 /* LaunchScreen.storyboard */; }; +/* End PBXBuildFile section */ + +/* Begin PBXFileReference section */ + 5508E2DF206567DA00EBEB83 /* Session.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = Session.swift; sourceTree = ""; }; + 5508E2E12065873A00EBEB83 /* Storage.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = Storage.swift; sourceTree = ""; }; + 552FFBA72072003F00B0E10E /* Paper.swift */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.swift; path = Paper.swift; sourceTree = ""; }; + 552FFBA92072003F00B0E10E /* BitmapPaperExporter.swift */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.swift; path = BitmapPaperExporter.swift; sourceTree = ""; }; + 552FFBAA2072003F00B0E10E /* PaperExporter.swift */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.swift; path = PaperExporter.swift; sourceTree = ""; }; + 552FFBAB2072003F00B0E10E /* CVPixelBufferExporter.swift */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.swift; path = CVPixelBufferExporter.swift; sourceTree = ""; }; + 552FFBB02072003F00B0E10E /* Classes.swift */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.swift; path = Classes.swift; sourceTree = ""; }; + 552FFBB12072003F00B0E10E /* PredictionSessionDelegate.swift */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.swift; path = PredictionSessionDelegate.swift; sourceTree = ""; }; + 552FFBB22072003F00B0E10E /* EmojiSketches.swift */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.swift; path = EmojiSketches.swift; sourceTree = ""; }; + 552FFBB32072003F00B0E10E /* PredictionSession.swift */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.swift; path = PredictionSession.swift; sourceTree = ""; }; + 55319E40206629AC00B866DD /* PaperContainerView.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = PaperContainerView.swift; sourceTree = ""; }; + 556DED2420652EA700094928 /* SampleCollecting.app */ = {isa = PBXFileReference; explicitFileType = wrapper.application; includeInIndex = 0; path = SampleCollecting.app; sourceTree = BUILT_PRODUCTS_DIR; }; + 556DED2720652EA700094928 /* AppDelegate.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = AppDelegate.swift; sourceTree = ""; }; + 556DED2920652EA700094928 /* ViewController.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = ViewController.swift; sourceTree = ""; }; + 556DED2C20652EA700094928 /* Base */ = {isa = PBXFileReference; lastKnownFileType = file.storyboard; name = Base; path = Base.lproj/Main.storyboard; sourceTree = ""; }; + 556DED2E20652EA700094928 /* Assets.xcassets */ = {isa = PBXFileReference; lastKnownFileType = folder.assetcatalog; path = Assets.xcassets; sourceTree = ""; }; + 556DED3120652EA700094928 /* Base */ = {isa = PBXFileReference; lastKnownFileType = file.storyboard; name = Base; path = Base.lproj/LaunchScreen.storyboard; sourceTree = ""; }; + 556DED3320652EA700094928 /* Info.plist */ = {isa = PBXFileReference; lastKnownFileType = text.plist.xml; path = Info.plist; sourceTree = ""; }; +/* End PBXFileReference section */ + +/* Begin PBXFrameworksBuildPhase section */ + 556DED2120652EA700094928 /* Frameworks */ = { + isa = PBXFrameworksBuildPhase; + buildActionMask = 2147483647; + files = ( + ); + runOnlyForDeploymentPostprocessing = 0; + }; +/* End PBXFrameworksBuildPhase section */ + +/* Begin PBXGroup section */ + 552FFBA52072003F00B0E10E /* Sources */ = { + isa = PBXGroup; + children = ( + 552FFBA62072003F00B0E10E /* Paper */, + 552FFBAF2072003F00B0E10E /* Prediction */, + ); + name = Sources; + path = ../../../../Playground/MLMOJI.playgroundbook/Contents/Sources; + sourceTree = ""; + }; + 552FFBA62072003F00B0E10E /* Paper */ = { + isa = PBXGroup; + children = ( + 552FFBA72072003F00B0E10E /* Paper.swift */, + 552FFBA82072003F00B0E10E /* Exporters */, + ); + path = Paper; + sourceTree = ""; + }; + 552FFBA82072003F00B0E10E /* Exporters */ = { + isa = PBXGroup; + children = ( + 552FFBA92072003F00B0E10E /* BitmapPaperExporter.swift */, + 552FFBAA2072003F00B0E10E /* PaperExporter.swift */, + 552FFBAB2072003F00B0E10E /* CVPixelBufferExporter.swift */, + ); + path = Exporters; + sourceTree = ""; + }; + 552FFBAF2072003F00B0E10E /* Prediction */ = { + isa = PBXGroup; + children = ( + 552FFBB02072003F00B0E10E /* Classes.swift */, + 552FFBB12072003F00B0E10E /* PredictionSessionDelegate.swift */, + 552FFBB22072003F00B0E10E /* EmojiSketches.swift */, + 552FFBB32072003F00B0E10E /* PredictionSession.swift */, + ); + path = Prediction; + sourceTree = ""; + }; + 55319E3F206593A600B866DD /* Supporting */ = { + isa = PBXGroup; + children = ( + 556DED2720652EA700094928 /* AppDelegate.swift */, + 55319E40206629AC00B866DD /* PaperContainerView.swift */, + 556DED2E20652EA700094928 /* Assets.xcassets */, + 556DED2B20652EA700094928 /* Main.storyboard */, + 556DED3020652EA700094928 /* LaunchScreen.storyboard */, + 556DED3320652EA700094928 /* Info.plist */, + ); + name = Supporting; + sourceTree = ""; + }; + 556DED1B20652EA700094928 = { + isa = PBXGroup; + children = ( + 556DED2620652EA700094928 /* SampleCollecting */, + 556DED2520652EA700094928 /* Products */, + ); + sourceTree = ""; + }; + 556DED2520652EA700094928 /* Products */ = { + isa = PBXGroup; + children = ( + 556DED2420652EA700094928 /* SampleCollecting.app */, + ); + name = Products; + sourceTree = ""; + }; + 556DED2620652EA700094928 /* SampleCollecting */ = { + isa = PBXGroup; + children = ( + 5508E2E12065873A00EBEB83 /* Storage.swift */, + 5508E2DF206567DA00EBEB83 /* Session.swift */, + 556DED2920652EA700094928 /* ViewController.swift */, + 552FFBA52072003F00B0E10E /* Sources */, + 55319E3F206593A600B866DD /* Supporting */, + ); + path = SampleCollecting; + sourceTree = ""; + }; +/* End PBXGroup section */ + +/* Begin PBXNativeTarget section */ + 556DED2320652EA700094928 /* SampleCollecting */ = { + isa = PBXNativeTarget; + buildConfigurationList = 556DED3620652EA700094928 /* Build configuration list for PBXNativeTarget "SampleCollecting" */; + buildPhases = ( + 556DED2020652EA700094928 /* Sources */, + 556DED2120652EA700094928 /* Frameworks */, + 556DED2220652EA700094928 /* Resources */, + ); + buildRules = ( + ); + dependencies = ( + ); + name = SampleCollecting; + productName = TestDrawing; + productReference = 556DED2420652EA700094928 /* SampleCollecting.app */; + productType = "com.apple.product-type.application"; + }; +/* End PBXNativeTarget section */ + +/* Begin PBXProject section */ + 556DED1C20652EA700094928 /* Project object */ = { + isa = PBXProject; + attributes = { + LastSwiftUpdateCheck = 0920; + LastUpgradeCheck = 0930; + ORGANIZATIONNAME = "Alexis Aubry"; + TargetAttributes = { + 556DED2320652EA700094928 = { + CreatedOnToolsVersion = 9.2; + ProvisioningStyle = Automatic; + }; + }; + }; + buildConfigurationList = 556DED1F20652EA700094928 /* Build configuration list for PBXProject "SampleCollecting" */; + compatibilityVersion = "Xcode 8.0"; + developmentRegion = en; + hasScannedForEncodings = 0; + knownRegions = ( + en, + Base, + ); + mainGroup = 556DED1B20652EA700094928; + productRefGroup = 556DED2520652EA700094928 /* Products */; + projectDirPath = ""; + projectRoot = ""; + targets = ( + 556DED2320652EA700094928 /* SampleCollecting */, + ); + }; +/* End PBXProject section */ + +/* Begin PBXResourcesBuildPhase section */ + 556DED2220652EA700094928 /* Resources */ = { + isa = PBXResourcesBuildPhase; + buildActionMask = 2147483647; + files = ( + 556DED3220652EA700094928 /* LaunchScreen.storyboard in Resources */, + 556DED2F20652EA700094928 /* Assets.xcassets in Resources */, + 556DED2D20652EA700094928 /* Main.storyboard in Resources */, + ); + runOnlyForDeploymentPostprocessing = 0; + }; +/* End PBXResourcesBuildPhase section */ + +/* Begin PBXSourcesBuildPhase section */ + 556DED2020652EA700094928 /* Sources */ = { + isa = PBXSourcesBuildPhase; + buildActionMask = 2147483647; + files = ( + 556DED2A20652EA700094928 /* ViewController.swift in Sources */, + 5508E2E22065873A00EBEB83 /* Storage.swift in Sources */, + 552FFBC22072003F00B0E10E /* EmojiSketches.swift in Sources */, + 552FFBC02072003F00B0E10E /* Classes.swift in Sources */, + 552FFBC32072003F00B0E10E /* PredictionSession.swift in Sources */, + 552FFBBA2072003F00B0E10E /* Paper.swift in Sources */, + 552FFBC12072003F00B0E10E /* PredictionSessionDelegate.swift in Sources */, + 55319E41206629AC00B866DD /* PaperContainerView.swift in Sources */, + 5508E2E0206567DA00EBEB83 /* Session.swift in Sources */, + 556DED2820652EA700094928 /* AppDelegate.swift in Sources */, + 552FFBBB2072003F00B0E10E /* BitmapPaperExporter.swift in Sources */, + 552FFBBC2072003F00B0E10E /* PaperExporter.swift in Sources */, + 552FFBBD2072003F00B0E10E /* CVPixelBufferExporter.swift in Sources */, + ); + runOnlyForDeploymentPostprocessing = 0; + }; +/* End PBXSourcesBuildPhase section */ + +/* Begin PBXVariantGroup section */ + 556DED2B20652EA700094928 /* Main.storyboard */ = { + isa = PBXVariantGroup; + children = ( + 556DED2C20652EA700094928 /* Base */, + ); + name = Main.storyboard; + sourceTree = ""; + }; + 556DED3020652EA700094928 /* LaunchScreen.storyboard */ = { + isa = PBXVariantGroup; + children = ( + 556DED3120652EA700094928 /* Base */, + ); + name = LaunchScreen.storyboard; + sourceTree = ""; + }; +/* End PBXVariantGroup section */ + +/* Begin XCBuildConfiguration section */ + 556DED3420652EA700094928 /* Debug */ = { + isa = XCBuildConfiguration; + buildSettings = { + ALWAYS_SEARCH_USER_PATHS = NO; + CLANG_ANALYZER_NONNULL = YES; + CLANG_ANALYZER_NUMBER_OBJECT_CONVERSION = YES_AGGRESSIVE; + CLANG_CXX_LANGUAGE_STANDARD = "gnu++14"; + CLANG_CXX_LIBRARY = "libc++"; + CLANG_ENABLE_MODULES = YES; + CLANG_ENABLE_OBJC_ARC = YES; + CLANG_WARN_BLOCK_CAPTURE_AUTORELEASING = YES; + CLANG_WARN_BOOL_CONVERSION = YES; + CLANG_WARN_COMMA = YES; + CLANG_WARN_CONSTANT_CONVERSION = YES; + CLANG_WARN_DEPRECATED_OBJC_IMPLEMENTATIONS = YES; + CLANG_WARN_DIRECT_OBJC_ISA_USAGE = YES_ERROR; + CLANG_WARN_DOCUMENTATION_COMMENTS = YES; + CLANG_WARN_EMPTY_BODY = YES; + CLANG_WARN_ENUM_CONVERSION = YES; + CLANG_WARN_INFINITE_RECURSION = YES; + CLANG_WARN_INT_CONVERSION = YES; + CLANG_WARN_NON_LITERAL_NULL_CONVERSION = YES; + CLANG_WARN_OBJC_IMPLICIT_RETAIN_SELF = YES; + CLANG_WARN_OBJC_LITERAL_CONVERSION = YES; + CLANG_WARN_OBJC_ROOT_CLASS = YES_ERROR; + CLANG_WARN_RANGE_LOOP_ANALYSIS = YES; + CLANG_WARN_STRICT_PROTOTYPES = YES; + CLANG_WARN_SUSPICIOUS_MOVE = YES; + CLANG_WARN_UNGUARDED_AVAILABILITY = YES_AGGRESSIVE; + CLANG_WARN_UNREACHABLE_CODE = YES; + CLANG_WARN__DUPLICATE_METHOD_MATCH = YES; + CODE_SIGN_IDENTITY = "iPhone Developer"; + COPY_PHASE_STRIP = NO; + DEBUG_INFORMATION_FORMAT = dwarf; + ENABLE_STRICT_OBJC_MSGSEND = YES; + ENABLE_TESTABILITY = YES; + GCC_C_LANGUAGE_STANDARD = gnu11; + GCC_DYNAMIC_NO_PIC = NO; + GCC_NO_COMMON_BLOCKS = YES; + GCC_OPTIMIZATION_LEVEL = 0; + GCC_PREPROCESSOR_DEFINITIONS = ( + "DEBUG=1", + "$(inherited)", + ); + GCC_WARN_64_TO_32_BIT_CONVERSION = YES; + GCC_WARN_ABOUT_RETURN_TYPE = YES_ERROR; + GCC_WARN_UNDECLARED_SELECTOR = YES; + GCC_WARN_UNINITIALIZED_AUTOS = YES_AGGRESSIVE; + GCC_WARN_UNUSED_FUNCTION = YES; + GCC_WARN_UNUSED_VARIABLE = YES; + IPHONEOS_DEPLOYMENT_TARGET = 11.2; + MTL_ENABLE_DEBUG_INFO = YES; + ONLY_ACTIVE_ARCH = YES; + SDKROOT = iphoneos; + SWIFT_ACTIVE_COMPILATION_CONDITIONS = DEBUG; + SWIFT_OPTIMIZATION_LEVEL = "-Onone"; + }; + name = Debug; + }; + 556DED3520652EA700094928 /* Release */ = { + isa = XCBuildConfiguration; + buildSettings = { + ALWAYS_SEARCH_USER_PATHS = NO; + CLANG_ANALYZER_NONNULL = YES; + CLANG_ANALYZER_NUMBER_OBJECT_CONVERSION = YES_AGGRESSIVE; + CLANG_CXX_LANGUAGE_STANDARD = "gnu++14"; + CLANG_CXX_LIBRARY = "libc++"; + CLANG_ENABLE_MODULES = YES; + CLANG_ENABLE_OBJC_ARC = YES; + CLANG_WARN_BLOCK_CAPTURE_AUTORELEASING = YES; + CLANG_WARN_BOOL_CONVERSION = YES; + CLANG_WARN_COMMA = YES; + CLANG_WARN_CONSTANT_CONVERSION = YES; + CLANG_WARN_DEPRECATED_OBJC_IMPLEMENTATIONS = YES; + CLANG_WARN_DIRECT_OBJC_ISA_USAGE = YES_ERROR; + CLANG_WARN_DOCUMENTATION_COMMENTS = YES; + CLANG_WARN_EMPTY_BODY = YES; + CLANG_WARN_ENUM_CONVERSION = YES; + CLANG_WARN_INFINITE_RECURSION = YES; + CLANG_WARN_INT_CONVERSION = YES; + CLANG_WARN_NON_LITERAL_NULL_CONVERSION = YES; + CLANG_WARN_OBJC_IMPLICIT_RETAIN_SELF = YES; + CLANG_WARN_OBJC_LITERAL_CONVERSION = YES; + CLANG_WARN_OBJC_ROOT_CLASS = YES_ERROR; + CLANG_WARN_RANGE_LOOP_ANALYSIS = YES; + CLANG_WARN_STRICT_PROTOTYPES = YES; + CLANG_WARN_SUSPICIOUS_MOVE = YES; + CLANG_WARN_UNGUARDED_AVAILABILITY = YES_AGGRESSIVE; + CLANG_WARN_UNREACHABLE_CODE = YES; + CLANG_WARN__DUPLICATE_METHOD_MATCH = YES; + CODE_SIGN_IDENTITY = "iPhone Developer"; + COPY_PHASE_STRIP = NO; + DEBUG_INFORMATION_FORMAT = "dwarf-with-dsym"; + ENABLE_NS_ASSERTIONS = NO; + ENABLE_STRICT_OBJC_MSGSEND = YES; + GCC_C_LANGUAGE_STANDARD = gnu11; + GCC_NO_COMMON_BLOCKS = YES; + GCC_WARN_64_TO_32_BIT_CONVERSION = YES; + GCC_WARN_ABOUT_RETURN_TYPE = YES_ERROR; + GCC_WARN_UNDECLARED_SELECTOR = YES; + GCC_WARN_UNINITIALIZED_AUTOS = YES_AGGRESSIVE; + GCC_WARN_UNUSED_FUNCTION = YES; + GCC_WARN_UNUSED_VARIABLE = YES; + IPHONEOS_DEPLOYMENT_TARGET = 11.2; + MTL_ENABLE_DEBUG_INFO = NO; + SDKROOT = iphoneos; + SWIFT_OPTIMIZATION_LEVEL = "-Owholemodule"; + VALIDATE_PRODUCT = YES; + }; + name = Release; + }; + 556DED3720652EA700094928 /* Debug */ = { + isa = XCBuildConfiguration; + buildSettings = { + ASSETCATALOG_COMPILER_APPICON_NAME = AppIcon; + CODE_SIGN_STYLE = Automatic; + DEVELOPMENT_TEAM = 75E8K39Z38; + INFOPLIST_FILE = "$(SRCROOT)/SampleCollecting/Info.plist"; + LD_RUNPATH_SEARCH_PATHS = "$(inherited) @executable_path/Frameworks"; + PRODUCT_BUNDLE_IDENTIFIER = fr.alexaubry.wwdc18.mlmoji.SampleCollecting; + PRODUCT_NAME = "$(TARGET_NAME)"; + SWIFT_VERSION = 4.0; + TARGETED_DEVICE_FAMILY = "1,2"; + }; + name = Debug; + }; + 556DED3820652EA700094928 /* Release */ = { + isa = XCBuildConfiguration; + buildSettings = { + ASSETCATALOG_COMPILER_APPICON_NAME = AppIcon; + CODE_SIGN_STYLE = Automatic; + DEVELOPMENT_TEAM = 75E8K39Z38; + INFOPLIST_FILE = "$(SRCROOT)/SampleCollecting/Info.plist"; + LD_RUNPATH_SEARCH_PATHS = "$(inherited) @executable_path/Frameworks"; + PRODUCT_BUNDLE_IDENTIFIER = fr.alexaubry.wwdc18.mlmoji.SampleCollecting; + PRODUCT_NAME = "$(TARGET_NAME)"; + SWIFT_VERSION = 4.0; + TARGETED_DEVICE_FAMILY = "1,2"; + }; + name = Release; + }; +/* End XCBuildConfiguration section */ + +/* Begin XCConfigurationList section */ + 556DED1F20652EA700094928 /* Build configuration list for PBXProject "SampleCollecting" */ = { + isa = XCConfigurationList; + buildConfigurations = ( + 556DED3420652EA700094928 /* Debug */, + 556DED3520652EA700094928 /* Release */, + ); + defaultConfigurationIsVisible = 0; + defaultConfigurationName = Release; + }; + 556DED3620652EA700094928 /* Build configuration list for PBXNativeTarget "SampleCollecting" */ = { + isa = XCConfigurationList; + buildConfigurations = ( + 556DED3720652EA700094928 /* Debug */, + 556DED3820652EA700094928 /* Release */, + ); + defaultConfigurationIsVisible = 0; + defaultConfigurationName = Release; + }; +/* End XCConfigurationList section */ + }; + rootObject = 556DED1C20652EA700094928 /* Project object */; +} diff --git a/Data-Model/Utilities/SampleCollecting/SampleCollecting.xcodeproj/project.xcworkspace/contents.xcworkspacedata b/Data-Model/Utilities/SampleCollecting/SampleCollecting.xcodeproj/project.xcworkspace/contents.xcworkspacedata new file mode 100644 index 0000000..34ee7c3 --- /dev/null +++ b/Data-Model/Utilities/SampleCollecting/SampleCollecting.xcodeproj/project.xcworkspace/contents.xcworkspacedata @@ -0,0 +1,7 @@ + + + + + diff --git a/Data-Model/Utilities/SampleCollecting/SampleCollecting.xcodeproj/project.xcworkspace/xcshareddata/IDEWorkspaceChecks.plist b/Data-Model/Utilities/SampleCollecting/SampleCollecting.xcodeproj/project.xcworkspace/xcshareddata/IDEWorkspaceChecks.plist new file mode 100644 index 0000000..18d9810 --- /dev/null +++ b/Data-Model/Utilities/SampleCollecting/SampleCollecting.xcodeproj/project.xcworkspace/xcshareddata/IDEWorkspaceChecks.plist @@ -0,0 +1,8 @@ + + + + + IDEDidComputeMac32BitWarning + + + diff --git a/Data-Model/Utilities/SampleCollecting/SampleCollecting/AppDelegate.swift b/Data-Model/Utilities/SampleCollecting/SampleCollecting/AppDelegate.swift new file mode 100644 index 0000000..0adfcc1 --- /dev/null +++ b/Data-Model/Utilities/SampleCollecting/SampleCollecting/AppDelegate.swift @@ -0,0 +1,13 @@ +// +// MLMOJI +// +// This file is part of Alexis Aubry's WWDC18 scolarship submission open source project. +// Copyright © 2018 Alexis Aubry. Available under the terms of the MIT License. +// + +import UIKit + +@UIApplicationMain +class AppDelegate: UIResponder, UIApplicationDelegate { + var window: UIWindow? +} diff --git a/Data-Model/Utilities/SampleCollecting/SampleCollecting/Assets.xcassets/AppIcon.appiconset/Contents.json b/Data-Model/Utilities/SampleCollecting/SampleCollecting/Assets.xcassets/AppIcon.appiconset/Contents.json new file mode 100644 index 0000000..d8db8d6 --- /dev/null +++ b/Data-Model/Utilities/SampleCollecting/SampleCollecting/Assets.xcassets/AppIcon.appiconset/Contents.json @@ -0,0 +1,98 @@ +{ + "images" : [ + { + "idiom" : "iphone", + "size" : "20x20", + "scale" : "2x" + }, + { + "idiom" : "iphone", + "size" : "20x20", + "scale" : "3x" + }, + { + "idiom" : "iphone", + "size" : "29x29", + "scale" : "2x" + }, + { + "idiom" : "iphone", + "size" : "29x29", + "scale" : "3x" + }, + { + "idiom" : "iphone", + "size" : "40x40", + "scale" : "2x" + }, + { + "idiom" : "iphone", + "size" : "40x40", + "scale" : "3x" + }, + { + "idiom" : "iphone", + "size" : "60x60", + "scale" : "2x" + }, + { + "idiom" : "iphone", + "size" : "60x60", + "scale" : "3x" + }, + { + "idiom" : "ipad", + "size" : "20x20", + "scale" : "1x" + }, + { + "idiom" : "ipad", + "size" : "20x20", + "scale" : "2x" + }, + { + "idiom" : "ipad", + "size" : "29x29", + "scale" : "1x" + }, + { + "idiom" : "ipad", + "size" : "29x29", + "scale" : "2x" + }, + { + "idiom" : "ipad", + "size" : "40x40", + "scale" : "1x" + }, + { + "idiom" : "ipad", + "size" : "40x40", + "scale" : "2x" + }, + { + "idiom" : "ipad", + "size" : "76x76", + "scale" : "1x" + }, + { + "idiom" : "ipad", + "size" : "76x76", + "scale" : "2x" + }, + { + "idiom" : "ipad", + "size" : "83.5x83.5", + "scale" : "2x" + }, + { + "idiom" : "ios-marketing", + "size" : "1024x1024", + "scale" : "1x" + } + ], + "info" : { + "version" : 1, + "author" : "xcode" + } +} \ No newline at end of file diff --git a/Data-Model/Utilities/SampleCollecting/SampleCollecting/Base.lproj/LaunchScreen.storyboard b/Data-Model/Utilities/SampleCollecting/SampleCollecting/Base.lproj/LaunchScreen.storyboard new file mode 100644 index 0000000..f83f6fd --- /dev/null +++ b/Data-Model/Utilities/SampleCollecting/SampleCollecting/Base.lproj/LaunchScreen.storyboard @@ -0,0 +1,25 @@ + + + + + + + + + + + + + + + + + + + + + + + + + diff --git a/Data-Model/Utilities/SampleCollecting/SampleCollecting/Base.lproj/Main.storyboard b/Data-Model/Utilities/SampleCollecting/SampleCollecting/Base.lproj/Main.storyboard new file mode 100644 index 0000000..03c13c2 --- /dev/null +++ b/Data-Model/Utilities/SampleCollecting/SampleCollecting/Base.lproj/Main.storyboard @@ -0,0 +1,24 @@ + + + + + + + + + + + + + + + + + + + + + + + + diff --git a/Data-Model/Utilities/SampleCollecting/SampleCollecting/Canvas.swift b/Data-Model/Utilities/SampleCollecting/SampleCollecting/Canvas.swift new file mode 100644 index 0000000..01281c2 --- /dev/null +++ b/Data-Model/Utilities/SampleCollecting/SampleCollecting/Canvas.swift @@ -0,0 +1,9 @@ +// +// Canvas.swift +// TestDrawing +// +// Created by Alexis AUBRY on 23/03/2018. +// Copyright © 2018 Alexis Aubry. All rights reserved. +// + +import Foundation diff --git a/Data-Model/Utilities/SampleCollecting/SampleCollecting/Info.plist b/Data-Model/Utilities/SampleCollecting/SampleCollecting/Info.plist new file mode 100644 index 0000000..32374b8 --- /dev/null +++ b/Data-Model/Utilities/SampleCollecting/SampleCollecting/Info.plist @@ -0,0 +1,49 @@ + + + + + CFBundleDevelopmentRegion + $(DEVELOPMENT_LANGUAGE) + CFBundleDisplayName + SampleCollecting + CFBundleExecutable + $(EXECUTABLE_NAME) + CFBundleIdentifier + $(PRODUCT_BUNDLE_IDENTIFIER) + CFBundleInfoDictionaryVersion + 6.0 + CFBundleName + $(PRODUCT_NAME) + CFBundlePackageType + APPL + CFBundleShortVersionString + 1.0 + CFBundleVersion + 1 + LSRequiresIPhoneOS + + UIFileSharingEnabled + + UILaunchStoryboardName + LaunchScreen + UIMainStoryboardFile + Main + UIRequiredDeviceCapabilities + + armv7 + + UISupportedInterfaceOrientations + + UIInterfaceOrientationPortrait + UIInterfaceOrientationLandscapeLeft + UIInterfaceOrientationLandscapeRight + + UISupportedInterfaceOrientations~ipad + + UIInterfaceOrientationPortrait + UIInterfaceOrientationPortraitUpsideDown + UIInterfaceOrientationLandscapeLeft + UIInterfaceOrientationLandscapeRight + + + diff --git a/Data-Model/Utilities/SampleCollecting/SampleCollecting/PaperContainerView.swift b/Data-Model/Utilities/SampleCollecting/SampleCollecting/PaperContainerView.swift new file mode 100644 index 0000000..9a21a3b --- /dev/null +++ b/Data-Model/Utilities/SampleCollecting/SampleCollecting/PaperContainerView.swift @@ -0,0 +1,44 @@ +// +// MLMOJI +// +// This file is part of Alexis Aubry's WWDC18 scolarship submission open source project. +// Copyright © 2018 Alexis Aubry. Available under the terms of the MIT License. +// + +import UIKit + +class PaperContainerView: UIView { + + let paper: Paper + + init(paper: Paper) { + self.paper = paper + super.init(frame: .zero) + createConstraints() + } + + required init?(coder aDecoder: NSCoder) { + self.paper = Paper() + super.init(coder: aDecoder) + createConstraints() + } + + private func createConstraints() { + addSubview(paper) + paper.translatesAutoresizingMaskIntoConstraints = false + paper.centerXAnchor.constraint(equalTo: centerXAnchor).isActive = true + paper.centerYAnchor.constraint(equalTo: centerYAnchor).isActive = true + paper.widthAnchor.constraint(equalToConstant: 300).isActive = true + paper.heightAnchor.constraint(equalToConstant: 300).isActive = true + + setContentCompressionResistancePriority(.defaultHigh, for: .horizontal) + setContentCompressionResistancePriority(.defaultHigh, for: .vertical) + setContentHuggingPriority(.defaultLow, for: .horizontal) + setContentHuggingPriority(.defaultLow, for: .vertical) + } + + override var intrinsicContentSize: CGSize { + return CGSize(width: 300, height: 300) + } + +} diff --git a/Data-Model/Utilities/SampleCollecting/SampleCollecting/Session.swift b/Data-Model/Utilities/SampleCollecting/SampleCollecting/Session.swift new file mode 100644 index 0000000..14d72ee --- /dev/null +++ b/Data-Model/Utilities/SampleCollecting/SampleCollecting/Session.swift @@ -0,0 +1,123 @@ +// +// MLMOJI +// +// This file is part of Alexis Aubry's WWDC18 scolarship submission open source project. +// Copyright © 2018 Alexis Aubry. Available under the terms of the MIT License. +// + +import Foundation +import UIKit + +/** + * The delegate of a session. + */ + +protocol SessionDelegate: class { + + /** + * The session did request a sample for the given label. Once you have the data, call + * `session.completeRequest(withID:,sample:)` to provide the sample data. + */ + + func sessionDidRequestSample(for label: Class, requestID: UUID) + + /** + * The session did add a contribution for a label (ig. the file was successfully saved). + */ + + func sessionDidAddContribution() + +} + +/** + * A sample data gathering session. + * + * Set the `delegate`, and call `start` to start a new session. + */ + +class Session { + + /// The delegate that receives informations about the session. + weak var delegate: SessionDelegate? + + /// The number of contributions for each label. + fileprivate(set) var contributions: [Class: Int] + + /// The desired size of ouput samples. + let sampleSize = CGSize(width: 300, height: 300) + + private let storage: Storage + private let labels: [Class] = Class.allLabels + private var activeRequest: (UUID, Class)? + + // MARK: - Initialization + + init() throws { + self.storage = try Storage() + self.contributions = storage.fetchContributions() + } + + // MARK: - Interacting with the session + + /** + * Starts the session. Make sure `delegate` is set before calling this method. + */ + + func start() { + + guard labels.count >= 1 else { + fatalError("No labels to classify.") + } + + guard activeRequest == nil else { + fatalError("Cannot start a request that is already active.") + } + + requestSampleForNextLabel() + + } + + /** + * Saves the provided sample for the given request. The sample data must be in the JPG format. + */ + + func completeRequest(withID requestID: UUID, sample: Data) throws { + + guard let activeRequest = self.activeRequest else { + fatalError("Cannot complete a request that never started...") + } + + guard activeRequest.0 == requestID else { + fatalError("Wrong request. Only one request can be active at a time.") + } + + guard storage.save(jpgImage: sample, for: activeRequest.1) == true else { + throw NSError(domain: "SessionErrorDomain", code: 1000, userInfo: [ + NSLocalizedDescriptionKey: "The file could not be saved on disk." + ]) + } + + let initialContriubutions = contributions[activeRequest.1] ?? 0 + contributions[activeRequest.1] = initialContriubutions + 1 + + delegate?.sessionDidAddContribution() + requestSampleForNextLabel() + + } + + private func requestSampleForNextLabel() { + + // Prioritize the labels with the lowest contributions + let lowestContributors = labels.sorted { + (self.contributions[$0]! < self.contributions[$1]!) + } + + let requestID = UUID() + let nextLabel = lowestContributors.first! + + self.activeRequest = (requestID, nextLabel) + delegate?.sessionDidRequestSample(for: nextLabel, requestID: requestID) + + } + +} diff --git a/Data-Model/Utilities/SampleCollecting/SampleCollecting/Storage.swift b/Data-Model/Utilities/SampleCollecting/SampleCollecting/Storage.swift new file mode 100644 index 0000000..9d7fec3 --- /dev/null +++ b/Data-Model/Utilities/SampleCollecting/SampleCollecting/Storage.swift @@ -0,0 +1,94 @@ +// +// MLMOJI +// +// This file is part of Alexis Aubry's WWDC18 scolarship submission open source project. +// Copyright © 2018 Alexis Aubry. Available under the terms of the MIT License. +// + +import Foundation + +/** + * Manages the storage of the generated training data. + */ + +class Storage { + + /// The URL of the output data set. + let dataSetURL: URL + + init() throws { + + let documentsDirectory = NSSearchPathForDirectoriesInDomains(.documentDirectory, .userDomainMask, true)[0] + let documentsDirectoryURL = URL(fileURLWithPath: documentsDirectory) + + self.dataSetURL = documentsDirectoryURL.appendingPathComponent("training-data") + + for label in Class.allLabels { + + let labelURL = dataSetURL.appendingPathComponent(label.rawValue) + var isDirectory: ObjCBool = false + + if FileManager.default.fileExists(atPath: labelURL.path, isDirectory: &isDirectory) { + + if isDirectory.boolValue == false { + try FileManager.default.removeItem(at: labelURL) + try FileManager.default.createDirectory(at: labelURL, withIntermediateDirectories: true, attributes: nil) + } + + } else { + try FileManager.default.createDirectory(at: labelURL, withIntermediateDirectories: true, attributes: nil) + } + + } + + // Clean up old images + + if let contents = try? FileManager.default.contentsOfDirectory(atPath: dataSetURL.path) { + + for item in contents { + + if Class(rawValue: item) == nil { + let url = dataSetURL.appendingPathComponent(item) + try FileManager.default.removeItem(at: url) + } + + } + + } + + } + + /** + * Saves the JPG image data for the specified label. Returns `true` if the operation was successful. + */ + + func save(jpgImage: Data, for label: Class) -> Bool { + + let itemURL = dataSetURL + .appendingPathComponent(label.rawValue) + .appendingPathComponent(UUID().uuidString) + .appendingPathExtension("jpg") + + return FileManager.default.createFile(atPath: itemURL.path, contents: jpgImage, attributes: nil) + + } + + /** + * The number of samples saved for every label. + */ + + func fetchContributions() -> [Class: Int] { + + var result: [Class: Int] = [:] + + for label in Class.allLabels { + let labelURL = dataSetURL.appendingPathComponent(label.rawValue) + let numberOfItems = (try? FileManager.default.contentsOfDirectory(atPath: labelURL.path))?.count ?? 0 + result[label] = numberOfItems + } + + return result + + } + +} diff --git a/Data-Model/Utilities/SampleCollecting/SampleCollecting/ViewController.swift b/Data-Model/Utilities/SampleCollecting/SampleCollecting/ViewController.swift new file mode 100644 index 0000000..da9b209 --- /dev/null +++ b/Data-Model/Utilities/SampleCollecting/SampleCollecting/ViewController.swift @@ -0,0 +1,246 @@ +// +// MLMOJI +// +// This file is part of Alexis Aubry's WWDC18 scolarship submission open source project. +// Copyright © 2018 Alexis Aubry. Available under the terms of the MIT License. +// + +import UIKit +import MobileCoreServices + +/** + * The view controller that records the samples for data labels. + */ + +class ViewController: UIViewController { + + /** + * Possible states for sample collection. + */ + + enum State { + case idle + case pendingUserInput(UUID) + case pendingUserConfirmation(UUID) + case pendingNewRequest + } + + /// The active session. + let session = try! Session() + + /// The state of sample collection. + private var state: State = .idle { + didSet { + refreshInterfaceForUpdatedState() + } + } + + + // MARK: - Views + + private let currentTitleLabel = UILabel() + private let paper = Paper() + private let submitButton = UIButton() + private let stackView = UIStackView() + private let contributionsLabel = UILabel() + + + // MARK: - UI Setup + + override func viewDidLoad() { + super.viewDidLoad() + createViews() + createContraints() + view.backgroundColor = UIColor.groupTableViewBackground + + session.delegate = self + session.start() + } + + /** + * Creates the views to display on the data collection screen. + */ + + private func createViews() { + + currentTitleLabel.adjustsFontSizeToFitWidth = true + currentTitleLabel.textAlignment = .center + currentTitleLabel.font = UIFont.systemFont(ofSize: 100) + + contributionsLabel.numberOfLines = 0 + contributionsLabel.textAlignment = .center + contributionsLabel.font = UIFont.systemFont(ofSize: 24) + + paper.backgroundColor = .white + paper.isUserInteractionEnabled = false + paper.delegate = self + + let paperContainer = PaperContainerView(paper: paper) + + submitButton.setTitleColor(.white, for: .normal) + submitButton.setTitle("Save", for: .normal) + submitButton.titleLabel?.font = UIFont.systemFont(ofSize: 17, weight: .semibold) + + let buttonBackgroundColor = UIGraphicsImageRenderer(size: CGSize(width: 1, height: 1)).image { context in + UIColor(red: 0, green: 122/255, blue: 1, alpha: 1).setFill() + context.fill(CGRect(x: 0, y: 0, width: 1, height: 1)) + } + + submitButton.setBackgroundImage(buttonBackgroundColor, for: .normal) + submitButton.setContentHuggingPriority(.defaultLow, for: .horizontal) + submitButton.layer.cornerRadius = 12 + submitButton.clipsToBounds = true + + submitButton.addTarget(self, action: #selector(submitSketch), for: .touchUpInside) + + stackView.alignment = .fill + stackView.axis = .vertical + stackView.addArrangedSubview(currentTitleLabel) + stackView.addArrangedSubview(paperContainer) + stackView.addArrangedSubview(submitButton) + + view.addSubview(stackView) + view.addSubview(contributionsLabel) + + refreshContributionsLabel() + configureViews(isRegular: traitCollection.horizontalSizeClass == .regular) + refreshInterfaceForUpdatedState() + + } + + /** + * Creates the Auto Layout constraints for the view. + */ + + private func createContraints() { + stackView.translatesAutoresizingMaskIntoConstraints = false + stackView.centerXAnchor.constraint(equalTo: view.centerXAnchor).isActive = true + stackView.centerYAnchor.constraint(equalTo: view.centerYAnchor).isActive = true + + contributionsLabel.translatesAutoresizingMaskIntoConstraints = false + contributionsLabel.bottomAnchor.constraint(equalTo: view.safeAreaLayoutGuide.bottomAnchor, constant: -16).isActive = true + contributionsLabel.leadingAnchor.constraint(equalTo: view.readableContentGuide.leadingAnchor, constant: 16).isActive = true + contributionsLabel.trailingAnchor.constraint(equalTo: view.readableContentGuide.trailingAnchor, constant: -16).isActive = true + + submitButton.heightAnchor.constraint(equalToConstant: 55).isActive = true + } + + /** + * Responds to size class changes. + */ + + override func traitCollectionDidChange(_ previousTraitCollection: UITraitCollection?) { + super.traitCollectionDidChange(previousTraitCollection) + configureViews(isRegular: traitCollection.horizontalSizeClass == .regular) + } + + /** + * Configures the view for adaptive presentation. + */ + + private func configureViews(isRegular: Bool) { + stackView.spacing = isRegular ? 64 : 32 + } + + // MARK: - Session + + /** + * Refreshes the number of contributions for each trained label. + */ + + private func refreshContributionsLabel() { + + let labels = session.contributions.map { label, count in + return "\(label.emojiValue)\u{a0}\(count)" + } + + contributionsLabel.text = labels.joined(separator: "\u{a0}\u{a0}• ") + + } + + /** + * Refreshes the UI when the state changes. + */ + + private func refreshInterfaceForUpdatedState() { + + switch state { + case .idle: + stackView.isHidden = true + case .pendingUserInput: + submitButton.isEnabled = false + paper.isUserInteractionEnabled = true + stackView.isHidden = false + stackView.alpha = 1 + case .pendingUserConfirmation: + submitButton.isEnabled = true + case .pendingNewRequest: + stackView.alpha = 0.5 + } + + } + + /** + * Attempts to save the skecth from the paper into the session. + */ + + @objc private func submitSketch() { + + guard case let .pendingUserConfirmation(requestID) = state else { + return + } + + state = .pendingNewRequest + let exporter = BitmapPaperExporter(size: session.sampleSize, scale: 1, fileType: kUTTypeJPEG) + + do { + + let jpgSample = try paper.export(with: exporter) + try session.completeRequest(withID: requestID, sample: jpgSample) + paper.clear() + + } catch { + + let alert = UIAlertController(title: "Could not save sample", + message: (error as NSError).localizedDescription, + preferredStyle: .alert) + + let cancelAction = UIAlertAction(title: "Cancel", style: .cancel, handler: nil) + alert.addAction(cancelAction) + + present(alert, animated: true) { + self.paper.clear() + } + + } + + } + +} + +// MARK: - SessionDelegate + PaperDelegate + +extension ViewController: SessionDelegate, PaperDelegate { + + func sessionDidRequestSample(for label: Class, requestID: UUID) { + currentTitleLabel.text = label.emojiValue + state = .pendingUserInput(requestID) + } + + func sessionDidAddContribution() { + refreshContributionsLabel() + } + + func paperDidStartDrawingStroke(_ paper: Paper) { + // no-op + } + + func paperDidFinishDrawingStroke(_ paper: Paper) { + guard case let .pendingUserInput(requestID) = state else { + return + } + + state = .pendingUserConfirmation(requestID) + } + +} diff --git a/LICENSE b/LICENSE new file mode 100644 index 0000000..98df3ab --- /dev/null +++ b/LICENSE @@ -0,0 +1,21 @@ +MIT License + +Copyright (c) 2018 Alexis Aubry + +Permission is hereby granted, free of charge, to any person obtaining a copy +of this software and associated documentation files (the "Software"), to deal +in the Software without restriction, including without limitation the rights +to use, copy, modify, merge, publish, distribute, sublicense, and/or sell +copies of the Software, and to permit persons to whom the Software is +furnished to do so, subject to the following conditions: + +The above copyright notice and this permission notice shall be included in all +copies or substantial portions of the Software. + +THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE +SOFTWARE. diff --git a/Playground/MLMOJI.playgroundbook/Contents/Chapters/Chapter1.playgroundchapter/Manifest.plist b/Playground/MLMOJI.playgroundbook/Contents/Chapters/Chapter1.playgroundchapter/Manifest.plist new file mode 100644 index 0000000..22725ce --- /dev/null +++ b/Playground/MLMOJI.playgroundbook/Contents/Chapters/Chapter1.playgroundchapter/Manifest.plist @@ -0,0 +1,14 @@ + + + + + Name + MLMOJI + Pages + + Page1.playgroundpage + Page2.playgroundpage + Page3.playgroundpage + + + diff --git a/Playground/MLMOJI.playgroundbook/Contents/Chapters/Chapter1.playgroundchapter/Pages/Page1.playgroundpage/Contents.swift b/Playground/MLMOJI.playgroundbook/Contents/Chapters/Chapter1.playgroundchapter/Pages/Page1.playgroundpage/Contents.swift new file mode 100644 index 0000000..e954ad2 --- /dev/null +++ b/Playground/MLMOJI.playgroundbook/Contents/Chapters/Chapter1.playgroundchapter/Pages/Page1.playgroundpage/Contents.swift @@ -0,0 +1,49 @@ +//#-hidden-code +import PlaygroundSupport + +var startedPredictions = false +PlaygroundPage.current.needsIndefiniteExecution = true + +/// Starts the predictive keyboard. +func startPredictions() { + PlaygroundPage.current.proxy?.sendEvent(.startPredictions) + startedPredictions = true +} +//#-end-hidden-code +/*: + # MLMOJI : Hand-Drawn Emoji Recognition + + --- + + With more than 2000 emoji supported on iOS, it can be difficult to find the character you're after in the system keyboard. What if there were a keyboard that converts drawings to the corresponding emoji? ✨ + + This is what this playground is all about! Using **Core Graphics** and **Core ML**, you'll be playing with a + [deep neural network](glossary://dnn), and explore a more fun and intuitive way to type. 🤖 + + The machine learning model was trained to recognize 7 hand-drawn objects: + + ![SupportedEmoji](SupportedEmoji.png) + + ## Getting started + + To launch the predictive keyboard, enter `startPredictions()` in the code editor below and run your code. + + Sketch your emoji in the white area on the right side of the screen. The keyboard + will predict the most likely matches as you keep drawing. Here are some examples, for inspiration: + + ![EmojiStrokes](EmojiStrokes.png) + + After this, if you want to learn how the Core ML model was prepared, you can read the [next page](@next). + + */ +//#-code-completion(everything, hide) +//#-code-completion(identifier, show, startPredictions()) +//#-editable-code Tap to write your code +//#-end-editable-code + +//#-hidden-code +if !startedPredictions { + PlaygroundPage.current.assessmentStatus = .fail(hints: ["You did not start the predictive keyboard."], + solution: "Enter `startPredictions()` in the code editor on the left and run your code again") +} +//#-end-hidden-code diff --git a/Playground/MLMOJI.playgroundbook/Contents/Chapters/Chapter1.playgroundchapter/Pages/Page1.playgroundpage/LiveView.swift b/Playground/MLMOJI.playgroundbook/Contents/Chapters/Chapter1.playgroundchapter/Pages/Page1.playgroundpage/LiveView.swift new file mode 100644 index 0000000..0cd4352 --- /dev/null +++ b/Playground/MLMOJI.playgroundbook/Contents/Chapters/Chapter1.playgroundchapter/Pages/Page1.playgroundpage/LiveView.swift @@ -0,0 +1,13 @@ +// +// MLMOJI +// +// This file is part of Alexis Aubry's WWDC18 scolarship submission open source project. +// Copyright © 2018 Alexis Aubry. Available under the terms of the MIT License. +// + +import UIKit +import PlaygroundSupport + +let vc = PlaygroundPredictionViewController() +PlaygroundPage.current.liveView = vc +PlaygroundPage.current.needsIndefiniteExecution = true diff --git a/Playground/MLMOJI.playgroundbook/Contents/Chapters/Chapter1.playgroundchapter/Pages/Page1.playgroundpage/Manifest.plist b/Playground/MLMOJI.playgroundbook/Contents/Chapters/Chapter1.playgroundchapter/Pages/Page1.playgroundpage/Manifest.plist new file mode 100644 index 0000000..8160c17 --- /dev/null +++ b/Playground/MLMOJI.playgroundbook/Contents/Chapters/Chapter1.playgroundchapter/Pages/Page1.playgroundpage/Manifest.plist @@ -0,0 +1,21 @@ + + + + + Name + Introducing MLMOJI + LiveViewEdgeToEdge + + LiveViewMode + VisibleByDefault + CodeCopySetup + + ReadyToCopyInstructions + Let's start this challenge! Import the code from the previous page to start. + NotReadyToCopyInstructions + Code 420: You need to complete the previous challenge before starting this one. + + PlaygroundLoggingMode + Off + + diff --git a/Playground/MLMOJI.playgroundbook/Contents/Chapters/Chapter1.playgroundchapter/Pages/Page1.playgroundpage/PrivateResources/EmojiStrokes@2x.png b/Playground/MLMOJI.playgroundbook/Contents/Chapters/Chapter1.playgroundchapter/Pages/Page1.playgroundpage/PrivateResources/EmojiStrokes@2x.png new file mode 100644 index 0000000..9846755 Binary files /dev/null and b/Playground/MLMOJI.playgroundbook/Contents/Chapters/Chapter1.playgroundchapter/Pages/Page1.playgroundpage/PrivateResources/EmojiStrokes@2x.png differ diff --git a/Playground/MLMOJI.playgroundbook/Contents/Chapters/Chapter1.playgroundchapter/Pages/Page1.playgroundpage/PrivateResources/SupportedEmoji@2x.png b/Playground/MLMOJI.playgroundbook/Contents/Chapters/Chapter1.playgroundchapter/Pages/Page1.playgroundpage/PrivateResources/SupportedEmoji@2x.png new file mode 100644 index 0000000..a3ef07f Binary files /dev/null and b/Playground/MLMOJI.playgroundbook/Contents/Chapters/Chapter1.playgroundchapter/Pages/Page1.playgroundpage/PrivateResources/SupportedEmoji@2x.png differ diff --git a/Playground/MLMOJI.playgroundbook/Contents/Chapters/Chapter1.playgroundchapter/Pages/Page1.playgroundpage/Sources/PredictionResultNode.swift b/Playground/MLMOJI.playgroundbook/Contents/Chapters/Chapter1.playgroundchapter/Pages/Page1.playgroundpage/Sources/PredictionResultNode.swift new file mode 100644 index 0000000..43defe1 --- /dev/null +++ b/Playground/MLMOJI.playgroundbook/Contents/Chapters/Chapter1.playgroundchapter/Pages/Page1.playgroundpage/Sources/PredictionResultNode.swift @@ -0,0 +1,131 @@ +// +// MLMOJI +// +// This file is part of Alexis Aubry's WWDC18 scolarship submission open source project. +// Copyright © 2018 Alexis Aubry. Available under the terms of the MIT License. +// + +import UIKit + +/** + * A view that displays the prediction result for a single emojis. + * + * It has two states: waiting for prediction and expanded. + */ + +class PredictionResultNode: UIView { + + /// The label that displays the emoji. + let emojiLabel = UILabel() + + /// The label that displays the likeliness percentage. + let percentageLabel = UILabel() + + /// The view that contains the percentage prediction. + let percentageContainerView = UIView() + + // MARK: - Initialization + + override init(frame: CGRect) { + super.init(frame: frame) + configureSubviews() + configureConstraints() + } + + required init?(coder aDecoder: NSCoder) { + super.init(coder: aDecoder) + configureSubviews() + configureConstraints() + } + + /** + * Configures the subviews appearance. + */ + + private func configureSubviews() { + + emojiLabel.font = UIFont.systemFont(ofSize: 40) + emojiLabel.adjustsFontSizeToFitWidth = true + emojiLabel.textAlignment = .center + emojiLabel.baselineAdjustment = .alignCenters + + percentageLabel.font = UIFont.systemFont(ofSize: 55, weight: .bold) + percentageLabel.textColor = .white + percentageLabel.textAlignment = .center + percentageLabel.adjustsFontSizeToFitWidth = true + percentageLabel.baselineAdjustment = .alignCenters + + percentageContainerView.layer.cornerRadius = 12 + percentageContainerView.addSubview(emojiLabel) + percentageContainerView.addSubview(percentageLabel) + + addSubview(percentageContainerView) + + } + + /** + * Creates the constraints for the given view. + */ + + private func configureConstraints() { + + emojiLabel.translatesAutoresizingMaskIntoConstraints = false + percentageLabel.translatesAutoresizingMaskIntoConstraints = false + percentageContainerView.translatesAutoresizingMaskIntoConstraints = false + + emojiLabel.leadingAnchor.constraint(equalTo: percentageContainerView.leadingAnchor, constant: 8).isActive = true + emojiLabel.topAnchor.constraint(equalTo: percentageContainerView.topAnchor, constant: 8).isActive = true + emojiLabel.bottomAnchor.constraint(equalTo: percentageContainerView.bottomAnchor, constant: -10).isActive = true + emojiLabel.setContentCompressionResistancePriority(.defaultHigh, for: .horizontal) + + percentageLabel.leadingAnchor.constraint(equalTo: emojiLabel.trailingAnchor, constant: 8).isActive = true + percentageLabel.trailingAnchor.constraint(equalTo: percentageContainerView.trailingAnchor, constant: -8).isActive = true + percentageLabel.centerYAnchor.constraint(equalTo: percentageContainerView.centerYAnchor).isActive = true + percentageLabel.heightAnchor.constraint(equalToConstant: 44).isActive = true + percentageLabel.setContentCompressionResistancePriority(.defaultLow, for: .horizontal) + + percentageContainerView.leadingAnchor.constraint(equalTo: leadingAnchor).isActive = true + percentageContainerView.widthAnchor.constraint(equalToConstant: 150).isActive = true + percentageContainerView.heightAnchor.constraint(equalToConstant: 55).isActive = true + + trailingAnchor.constraint(equalTo: percentageContainerView.trailingAnchor).isActive = true + bottomAnchor.constraint(equalTo: percentageContainerView.bottomAnchor).isActive = true + + } + + override var intrinsicContentSize: CGSize { + return CGSize(width: 150, height: 55) + } + + // MARK: - Changing the Data + + /** + * Displays the given prediction output. + */ + + func display(output: Double, for predictionClass: Class) { + + emojiLabel.text = predictionClass.emojiValue + percentageLabel.text = String(format: "%.2f", output * 100) + "%" + + if output >= 0.2 { + percentageLabel.font = UIFont.systemFont(ofSize: 55, weight: .bold) + percentageContainerView.backgroundColor = predictionClass.highlightColor + } else { + percentageLabel.font = UIFont.systemFont(ofSize: 22, weight: .light) + percentageContainerView.backgroundColor = predictionClass.highlightColor.withAlphaComponent(0.5) + } + + } + + /** + * Resets the result node. + */ + + func reset() { + emojiLabel.text = nil + percentageLabel.text = nil + percentageContainerView.backgroundColor = nil + } + +} diff --git a/Playground/MLMOJI.playgroundbook/Contents/Chapters/Chapter1.playgroundchapter/Pages/Page1.playgroundpage/Sources/PredictionResultsContainer.swift b/Playground/MLMOJI.playgroundbook/Contents/Chapters/Chapter1.playgroundchapter/Pages/Page1.playgroundpage/Sources/PredictionResultsContainer.swift new file mode 100644 index 0000000..ab2e1e0 --- /dev/null +++ b/Playground/MLMOJI.playgroundbook/Contents/Chapters/Chapter1.playgroundchapter/Pages/Page1.playgroundpage/Sources/PredictionResultsContainer.swift @@ -0,0 +1,521 @@ +// +// MLMOJI +// +// This file is part of Alexis Aubry's WWDC18 scolarship submission open source project. +// Copyright © 2018 Alexis Aubry. Available under the terms of the MIT License. +// + +import UIKit + +private let kPlaceholderFloatAnimation = "PredictionResultsContainer.PlaceholderFloat" +private let kStatusDraw = "🖌 Draw on the canvas below" +private let kStatusFailure = "😿 Could not guess the emoji" + +/** + * A view that displays the results of the prediction. + */ + +class PredictionResultsContainer: UIView { + + enum State { + case idle + case displayed + case startedPredictions + case waitingInitialPrediction + case predicted + case noPrediction + } + + /// The state of the results. + var state: State = .idle + + // MARK: - Properties + + /// The main content stack view. + let stackView = UIStackView() + + /// The lines displaying the placeholder result. + private var placeholderLineStacks: [UIStackView] = [] + + /// The view that contains the status placeholder. + private let statusPlaceholderContainer = UIView() + + /// The placeholder to display status messages. + private let statusPlaceholder = UILabel() + + /// The nodes that display the result of the prediction. + private let resultNodes: [PredictionResultNode] = { + return [PredictionResultNode(), PredictionResultNode(), PredictionResultNode()] + }() + + private var small_left_trailing: NSLayoutConstraint? + private var small_left_bottom: NSLayoutConstraint? + private var small_middle_leading: NSLayoutConstraint? + private var small_middle_bottom: NSLayoutConstraint? + private var small_right_top: NSLayoutConstraint? + private var small_right_centerX: NSLayoutConstraint? + + private var large_middle_centerX: NSLayoutConstraint? + private var large_middle_centerY: NSLayoutConstraint? + private var large_left_trailing: NSLayoutConstraint? + private var large_left_centerY: NSLayoutConstraint? + private var large_right_leading: NSLayoutConstraint? + private var large_right_centerY: NSLayoutConstraint? + + // MARK: - Initialization + + override init(frame: CGRect) { + super.init(frame: frame) + configureSubviews() + configureConstraints() + } + + required init?(coder aDecoder: NSCoder) { + super.init(coder: aDecoder) + configureSubviews() + configureConstraints() + } + + /** + * Configures the view's subviews. + */ + + private func configureSubviews() { + + // 1) Create the general stack + + stackView.axis = .vertical + stackView.spacing = 55 + stackView.alignment = .center + stackView.distribution = .fillEqually + + addSubview(stackView) + + // 2) Create the placeholder lines + + let allClasses = Class.allLabels + let (numberOfLines, remainingEmojis) = allClasses.count.quotientAndRemainder(dividingBy: 4) + + // 2.1 - Create the lines + + for lineIndex in 0 ..< numberOfLines { + + let startIndex = allClasses.startIndex + (lineIndex * 4) + let endIndex = startIndex + 4 + + let stack = makePlaceholderLineStack(content: allClasses, startIndex: startIndex, endIndex: endIndex) + placeholderLineStacks.append(stack) + stackView.addArrangedSubview(stack) + + } + + // 2.2 - Create a line for the remaining classes + + let startIndex = allClasses.startIndex + (numberOfLines * 4) + let endIndex = startIndex + remainingEmojis + + let stack = makePlaceholderLineStack(content: allClasses, startIndex: startIndex, endIndex: endIndex) + placeholderLineStacks.append(stack) + stackView.addArrangedSubview(stack) + + // 3) Create the result nodes + + for node in resultNodes { + addSubview(node) + } + + // 4) Create the status placeholder + + statusPlaceholder.font = UIFont.systemFont(ofSize: 30, weight: .medium) + statusPlaceholder.numberOfLines = 1 + statusPlaceholder.textColor = .black + + statusPlaceholderContainer.backgroundColor = .white + statusPlaceholderContainer.layer.cornerRadius = 12 + statusPlaceholderContainer.alpha = 0 + + statusPlaceholderContainer.addSubview(statusPlaceholder) + addSubview(statusPlaceholderContainer) + + } + + /** + * Creates a stack view that displays a line of elements. + */ + + private func makeLineStack() -> UIStackView { + + let stack = UIStackView() + stack.spacing = 55 + stack.distribution = .fillEqually + stack.alignment = .center + stack.axis = .horizontal + + return stack + + } + + /** + * Creates a placeholder line stack for the classes between the given indices. + */ + + private func makePlaceholderLineStack(content: [Class], startIndex: Int, endIndex: Int) -> UIStackView { + + let stack = makeLineStack() + + for classIndex in startIndex ..< endIndex { + + let predictionClass = content[classIndex] + + let floatingLabel = UILabel() + floatingLabel.font = UIFont.monospacedDigitSystemFont(ofSize: 69, weight: .regular) + floatingLabel.adjustsFontSizeToFitWidth = true + floatingLabel.text = predictionClass.emojiValue + + stack.addArrangedSubview(floatingLabel) + + } + + return stack + + } + + /** + * Configures the Auto Layout constraints + */ + + private func configureConstraints() { + + stackView.translatesAutoresizingMaskIntoConstraints = false + stackView.centerXAnchor.constraint(equalTo: centerXAnchor).isActive = true + stackView.centerYAnchor.constraint(equalTo: centerYAnchor).isActive = true + + statusPlaceholderContainer.translatesAutoresizingMaskIntoConstraints = false + statusPlaceholderContainer.centerXAnchor.constraint(equalTo: centerXAnchor).isActive = true + statusPlaceholderContainer.centerYAnchor.constraint(equalTo: centerYAnchor).isActive = true + + statusPlaceholder.translatesAutoresizingMaskIntoConstraints = false + statusPlaceholder.leadingAnchor.constraint(equalTo: statusPlaceholderContainer.leadingAnchor, constant: 10).isActive = true + statusPlaceholder.topAnchor.constraint(equalTo: statusPlaceholderContainer.topAnchor, constant: 10).isActive = true + statusPlaceholderContainer.trailingAnchor.constraint(equalTo: statusPlaceholder.trailingAnchor, constant: 15).isActive = true + statusPlaceholderContainer.bottomAnchor.constraint(equalTo: statusPlaceholder.bottomAnchor, constant: 10).isActive = true + + // Result nodes + + let leftNode = resultNodes[0] + let middleNode = resultNodes[1] + let rightNode = resultNodes[2] + + leftNode.translatesAutoresizingMaskIntoConstraints = false + middleNode.translatesAutoresizingMaskIntoConstraints = false + rightNode.translatesAutoresizingMaskIntoConstraints = false + + // Compact width + + small_left_trailing = leftNode.trailingAnchor.constraint(equalTo: centerXAnchor, constant: -25) + small_left_bottom = leftNode.bottomAnchor.constraint(equalTo: centerYAnchor, constant: -25) + small_middle_leading = middleNode.leadingAnchor.constraint(equalTo: centerXAnchor, constant: 25) + small_middle_bottom = middleNode.bottomAnchor.constraint(equalTo: centerYAnchor, constant: -25) + small_right_top = rightNode.topAnchor.constraint(equalTo: centerYAnchor, constant: 25) + small_right_centerX = rightNode.centerXAnchor.constraint(equalTo: centerXAnchor) + + large_middle_centerX = middleNode.centerXAnchor.constraint(equalTo: centerXAnchor) + large_middle_centerY = middleNode.centerYAnchor.constraint(equalTo: centerYAnchor) + large_left_trailing = leftNode.trailingAnchor.constraint(equalTo: middleNode.leadingAnchor, constant: -55) + large_left_centerY = leftNode.centerYAnchor.constraint(equalTo: centerYAnchor) + large_right_leading = rightNode.leadingAnchor.constraint(equalTo: middleNode.trailingAnchor, constant: 55) + large_right_centerY = rightNode.centerYAnchor.constraint(equalTo: centerYAnchor) + + // Large Width + + } + + // MARK: - Animations + + /** + * Adds the gravity floating animation to the placeholders. + */ + + private func floatPlaceholders() { + + for line in self.placeholderLineStacks.reversed() { + + for placeholder in line.arrangedSubviews { + let rotateAnimation = makeGravityAnimation(for: placeholder) + placeholder.layer.removeAnimation(forKey: kPlaceholderFloatAnimation) + placeholder.layer.add(rotateAnimation, forKey: kPlaceholderFloatAnimation) + } + + } + + } + + /** + * Creates an animation that makes the layer rotate around a random orbit. + */ + + private func makeGravityAnimation(for placeholder: UIView) -> CAKeyframeAnimation { + + // Define the distance between the center of the layer and its roation orbit + let radius: CGFloat = 7.5 + + // Calculate a random angle to define the direction of the orbit and convert it to radians + let angle = (CGFloat(arc4random_uniform(360 - 25 + 1) + 25) * CGFloat.pi) / 180 + + // Calculate a random rotation duration (between 4 and 6s) + let duration = Double(arc4random_uniform(6 - 4 + 1) + 4) + + // Calculate the offset between the center of the layer and the edge of the orbit (thanks, trig!) + let yOffset = sin(angle) * radius + let xOffset = cos(angle) * radius + + // Calculate the path of the orbit + + let center = placeholder.center + + let rotationPoint = CGPoint(x: center.x - xOffset, + y: center.y - yOffset) + + let minX = min(center.x, rotationPoint.x) + let minY = min(center.y, rotationPoint.y) + let maxX = max(center.x, rotationPoint.x) + let maxY = max(center.y, rotationPoint.y) + + let size = max(maxX - minX, maxY - minY) + + let ovalRect = CGRect(x: minX, y: minY, + width: size, height:size) + + let ovalPath = UIBezierPath(ovalIn: ovalRect) + + // Create and return the animation + + let animation = CAKeyframeAnimation(keyPath: "position") + animation.path = ovalPath.cgPath + animation.isCumulative = false + animation.fillMode = kCAFillModeForwards + animation.isRemovedOnCompletion = false + animation.repeatCount = .infinity + animation.duration = duration + animation.calculationMode = kCAAnimationPaced + + return animation + + } + + /** + * Creates the animations to display the placeholder status. The returned animations should + * be enqueued as soon as possible. + */ + + private func displayStatusPlaceholder() -> [Animation] { + + var animations: [Animation] = [] + statusPlaceholderContainer.transform = CGAffineTransform(scaleX: 0, y: 0) + + let upscaleStatusPrompt = Animation(type: .property(0.4, .easeIn)) { + self.statusPlaceholderContainer.transform = CGAffineTransform(scaleX: 1.1, y: 1.1) + self.statusPlaceholderContainer.alpha = 0.69 + } + + let finalizeStatusPrompt = Animation(type: .property(0.2, .easeOut)) { + self.statusPlaceholderContainer.transform = .identity + self.statusPlaceholderContainer.alpha = 1 + } + + animations.append(upscaleStatusPrompt) + animations.append(finalizeStatusPrompt) + + return animations + + } + + // MARK: - Interacting with the Results + + /** + * Starts the result view. + * + * This method adds the gravity floating animation. + */ + + func prepareForDisplay() { + state = .displayed + floatPlaceholders() + setNeedsLayout() + } + + /** + * Returns the animations to remove the placeholder and display the draw prompt. + */ + + func startPredictions() -> [Animation] { + + state = .startedPredictions + + var animations: [Animation] = [] + + // Remove the placeholders + + for line in self.placeholderLineStacks.reversed() { + + let removePlaceholders = line.arrangedSubviews.reversed().map { placeholder in + Animation(type: .property(0.25, .linear)) { placeholder.alpha = 0 } + } + + animations.append(contentsOf: removePlaceholders) + + } + + // Hide the lines + + let hideLines = self.placeholderLineStacks.map { line in + Animation(type: .notAnimated) { line.isHidden = true } + } + + animations.append(contentsOf: hideLines) + + // Show the drawing prompt + + statusPlaceholder.text = kStatusDraw + + let statusOnAnimations = displayStatusPlaceholder() + animations.append(contentsOf: statusOnAnimations) + + return animations + + } + + /** + * Removes the state placeholder. + */ + + func startDrawing() { + + guard state == .startedPredictions else { + return + } + + state = .waitingInitialPrediction + + enqueueChanges(animation: .property(0.25, .linear), changes: { + self.statusPlaceholderContainer.alpha = 0 + }) + + } + + /** + * Removes all the result nodes and displays the placeholders. + */ + + func clear() { + + enqueueChanges(animation: .transition(0.35, .transitionCrossDissolve), changes: { + self.resultNodes.forEach({ $0.reset() }) + }) + + } + + /** + * Handles the output of a Core ML prediction. + */ + + func handle(result: EmojiPrediction) { + + state = .predicted + + // Get the result sorted by highest likeliness + + let sortedResuls = result.predictions.sorted { + $0.value > $1.value + } + + // Only display the first 3 results + let top3 = sortedResuls.prefix(3) + + // If there are no predictions, return early + + if top3.count < 3 { + handleNoPrediction() + return + } + + // Remove the status + + enqueueChanges(animation: .property(0.25, .linear), changes: { + self.statusPlaceholderContainer.alpha = 0 + }) + + // Update the elements + + for (prediction, node) in zip(top3, resultNodes) { + + guard let classLabel = Class(rawValue: prediction.key) else { + continue + } + + node.alpha = 1 + node.display(output: prediction.value, for: classLabel) + + } + + } + + /** + * Handles the case where no predictions are found. + */ + + private func handleNoPrediction() { + + var animations: [Animation] = [] + + statusPlaceholder.text = kStatusFailure + let statusOnAnimations = displayStatusPlaceholder() + animations.append(contentsOf: statusOnAnimations) + + if state == .predicted { + + let removeResults = Animation(type: .transition(0.35, .transitionCrossDissolve)) { + self.resultNodes.forEach({ $0.reset() }) + } + + animations.append(removeResults) + + } + + state = .noPrediction + enqueueChanges(animations) + + } + + // MARK: - Layout + + override func layoutSubviews() { + + if case .displayed = state { + floatPlaceholders() + } + + let isCompactWidth = frame.size.width < 624 + updateLayout(isCompact: isCompactWidth) + + } + + func updateLayout(isCompact: Bool) { + + small_left_trailing?.isActive = isCompact + small_left_bottom?.isActive = isCompact + small_middle_leading?.isActive = isCompact + small_middle_bottom?.isActive = isCompact + small_right_top?.isActive = isCompact + small_right_centerX?.isActive = isCompact + + large_middle_centerX?.isActive = !isCompact + large_middle_centerY?.isActive = !isCompact + large_left_trailing?.isActive = !isCompact + large_left_centerY?.isActive = !isCompact + large_right_leading?.isActive = !isCompact + large_right_centerY?.isActive = !isCompact + + } + +} diff --git a/Playground/MLMOJI.playgroundbook/Contents/Chapters/Chapter1.playgroundchapter/Pages/Page1.playgroundpage/Sources/PredictionViewController+Playground.swift b/Playground/MLMOJI.playgroundbook/Contents/Chapters/Chapter1.playgroundchapter/Pages/Page1.playgroundpage/Sources/PredictionViewController+Playground.swift new file mode 100644 index 0000000..0329fb5 --- /dev/null +++ b/Playground/MLMOJI.playgroundbook/Contents/Chapters/Chapter1.playgroundchapter/Pages/Page1.playgroundpage/Sources/PredictionViewController+Playground.swift @@ -0,0 +1,35 @@ +// +// MLMOJI +// +// This file is part of Alexis Aubry's WWDC18 scolarship submission open source project. +// Copyright © 2018 Alexis Aubry. Available under the terms of the MIT License. +// + +import UIKit +import PlaygroundSupport + +extension PredictionViewController: PlaygroundContainable {} + +/** + * Displays a prediction view controller in the safe are, and handles incoming messages. + */ + +public class PlaygroundPredictionViewController: PlaygroundViewController, PlaygroundLiveViewMessageHandler { + + public convenience init() { + let vc = PredictionViewController() + self.init(child: vc) + } + + public func receive(_ message: PlaygroundValue) { + + guard case let .dictionary(dict) = message else { return } + guard case let .string(eventName)? = dict[MessageType.event] else { return } + + if eventName == Event.startPredictions.rawValue { + child.startPredictions() + } + + } + +} diff --git a/Playground/MLMOJI.playgroundbook/Contents/Chapters/Chapter1.playgroundchapter/Pages/Page1.playgroundpage/Sources/PredictionViewController.swift b/Playground/MLMOJI.playgroundbook/Contents/Chapters/Chapter1.playgroundchapter/Pages/Page1.playgroundpage/Sources/PredictionViewController.swift new file mode 100644 index 0000000..579787e --- /dev/null +++ b/Playground/MLMOJI.playgroundbook/Contents/Chapters/Chapter1.playgroundchapter/Pages/Page1.playgroundpage/Sources/PredictionViewController.swift @@ -0,0 +1,222 @@ +// +// MLMOJI +// +// This file is part of Alexis Aubry's WWDC18 scolarship submission open source project. +// Copyright © 2018 Alexis Aubry. Available under the terms of the MIT License. +// + +import UIKit + +/** + * A view controller that displays a predictive keyboard and the results interface. + */ + +public class PredictionViewController: UIViewController { + + /// The session that will handle predictions. + let session = PredictionSession() + + /// Whether the keyboard is started. + private(set) var startedPredictions: Bool = false + + /// The layout guide of the container. + public var containerLayoutGuide: UILayoutGuide! + + // MARK: - UI Properties + + /// The paper where the user will draw. + let paper = Paper() + + /// The view that displays the results. + let resultsContainer = PredictionResultsContainer() + + /// The view containing the paper. + let keyboardContainer = UIView() + + /// The button to clear the contents of the canvas. + let clearButton = UIButton(type: .system) + + // MARK: - Initialization + + override public func loadView() { + super.loadView() + + // Configure the keyboard container + keyboardContainer.backgroundColor = #colorLiteral(red: 0.8196078431, green: 0.8352941176, blue: 0.8588235294, alpha: 1) + + // Configure the paper + paper.backgroundColor = .white + paper.isUserInteractionEnabled = false + + // Configure the clear button + let clearIcon = UIImage(named: "KeyboardClear")!.withRenderingMode(.alwaysTemplate) + clearButton.setImage(clearIcon, for: .normal) + clearButton.adjustsImageWhenHighlighted = true + clearButton.isEnabled = false + clearButton.addTarget(self, action: #selector(clearCanvas), for: .touchUpInside) + + // Configure the session + session.delegate = self + + } + + override public func viewDidAppear(_ animated: Bool) { + super.viewDidAppear(animated) + resultsContainer.prepareForDisplay() + } + + /// Configures the Auto Layout constraints of the view hierarchy. + public func configureConstraints() { + + let keyboardSize: CGFloat = 250 + + // Add the keyboard container and move it outside of the screen + + view.addSubview(keyboardContainer) + keyboardContainer.translatesAutoresizingMaskIntoConstraints = false + keyboardContainer.leadingAnchor.constraint(equalTo: view.leadingAnchor).isActive = true + keyboardContainer.trailingAnchor.constraint(equalTo: view.trailingAnchor).isActive = true + keyboardContainer.bottomAnchor.constraint(equalTo: view.bottomAnchor).isActive = true + + keyboardContainer.transform = CGAffineTransform(translationX: 0, y: keyboardSize + 150) + resultsContainer.stackView.transform = CGAffineTransform(translationX: 0, y: 150) + + // Add the paper + keyboardContainer.addSubview(paper) + paper.translatesAutoresizingMaskIntoConstraints = false + paper.bottomAnchor.constraint(equalTo: containerLayoutGuide.bottomAnchor).isActive = true + paper.centerXAnchor.constraint(equalTo: containerLayoutGuide.centerXAnchor).isActive = true + + // Add the undo button + keyboardContainer.addSubview(clearButton) + clearButton.translatesAutoresizingMaskIntoConstraints = false + clearButton.topAnchor.constraint(equalTo: keyboardContainer.topAnchor, constant: 16).isActive = true + clearButton.trailingAnchor.constraint(equalTo: keyboardContainer.trailingAnchor, constant: -16).isActive = true + + // Constrain the size of the keyboard + paper.heightAnchor.constraint(equalToConstant: keyboardSize).isActive = true + paper.widthAnchor.constraint(equalToConstant: keyboardSize).isActive = true + keyboardContainer.topAnchor.constraint(equalTo: paper.topAnchor).isActive = true + + // Add the results container + view.addSubview(resultsContainer) + resultsContainer.translatesAutoresizingMaskIntoConstraints = false + resultsContainer.leadingAnchor.constraint(equalTo: containerLayoutGuide.leadingAnchor).isActive = true + resultsContainer.trailingAnchor.constraint(equalTo: containerLayoutGuide.trailingAnchor).isActive = true + resultsContainer.topAnchor.constraint(equalTo: containerLayoutGuide.topAnchor, constant: 16).isActive = true + resultsContainer.bottomAnchor.constraint(equalTo: keyboardContainer.topAnchor, constant: -16).isActive = true + + } + + // MARK: - Interacting with the Paper + + /** + * Starts the predictive keyboard. + */ + + func startPredictions() { + + paper.delegate = self + paper.isUserInteractionEnabled = true + paper.becomeFirstResponder() + + clearCanvas() + + let curve = UIViewAnimationCurve(rawValue: 7)! + + // Move the keyboard on the screen + + guard !startedPredictions else { + return + } + + var animations = resultsContainer.startPredictions() + + let showKeyboard = Animation(type: .property(0.35, curve)) { + self.resultsContainer.stackView.transform = .identity + self.keyboardContainer.transform = .identity + } + + animations.append(showKeyboard) + view.enqueueChanges(animations) + + startedPredictions = true + + } + + /** + * Clears the canvas and resets the state of the prediction engine. + */ + + @objc private func clearCanvas() { + paper.clear() + resultsContainer.clear() + clearButton.isEnabled = false + resultsContainer.state = startedPredictions ? .startedPredictions : .displayed + } + +} + +// MARK: - PaperDelegate + +extension PredictionViewController: PaperDelegate { + + /** + * When the user starts drawing, notify the results view. + */ + + public func paperDidStartDrawingStroke(_ paper: Paper) { + resultsContainer.startDrawing() + } + + /** + * When a new stroke is added, request a new prediction. + */ + + public func paperDidFinishDrawingStroke(_ paper: Paper) { + + clearButton.isEnabled = true + + let exporter = CVPixelBufferExporter(size: session.inputSize, scale: 1) + + do { + let imageBuffer = try paper.export(with: exporter) + session.requestPrediction(for: imageBuffer) + } catch { + handleError(error) + } + + } + +} + +extension PredictionViewController: PredictionSessionDelegate { + + public func predictionSession(_ session: PredictionSession, didUpdatePrediction prediction: EmojiPrediction) { + resultsContainer.handle(result: prediction) + } + + public func predictionSession(_ session: PredictionSession, didFailToProvidePredictionWith error: NSError) { + handleError(error) + } + + /** + * Handles a prediction error. + */ + + private func handleError(_ error: Error) { + + let alert = UIAlertController(title: "Could not recognize emoji", + message: (error as NSError).localizedDescription, + preferredStyle: .alert) + + let ok = UIAlertAction(title: "OK", style: .cancel) { _ in + self.clearCanvas() + } + + alert.addAction(ok) + present(alert, animated: true, completion: nil) + + } + +} diff --git a/Playground/MLMOJI.playgroundbook/Contents/Chapters/Chapter1.playgroundchapter/Pages/Page2.playgroundpage/Contents.swift b/Playground/MLMOJI.playgroundbook/Contents/Chapters/Chapter1.playgroundchapter/Pages/Page2.playgroundpage/Contents.swift new file mode 100644 index 0000000..d7b4349 --- /dev/null +++ b/Playground/MLMOJI.playgroundbook/Contents/Chapters/Chapter1.playgroundchapter/Pages/Page2.playgroundpage/Contents.swift @@ -0,0 +1,139 @@ +//#-hidden-code +import UIKit +import PlaygroundSupport + +var pendingFilters: [AugmentationFilter] = [] +PlaygroundPage.current.needsIndefiniteExecution = true + +/** + * Rotates the image by the specified angle, in degrees. + * - parameter degress: The angle to rotate the image by. + */ + +func rotate(by degrees: CGFloat) { + pendingFilters.append(.rotate(degrees)) +} + +/** + * Moves the image on the screen. + * - parameter x: The number of pixels to move the image by, horizontally. Can be negative. + * - parameter y: The number of pixels to move the image by, vertically. Can be negative. + */ + +func move(x: CGFloat, y: CGFloat) { + pendingFilters.append(.translate(x, y)) +} + +/** + * Zooms in or out of the image from the center, to the specified percentage. + * + * - parameter percentage: The percentage to scale the image to. Use a negative value to zoom + * out. Use a positive value to zoom in. + */ + +func zoom(percentage: CGFloat) { + pendingFilters.append(.scale(percentage)) +} + +/** + * Applies a light gaussian blur to the image. + */ + +func blur() { + pendingFilters.append(.blur) +} + +//#-end-hidden-code +/*: + # Building a Data Set + + --- + + The emoji recognizer you just tried is powered by a neural network, which was trained with a **data set** prior + to being embedded into the playground. + + Neural networks need to be trained with real world images to learn the [features](glossary://feature) of each outcome + before they can be used. So, for this task, we need to draw emoji for each class we will be predicting. + + However, the [classifier](glossary://classifier) needs hundreds, if not thousands of very different images + for each class to reach satisfactory accuracy. As drawing is a very, *very* repetitive task, we need a + process that generates the sketches for us. + + ## Meet Data Augmentation + + Some research are being conducted to explore generating synthetic images programmatically. For simplicity + reasons, we will use a more approachable technique: **data augmentation**. + + Data augmentation allows us to generate new data from existing real images. With operations such as scaling, + rotation, translation or blurring, we can create images that derive from an existing shape, but that include + new features for the classifier to learn; because the augmented images look different. + + This is an example set of augmented images from a heart sketch: + + ![AugmentedSet.png](AugmentedSet.png) + + As you can see, 9 different drawings were generated from one sketch. The main benefit of this technique is + its speed: it can increase the size of a medium-sized data set by ten times in less than 5 minutes. + + ## Augment your own drawing + + Now that we've learned about data augmentation, why not try it with our own drawing? To complete this challenge, you can follow these steps: + + 1. Add a list of filters in the code editor below + + We will be using **cumulative augmentation**. Each filter is a function that will be applied on top of the previous one to produce the final image, in the order you write them. + + You can use these augmentation filters: + + - `rotate(by:)` using an angle in degrees. + + - `move(x:y:)` with numbers describing the movement of the image on the screen, in horizontal + and vertical points. + + - `zoom(percentage:)` with a zoom percentage. A percentage lower than `100` makes the + image smaller. A percentage greater than `100` makes the image bigger. + +- `blur()` to blur the image with a light Gaussian blur. + + You drawing will be converted to a 250x250 pixel image, to which the filters will be applied. Keep + these dimensions in mind when using the `move(x:y:)` filter, or your image may + move off the screen. + + 2. Run your code + + Feel free to experiment with different augmentation sequences by combining filters in multiple orders, + using different arguments and repeating filters. + + */ +//#-code-completion(everything, hide) +//#-code-completion(identifier, show, rotate(by:)) +//#-code-completion(identifier, show, move(x:y:)) +//#-code-completion(identifier, show, zoom(percentage:)) +//#-code-completion(identifier, show, blur()) +//#-editable-code Tap to write your code +//#-end-editable-code + +//#-hidden-code + +if !pendingFilters.isEmpty { + + let vc = AugmentViewController(filters: pendingFilters) + + vc.completionHandler = { + + let pluralMark = pendingFilters.count > 1 ? "s" : "" + + PlaygroundPage.current.assessmentStatus = .pass(message: "You have created an augmented data set with \(pendingFilters.count) new image\(pluralMark)! You can move to [next page](@next), or keep experimenting with augmentation by editing your code.") + + } + + PlaygroundPage.current.liveView = PlaygroundViewController(child: vc) + +} else { + + PlaygroundPage.current.assessmentStatus = .fail(hints: ["You did not add filters to augment your image."], + solution: "Add at least one filter between the brackets in the `augmentImage()` function.") + +} + +//#-end-hidden-code diff --git a/Playground/MLMOJI.playgroundbook/Contents/Chapters/Chapter1.playgroundchapter/Pages/Page2.playgroundpage/Manifest.plist b/Playground/MLMOJI.playgroundbook/Contents/Chapters/Chapter1.playgroundchapter/Pages/Page2.playgroundpage/Manifest.plist new file mode 100644 index 0000000..f5d4732 --- /dev/null +++ b/Playground/MLMOJI.playgroundbook/Contents/Chapters/Chapter1.playgroundchapter/Pages/Page2.playgroundpage/Manifest.plist @@ -0,0 +1,21 @@ + + + + + Name + Building a Data Set + LiveViewEdgeToEdge + + LiveViewMode + VisibleByDefault + CodeCopySetup + + ReadyToCopyInstructions + Let's start this challenge! Import the code from the previous page to start. + NotReadyToCopyInstructions + Code 420: You need to complete the previous challenge before starting this one. + + PlaygroundLoggingMode + Off + + diff --git a/Playground/MLMOJI.playgroundbook/Contents/Chapters/Chapter1.playgroundchapter/Pages/Page2.playgroundpage/PrivateResources/AugmentedSet@2x.png b/Playground/MLMOJI.playgroundbook/Contents/Chapters/Chapter1.playgroundchapter/Pages/Page2.playgroundpage/PrivateResources/AugmentedSet@2x.png new file mode 100644 index 0000000..0063ede Binary files /dev/null and b/Playground/MLMOJI.playgroundbook/Contents/Chapters/Chapter1.playgroundchapter/Pages/Page2.playgroundpage/PrivateResources/AugmentedSet@2x.png differ diff --git a/Playground/MLMOJI.playgroundbook/Contents/Chapters/Chapter1.playgroundchapter/Pages/Page2.playgroundpage/Sources/AugmentViewController+Playground.swift b/Playground/MLMOJI.playgroundbook/Contents/Chapters/Chapter1.playgroundchapter/Pages/Page2.playgroundpage/Sources/AugmentViewController+Playground.swift new file mode 100644 index 0000000..458f696 --- /dev/null +++ b/Playground/MLMOJI.playgroundbook/Contents/Chapters/Chapter1.playgroundchapter/Pages/Page2.playgroundpage/Sources/AugmentViewController+Playground.swift @@ -0,0 +1,8 @@ +// +// MLMOJI +// +// This file is part of Alexis Aubry's WWDC18 scolarship submission open source project. +// Copyright © 2018 Alexis Aubry. Available under the terms of the MIT License. +// + +extension AugmentViewController: PlaygroundContainable {} diff --git a/Playground/MLMOJI.playgroundbook/Contents/Chapters/Chapter1.playgroundchapter/Pages/Page2.playgroundpage/Sources/AugmentViewController.swift b/Playground/MLMOJI.playgroundbook/Contents/Chapters/Chapter1.playgroundchapter/Pages/Page2.playgroundpage/Sources/AugmentViewController.swift new file mode 100644 index 0000000..416bac1 --- /dev/null +++ b/Playground/MLMOJI.playgroundbook/Contents/Chapters/Chapter1.playgroundchapter/Pages/Page2.playgroundpage/Sources/AugmentViewController.swift @@ -0,0 +1,279 @@ +// +// MLMOJI +// +// This file is part of Alexis Aubry's WWDC18 scolarship submission open source project. +// Copyright © 2018 Alexis Aubry. Available under the terms of the MIT License. +// + +import UIKit +import MobileCoreServices + +private let kAugmentCell = "ImageCollectionViewCell" + +/** + * A view controller to perform data augmentation. + */ + +public class AugmentViewController: UIViewController { + + /// The filters to apply. + let filters: [AugmentationFilter] + + /// The list of augmented images received so far. + var augmentedImages: [UIImage] = [] + + /// The block called when augmentation has finished. + public var completionHandler: (() -> Void)? = nil + + /// The active augmentation session. + private var currentSession: AugmentationSession? = nil + + // MARK: - Views + + let paper = Paper() + let clearButton = UIButton() + let confirmButton = UIButton(type: .system) + let drawContainer = UIView() + + let layout: UICollectionViewFlowLayout = { + let layout = UICollectionViewFlowLayout() + layout.minimumInteritemSpacing = 1 + return layout + }() + + lazy var collectionView: UICollectionView = { + return UICollectionView(frame: .zero, collectionViewLayout: layout) + }() + + public var containerLayoutGuide: UILayoutGuide! = nil + + // MARK: - Initialization + + public init(filters: [AugmentationFilter]) { + self.filters = filters + super.init(nibName: nil, bundle: nil) + } + + public required init?(coder aDecoder: NSCoder) { + fatalError("init(coder:) was not implemented.") + } + + public override func loadView() { + super.loadView() + configureViews() + } + + /** + * Configures the views. + */ + + private func configureViews() { + + // Configure the clear button + + let clearIcon = UIImage(named: "KeyboardClear")!.withRenderingMode(.alwaysTemplate) + clearButton.setImage(clearIcon, for: .normal) + clearButton.adjustsImageWhenHighlighted = true + clearButton.isEnabled = false + clearButton.addTarget(self, action: #selector(clearCanvas), for: .touchUpInside) + + // Configure the confirm button + + confirmButton.setTitle("Augment Drawing", for: .normal) + confirmButton.setTitleColor(.white, for: .normal) + confirmButton.titleLabel?.font = UIFont.systemFont(ofSize: 17, weight: .semibold) + + let bgImage = UIGraphicsImageRenderer(size: CGSize(width: 1, height: 1)).image { context in + #colorLiteral(red: 0, green: 0.4352941176, blue: 1, alpha: 1).setFill() + context.fill(CGRect(x: 0, y: 0, width: 1, height: 1)) + } + + confirmButton.setBackgroundImage(bgImage, for: .normal) + confirmButton.layer.cornerRadius = 12 + confirmButton.clipsToBounds = true + + confirmButton.addTarget(self, action: #selector(confirm), for: .touchUpInside) + confirmButton.adjustsImageWhenHighlighted = true + confirmButton.adjustsImageWhenDisabled = true + confirmButton.isEnabled = false + + // Configure the paper + + paper.backgroundColor = .white + paper.layer.cornerRadius = 34 + paper.clipsToBounds = true + paper.delegate = self + + // Configure the collection view + + collectionView.backgroundColor = .clear + collectionView.register(ImageCollectionViewCell.self, forCellWithReuseIdentifier:kAugmentCell) + collectionView.delegate = self + collectionView.dataSource = self + + } + + /** + * Configures the Auto Layout constraint. + */ + + public func configureConstraints() { + + view.addSubview(collectionView) + collectionView.translatesAutoresizingMaskIntoConstraints = false + collectionView.leadingAnchor.constraint(equalTo: containerLayoutGuide.leadingAnchor, constant: 44).isActive = true + collectionView.trailingAnchor.constraint(equalTo: containerLayoutGuide.trailingAnchor, constant: -44).isActive = true + collectionView.topAnchor.constraint(equalTo: containerLayoutGuide.topAnchor, constant: 44).isActive = true + collectionView.bottomAnchor.constraint(equalTo: containerLayoutGuide.bottomAnchor, constant: -44).isActive = true + + view.addSubview(drawContainer) + drawContainer.translatesAutoresizingMaskIntoConstraints = false + drawContainer.centerXAnchor.constraint(equalTo: view.centerXAnchor).isActive = true + drawContainer.centerYAnchor.constraint(equalTo: view.centerYAnchor).isActive = true + + drawContainer.addSubview(paper) + paper.translatesAutoresizingMaskIntoConstraints = false + paper.widthAnchor.constraint(equalToConstant: 250).isActive = true + paper.heightAnchor.constraint(equalToConstant: 250).isActive = true + paper.topAnchor.constraint(equalTo: drawContainer.topAnchor).isActive = true + paper.leadingAnchor.constraint(equalTo: drawContainer.leadingAnchor).isActive = true + + drawContainer.trailingAnchor.constraint(equalTo: paper.trailingAnchor).isActive = true + + view.addSubview(clearButton) + clearButton.translatesAutoresizingMaskIntoConstraints = false + clearButton.leadingAnchor.constraint(equalTo: drawContainer.trailingAnchor, constant: 16).isActive = true + clearButton.topAnchor.constraint(equalTo: drawContainer.topAnchor).isActive = true + clearButton.widthAnchor.constraint(equalToConstant: 44).isActive = true + clearButton.heightAnchor.constraint(equalToConstant: 44).isActive = true + + drawContainer.addSubview(confirmButton) + confirmButton.translatesAutoresizingMaskIntoConstraints = false + confirmButton.topAnchor.constraint(equalTo: paper.bottomAnchor, constant: 16).isActive = true + confirmButton.leadingAnchor.constraint(equalTo: paper.leadingAnchor).isActive = true + confirmButton.trailingAnchor.constraint(equalTo: paper.trailingAnchor).isActive = true + confirmButton.heightAnchor.constraint(equalToConstant: 55).isActive = true + drawContainer.bottomAnchor.constraint(equalTo: confirmButton.bottomAnchor).isActive = true + + } + + public override func viewDidLayoutSubviews() { + super.viewDidLayoutSubviews() + layout.invalidateLayout() + } + + // MARK: - Drawing + + /** + * Clears the canvas. + */ + + @objc private func clearCanvas() { + paper.clear() + clearButton.isEnabled = false + confirmButton.isEnabled = false + } + + /** + * Confirms the drawing and starts augmentation. + */ + + @objc private func confirm() { + + guard self.currentSession == nil else { + return + } + + let startAugmentation: (Bool) -> Void = { _ in + + let session = AugmentationSession(actions: self.filters) + session.delegate = self + + let imageSize = CGSize(width: 250, height: 250) + let initialImage = self.paper.exportImage(size: imageSize) + let initialUIImage = UIImage(cgImage: initialImage) + + self.currentSession = session + self.insertImage(initialUIImage) + session.start(initialImage: initialImage) + + } + + view.enqueueChanges(animation: .property(0.35, .linear), changes: { + self.drawContainer.alpha = 0 + self.drawContainer.isUserInteractionEnabled = false + self.clearButton.alpha = 0 + self.clearButton.isUserInteractionEnabled = false + }, completion: startAugmentation) + + } + +} + +extension AugmentViewController: AugmentationSessionDelegate { + + public func augmentationSession(_ session: AugmentationSession, didCreateImage image: UIImage) { + guard currentSession == session else { + return + } + + insertImage(image) + } + + public func augmentationSessionDidFinish(_ session: AugmentationSession) { + guard currentSession == session else { + return + } + + currentSession = nil + completionHandler?() + } + + func insertImage(_ image: UIImage) { + let collectionEndIndex = augmentedImages.endIndex + augmentedImages.append(image) + collectionView.insertItems(at: [IndexPath(row: collectionEndIndex, section: 0)]) + } + +} + +extension AugmentViewController: PaperDelegate { + + public func paperDidStartDrawingStroke(_ paper: Paper) {} + + public func paperDidFinishDrawingStroke(_ paper: Paper) { + clearButton.isEnabled = true + confirmButton.isEnabled = true + } + +} + +extension AugmentViewController: UICollectionViewDelegateFlowLayout, UICollectionViewDataSource { + + public func numberOfSections(in collectionView: UICollectionView) -> Int { + return 1 + } + + public func collectionView(_ collectionView: UICollectionView, numberOfItemsInSection section: Int) -> Int { + return augmentedImages.count + } + + public func collectionView(_ collectionView: UICollectionView, cellForItemAt indexPath: IndexPath) -> UICollectionViewCell { + + let cell = collectionView.dequeueReusableCell(withReuseIdentifier: kAugmentCell, for: indexPath) as! ImageCollectionViewCell + + cell.configure(image: augmentedImages[indexPath.row]) + return cell + + } + + public func collectionView(_ collectionView: UICollectionView, + layout collectionViewLayout: UICollectionViewLayout, + sizeForItemAt indexPath: IndexPath) -> CGSize { + let containerSize = collectionView.frame.size.width + let numberOfCellsPerLine: CGFloat = containerSize < 700 ? 3 : 5 + let cellWidth = (containerSize / numberOfCellsPerLine) - (numberOfCellsPerLine * 3) + return CGSize(width: cellWidth, height: cellWidth) + } + +} diff --git a/Playground/MLMOJI.playgroundbook/Contents/Chapters/Chapter1.playgroundchapter/Pages/Page2.playgroundpage/Sources/ImageCollectionViewCell.swift b/Playground/MLMOJI.playgroundbook/Contents/Chapters/Chapter1.playgroundchapter/Pages/Page2.playgroundpage/Sources/ImageCollectionViewCell.swift new file mode 100644 index 0000000..7645e28 --- /dev/null +++ b/Playground/MLMOJI.playgroundbook/Contents/Chapters/Chapter1.playgroundchapter/Pages/Page2.playgroundpage/Sources/ImageCollectionViewCell.swift @@ -0,0 +1,55 @@ +// +// MLMOJI +// +// This file is part of Alexis Aubry's WWDC18 scolarship submission open source project. +// Copyright © 2018 Alexis Aubry. Available under the terms of the MIT License. +// + +import UIKit + +/** + * A collection view cell that displays an image. + */ + +class ImageCollectionViewCell: UICollectionViewCell { + + /// The image view that displays the contents of the cell. + private let imageView = UIImageView() + + // MARK: - Initializtion + + override init(frame: CGRect) { + super.init(frame: frame) + configureViews() + } + + required init?(coder aDecoder: NSCoder) { + super.init(coder: aDecoder) + configureViews() + } + + /** + * Configures the subviews of the cell. + */ + + private func configureViews() { + addSubview(imageView) + imageView.contentMode = .scaleAspectFill + imageView.pinEdges(to: self) + } + + // MARK: - Changing the Contents + + /** + * Configures the cell to display the specified image. + */ + + func configure(image: UIImage) { + self.imageView.image = image + } + + override func prepareForReuse() { + imageView.image = nil + } + +} diff --git a/Playground/MLMOJI.playgroundbook/Contents/Chapters/Chapter1.playgroundchapter/Pages/Page3.playgroundpage/Contents.swift b/Playground/MLMOJI.playgroundbook/Contents/Chapters/Chapter1.playgroundchapter/Pages/Page3.playgroundpage/Contents.swift new file mode 100644 index 0000000..75d490a --- /dev/null +++ b/Playground/MLMOJI.playgroundbook/Contents/Chapters/Chapter1.playgroundchapter/Pages/Page3.playgroundpage/Contents.swift @@ -0,0 +1,29 @@ +/*: + + # Congratulations! + + You now have completed this book. 💯 + + In this Playground, we have explored an interactive application of machine learning with the [emoji classifier](MLMOJI/Introducing%20MLMOJI), + and learned more about building a data set from scratch by taking advantage of [data augmentation](MLMOJI/Building%20a%20Data%20Set). + + ## Where to Go Next? + + --- + + If you wish to learn more, reusing the knowledge from this book, there are some related topics and questions worth exploring: + + - What is the structure of a neural network? What is an **activation function**? + + - What are **convolutions** ? Why are they commonly used for image processing? + + - How to train a neural network faster by using another trained model with **transfer learning**? + + - What is **overfitting** and how to prevent it? + + - What are the different machine learning tools? + + - Callout(Thank you for reading! 📖): I hope this quick introduction to machine learning and image + classification interested you. + + */ diff --git a/Playground/MLMOJI.playgroundbook/Contents/Chapters/Chapter1.playgroundchapter/Pages/Page3.playgroundpage/LiveView.swift b/Playground/MLMOJI.playgroundbook/Contents/Chapters/Chapter1.playgroundchapter/Pages/Page3.playgroundpage/LiveView.swift new file mode 100644 index 0000000..bb571e2 --- /dev/null +++ b/Playground/MLMOJI.playgroundbook/Contents/Chapters/Chapter1.playgroundchapter/Pages/Page3.playgroundpage/LiveView.swift @@ -0,0 +1,25 @@ +// +// MLMOJI +// +// This file is part of Alexis Aubry's WWDC18 scolarship submission open source project. +// Copyright © 2018 Alexis Aubry. Available under the terms of the MIT License. +// + +import UIKit +import PlaygroundSupport + +let label = UILabel() +label.text = "🎉" +label.font = UIFont.systemFont(ofSize: 123) +label.textAlignment = .center +label.baselineAdjustment = .alignCenters +label.numberOfLines = 1 + +let pulseAnimation = CABasicAnimation(keyPath: "transform.scale") +pulseAnimation.duration = 3.0 +pulseAnimation.toValue = NSNumber(value: 0.8) +pulseAnimation.repeatCount = Float.infinity +pulseAnimation.autoreverses = true + +label.layer.add(pulseAnimation, forKey: nil) +PlaygroundPage.current.liveView = label diff --git a/Playground/MLMOJI.playgroundbook/Contents/Chapters/Chapter1.playgroundchapter/Pages/Page3.playgroundpage/Manifest.plist b/Playground/MLMOJI.playgroundbook/Contents/Chapters/Chapter1.playgroundchapter/Pages/Page3.playgroundpage/Manifest.plist new file mode 100644 index 0000000..8e930b3 --- /dev/null +++ b/Playground/MLMOJI.playgroundbook/Contents/Chapters/Chapter1.playgroundchapter/Pages/Page3.playgroundpage/Manifest.plist @@ -0,0 +1,21 @@ + + + + + Name + Where to Go Next? + LiveViewEdgeToEdge + + LiveViewMode + VisibleByDefault + CodeCopySetup + + ReadyToCopyInstructions + Let's start this challenge! Import the code from the previous page to start. + NotReadyToCopyInstructions + Code 420: You need to complete the previous challenge before starting this one. + + PlaygroundLoggingMode + Off + + diff --git a/Playground/MLMOJI.playgroundbook/Contents/Manifest.plist b/Playground/MLMOJI.playgroundbook/Contents/Manifest.plist new file mode 100644 index 0000000..6f171dd --- /dev/null +++ b/Playground/MLMOJI.playgroundbook/Contents/Manifest.plist @@ -0,0 +1,28 @@ + + + + + Name + MLMOJI + DevelopmentRegion + en-US + ContentIdentifier + fr.alexaubry.wwdc18.mlmoji + ContentVersion + 1.0.0 + ImageReference + Cover.png + DeploymentTarget + ios11.0 + SwiftVersion + 4.0 + MinimumSwiftPlaygroundsVersion + 1.6 + Version + 3.0 + Chapters + + Chapter1.playgroundchapter + + + diff --git a/Playground/MLMOJI.playgroundbook/Contents/PrivateResources/Cover.png b/Playground/MLMOJI.playgroundbook/Contents/PrivateResources/Cover.png new file mode 100644 index 0000000..8b1f9f6 Binary files /dev/null and b/Playground/MLMOJI.playgroundbook/Contents/PrivateResources/Cover.png differ diff --git a/Playground/MLMOJI.playgroundbook/Contents/PrivateResources/EmojiSketches.mlmodelc/coremldata.bin b/Playground/MLMOJI.playgroundbook/Contents/PrivateResources/EmojiSketches.mlmodelc/coremldata.bin new file mode 100644 index 0000000..61114e9 Binary files /dev/null and b/Playground/MLMOJI.playgroundbook/Contents/PrivateResources/EmojiSketches.mlmodelc/coremldata.bin differ diff --git a/Playground/MLMOJI.playgroundbook/Contents/PrivateResources/EmojiSketches.mlmodelc/model.espresso.net b/Playground/MLMOJI.playgroundbook/Contents/PrivateResources/EmojiSketches.mlmodelc/model.espresso.net new file mode 100644 index 0000000..5d2e3a0 --- /dev/null +++ b/Playground/MLMOJI.playgroundbook/Contents/PrivateResources/EmojiSketches.mlmodelc/model.espresso.net @@ -0,0 +1,1852 @@ +{ + "storage" : "model.espresso.weights", + "properties" : { + + }, + "format_version" : 200, + "layers" : [ + { + "pad_r" : 0, + "fused_relu" : 1, + "fused_tanh" : 0, + "pad_fill_mode" : 0, + "pad_b" : 0, + "pad_l" : 0, + "top" : "relu_MobilenetV1\/MobilenetV1\/Conv2d_0\/Relu6:0", + "blob_weights" : 3, + "K" : 3, + "blob_biases" : 1, + "stride_x" : 2, + "name" : "MobilenetV1\/MobilenetV1\/Conv2d_0\/Conv2D:0", + "has_batch_norm" : 0, + "type" : "convolution", + "n_groups" : 1, + "pad_t" : 0, + "stride_y" : 2, + "has_biases" : 1, + "C" : 32, + "bottom" : "input__0", + "weights" : { + + }, + "Nx" : 3, + "pad_mode" : 1, + "pad_value" : 0, + "Ny" : 3, + "n_parallel" : 1 + }, + { + "alpha" : -1, + "bottom" : "relu_MobilenetV1\/MobilenetV1\/Conv2d_0\/Relu6:0", + "weights" : { + + }, + "mode" : 6, + "top" : "relu_MobilenetV1\/MobilenetV1\/Conv2d_0\/Relu6:0_neg", + "type" : "activation", + "name" : "relu_MobilenetV1\/MobilenetV1\/Conv2d_0\/Relu6:0_neg", + "beta" : 0 + }, + { + "alpha" : -6, + "bottom" : "relu_MobilenetV1\/MobilenetV1\/Conv2d_0\/Relu6:0_neg", + "operation" : 25, + "weights" : { + + }, + "fused_relu" : 0, + "top" : "relu_MobilenetV1\/MobilenetV1\/Conv2d_0\/Relu6:0_clip", + "type" : "elementwise", + "name" : "relu_MobilenetV1\/MobilenetV1\/Conv2d_0\/Relu6:0_clip", + "beta" : 0 + }, + { + "alpha" : -1, + "bottom" : "relu_MobilenetV1\/MobilenetV1\/Conv2d_0\/Relu6:0_clip", + "weights" : { + + }, + "mode" : 6, + "top" : "MobilenetV1\/MobilenetV1\/Conv2d_0\/Relu6:0", + "type" : "activation", + "name" : "MobilenetV1\/MobilenetV1\/Conv2d_0\/Relu6:0", + "beta" : 0 + }, + { + "pad_r" : 0, + "fused_relu" : 1, + "fused_tanh" : 0, + "pad_fill_mode" : 0, + "pad_b" : 0, + "pad_l" : 0, + "top" : "relu_MobilenetV1\/MobilenetV1\/Conv2d_1_depthwise\/Relu6:0", + "blob_weights" : 7, + "K" : 32, + "blob_biases" : 5, + "name" : "MobilenetV1\/MobilenetV1\/Conv2d_1_depthwise\/depthwise:0", + "has_batch_norm" : 0, + "type" : "convolution", + "n_groups" : 32, + "pad_t" : 0, + "has_biases" : 1, + "C" : 32, + "bottom" : "MobilenetV1\/MobilenetV1\/Conv2d_0\/Relu6:0", + "weights" : { + + }, + "pad_mode" : 1, + "Nx" : 3, + "pad_value" : 0, + "Ny" : 3, + "n_parallel" : 1 + }, + { + "alpha" : -1, + "bottom" : "relu_MobilenetV1\/MobilenetV1\/Conv2d_1_depthwise\/Relu6:0", + "weights" : { + + }, + "mode" : 6, + "top" : "relu_MobilenetV1\/MobilenetV1\/Conv2d_1_depthwise\/Relu6:0_neg", + "type" : "activation", + "name" : "relu_MobilenetV1\/MobilenetV1\/Conv2d_1_depthwise\/Relu6:0_neg", + "beta" : 0 + }, + { + "alpha" : -6, + "bottom" : "relu_MobilenetV1\/MobilenetV1\/Conv2d_1_depthwise\/Relu6:0_neg", + "operation" : 25, + "weights" : { + + }, + "fused_relu" : 0, + "top" : "relu_MobilenetV1\/MobilenetV1\/Conv2d_1_depthwise\/Relu6:0_clip", + "type" : "elementwise", + "name" : "relu_MobilenetV1\/MobilenetV1\/Conv2d_1_depthwise\/Relu6:0_clip", + "beta" : 0 + }, + { + "alpha" : -1, + "bottom" : "relu_MobilenetV1\/MobilenetV1\/Conv2d_1_depthwise\/Relu6:0_clip", + "weights" : { + + }, + "mode" : 6, + "top" : "MobilenetV1\/MobilenetV1\/Conv2d_1_depthwise\/Relu6:0", + "type" : "activation", + "name" : "MobilenetV1\/MobilenetV1\/Conv2d_1_depthwise\/Relu6:0", + "beta" : 0 + }, + { + "pad_r" : 0, + "fused_relu" : 1, + "fused_tanh" : 0, + "pad_fill_mode" : 0, + "pad_b" : 0, + "pad_l" : 0, + "top" : "relu_MobilenetV1\/MobilenetV1\/Conv2d_1_pointwise\/Relu6:0", + "blob_weights" : 11, + "K" : 32, + "blob_biases" : 9, + "name" : "MobilenetV1\/MobilenetV1\/Conv2d_1_pointwise\/Conv2D:0", + "has_batch_norm" : 0, + "type" : "convolution", + "n_groups" : 1, + "pad_t" : 0, + "has_biases" : 1, + "C" : 64, + "bottom" : "MobilenetV1\/MobilenetV1\/Conv2d_1_depthwise\/Relu6:0", + "weights" : { + + }, + "pad_mode" : 1, + "Nx" : 1, + "pad_value" : 0, + "Ny" : 1, + "n_parallel" : 1 + }, + { + "alpha" : -1, + "bottom" : "relu_MobilenetV1\/MobilenetV1\/Conv2d_1_pointwise\/Relu6:0", + "weights" : { + + }, + "mode" : 6, + "top" : "relu_MobilenetV1\/MobilenetV1\/Conv2d_1_pointwise\/Relu6:0_neg", + "type" : "activation", + "name" : "relu_MobilenetV1\/MobilenetV1\/Conv2d_1_pointwise\/Relu6:0_neg", + "beta" : 0 + }, + { + "alpha" : -6, + "bottom" : "relu_MobilenetV1\/MobilenetV1\/Conv2d_1_pointwise\/Relu6:0_neg", + "operation" : 25, + "weights" : { + + }, + "fused_relu" : 0, + "top" : "relu_MobilenetV1\/MobilenetV1\/Conv2d_1_pointwise\/Relu6:0_clip", + "type" : "elementwise", + "name" : "relu_MobilenetV1\/MobilenetV1\/Conv2d_1_pointwise\/Relu6:0_clip", + "beta" : 0 + }, + { + "alpha" : -1, + "bottom" : "relu_MobilenetV1\/MobilenetV1\/Conv2d_1_pointwise\/Relu6:0_clip", + "weights" : { + + }, + "mode" : 6, + "top" : "MobilenetV1\/MobilenetV1\/Conv2d_1_pointwise\/Relu6:0", + "type" : "activation", + "name" : "MobilenetV1\/MobilenetV1\/Conv2d_1_pointwise\/Relu6:0", + "beta" : 0 + }, + { + "pad_r" : 0, + "fused_relu" : 1, + "fused_tanh" : 0, + "pad_fill_mode" : 0, + "pad_b" : 0, + "pad_l" : 0, + "top" : "relu_MobilenetV1\/MobilenetV1\/Conv2d_2_depthwise\/Relu6:0", + "blob_weights" : 15, + "K" : 64, + "blob_biases" : 13, + "stride_x" : 2, + "name" : "MobilenetV1\/MobilenetV1\/Conv2d_2_depthwise\/depthwise:0", + "has_batch_norm" : 0, + "type" : "convolution", + "n_groups" : 64, + "pad_t" : 0, + "stride_y" : 2, + "has_biases" : 1, + "C" : 64, + "bottom" : "MobilenetV1\/MobilenetV1\/Conv2d_1_pointwise\/Relu6:0", + "weights" : { + + }, + "Nx" : 3, + "pad_mode" : 1, + "pad_value" : 0, + "Ny" : 3, + "n_parallel" : 1 + }, + { + "alpha" : -1, + "bottom" : "relu_MobilenetV1\/MobilenetV1\/Conv2d_2_depthwise\/Relu6:0", + "weights" : { + + }, + "mode" : 6, + "top" : "relu_MobilenetV1\/MobilenetV1\/Conv2d_2_depthwise\/Relu6:0_neg", + "type" : "activation", + "name" : "relu_MobilenetV1\/MobilenetV1\/Conv2d_2_depthwise\/Relu6:0_neg", + "beta" : 0 + }, + { + "alpha" : -6, + "bottom" : "relu_MobilenetV1\/MobilenetV1\/Conv2d_2_depthwise\/Relu6:0_neg", + "operation" : 25, + "weights" : { + + }, + "fused_relu" : 0, + "top" : "relu_MobilenetV1\/MobilenetV1\/Conv2d_2_depthwise\/Relu6:0_clip", + "type" : "elementwise", + "name" : "relu_MobilenetV1\/MobilenetV1\/Conv2d_2_depthwise\/Relu6:0_clip", + "beta" : 0 + }, + { + "alpha" : -1, + "bottom" : "relu_MobilenetV1\/MobilenetV1\/Conv2d_2_depthwise\/Relu6:0_clip", + "weights" : { + + }, + "mode" : 6, + "top" : "MobilenetV1\/MobilenetV1\/Conv2d_2_depthwise\/Relu6:0", + "type" : "activation", + "name" : "MobilenetV1\/MobilenetV1\/Conv2d_2_depthwise\/Relu6:0", + "beta" : 0 + }, + { + "pad_r" : 0, + "fused_relu" : 1, + "fused_tanh" : 0, + "pad_fill_mode" : 0, + "pad_b" : 0, + "pad_l" : 0, + "top" : "relu_MobilenetV1\/MobilenetV1\/Conv2d_2_pointwise\/Relu6:0", + "blob_weights" : 19, + "K" : 64, + "blob_biases" : 17, + "name" : "MobilenetV1\/MobilenetV1\/Conv2d_2_pointwise\/Conv2D:0", + "has_batch_norm" : 0, + "type" : "convolution", + "n_groups" : 1, + "pad_t" : 0, + "has_biases" : 1, + "C" : 128, + "bottom" : "MobilenetV1\/MobilenetV1\/Conv2d_2_depthwise\/Relu6:0", + "weights" : { + + }, + "pad_mode" : 1, + "Nx" : 1, + "pad_value" : 0, + "Ny" : 1, + "n_parallel" : 1 + }, + { + "alpha" : -1, + "bottom" : "relu_MobilenetV1\/MobilenetV1\/Conv2d_2_pointwise\/Relu6:0", + "weights" : { + + }, + "mode" : 6, + "top" : "relu_MobilenetV1\/MobilenetV1\/Conv2d_2_pointwise\/Relu6:0_neg", + "type" : "activation", + "name" : "relu_MobilenetV1\/MobilenetV1\/Conv2d_2_pointwise\/Relu6:0_neg", + "beta" : 0 + }, + { + "alpha" : -6, + "bottom" : "relu_MobilenetV1\/MobilenetV1\/Conv2d_2_pointwise\/Relu6:0_neg", + "operation" : 25, + "weights" : { + + }, + "fused_relu" : 0, + "top" : "relu_MobilenetV1\/MobilenetV1\/Conv2d_2_pointwise\/Relu6:0_clip", + "type" : "elementwise", + "name" : "relu_MobilenetV1\/MobilenetV1\/Conv2d_2_pointwise\/Relu6:0_clip", + "beta" : 0 + }, + { + "alpha" : -1, + "bottom" : "relu_MobilenetV1\/MobilenetV1\/Conv2d_2_pointwise\/Relu6:0_clip", + "weights" : { + + }, + "mode" : 6, + "top" : "MobilenetV1\/MobilenetV1\/Conv2d_2_pointwise\/Relu6:0", + "type" : "activation", + "name" : "MobilenetV1\/MobilenetV1\/Conv2d_2_pointwise\/Relu6:0", + "beta" : 0 + }, + { + "pad_r" : 0, + "fused_relu" : 1, + "fused_tanh" : 0, + "pad_fill_mode" : 0, + "pad_b" : 0, + "pad_l" : 0, + "top" : "relu_MobilenetV1\/MobilenetV1\/Conv2d_3_depthwise\/Relu6:0", + "blob_weights" : 23, + "K" : 128, + "blob_biases" : 21, + "name" : "MobilenetV1\/MobilenetV1\/Conv2d_3_depthwise\/depthwise:0", + "has_batch_norm" : 0, + "type" : "convolution", + "n_groups" : 128, + "pad_t" : 0, + "has_biases" : 1, + "C" : 128, + "bottom" : "MobilenetV1\/MobilenetV1\/Conv2d_2_pointwise\/Relu6:0", + "weights" : { + + }, + "pad_mode" : 1, + "Nx" : 3, + "pad_value" : 0, + "Ny" : 3, + "n_parallel" : 1 + }, + { + "alpha" : -1, + "bottom" : "relu_MobilenetV1\/MobilenetV1\/Conv2d_3_depthwise\/Relu6:0", + "weights" : { + + }, + "mode" : 6, + "top" : "relu_MobilenetV1\/MobilenetV1\/Conv2d_3_depthwise\/Relu6:0_neg", + "type" : "activation", + "name" : "relu_MobilenetV1\/MobilenetV1\/Conv2d_3_depthwise\/Relu6:0_neg", + "beta" : 0 + }, + { + "alpha" : -6, + "bottom" : "relu_MobilenetV1\/MobilenetV1\/Conv2d_3_depthwise\/Relu6:0_neg", + "operation" : 25, + "weights" : { + + }, + "fused_relu" : 0, + "top" : "relu_MobilenetV1\/MobilenetV1\/Conv2d_3_depthwise\/Relu6:0_clip", + "type" : "elementwise", + "name" : "relu_MobilenetV1\/MobilenetV1\/Conv2d_3_depthwise\/Relu6:0_clip", + "beta" : 0 + }, + { + "alpha" : -1, + "bottom" : "relu_MobilenetV1\/MobilenetV1\/Conv2d_3_depthwise\/Relu6:0_clip", + "weights" : { + + }, + "mode" : 6, + "top" : "MobilenetV1\/MobilenetV1\/Conv2d_3_depthwise\/Relu6:0", + "type" : "activation", + "name" : "MobilenetV1\/MobilenetV1\/Conv2d_3_depthwise\/Relu6:0", + "beta" : 0 + }, + { + "pad_r" : 0, + "fused_relu" : 1, + "fused_tanh" : 0, + "pad_fill_mode" : 0, + "pad_b" : 0, + "pad_l" : 0, + "top" : "relu_MobilenetV1\/MobilenetV1\/Conv2d_3_pointwise\/Relu6:0", + "blob_weights" : 27, + "K" : 128, + "blob_biases" : 25, + "name" : "MobilenetV1\/MobilenetV1\/Conv2d_3_pointwise\/Conv2D:0", + "has_batch_norm" : 0, + "type" : "convolution", + "n_groups" : 1, + "pad_t" : 0, + "has_biases" : 1, + "C" : 128, + "bottom" : "MobilenetV1\/MobilenetV1\/Conv2d_3_depthwise\/Relu6:0", + "weights" : { + + }, + "pad_mode" : 1, + "Nx" : 1, + "pad_value" : 0, + "Ny" : 1, + "n_parallel" : 1 + }, + { + "alpha" : -1, + "bottom" : "relu_MobilenetV1\/MobilenetV1\/Conv2d_3_pointwise\/Relu6:0", + "weights" : { + + }, + "mode" : 6, + "top" : "relu_MobilenetV1\/MobilenetV1\/Conv2d_3_pointwise\/Relu6:0_neg", + "type" : "activation", + "name" : "relu_MobilenetV1\/MobilenetV1\/Conv2d_3_pointwise\/Relu6:0_neg", + "beta" : 0 + }, + { + "alpha" : -6, + "bottom" : "relu_MobilenetV1\/MobilenetV1\/Conv2d_3_pointwise\/Relu6:0_neg", + "operation" : 25, + "weights" : { + + }, + "fused_relu" : 0, + "top" : "relu_MobilenetV1\/MobilenetV1\/Conv2d_3_pointwise\/Relu6:0_clip", + "type" : "elementwise", + "name" : "relu_MobilenetV1\/MobilenetV1\/Conv2d_3_pointwise\/Relu6:0_clip", + "beta" : 0 + }, + { + "alpha" : -1, + "bottom" : "relu_MobilenetV1\/MobilenetV1\/Conv2d_3_pointwise\/Relu6:0_clip", + "weights" : { + + }, + "mode" : 6, + "top" : "MobilenetV1\/MobilenetV1\/Conv2d_3_pointwise\/Relu6:0", + "type" : "activation", + "name" : "MobilenetV1\/MobilenetV1\/Conv2d_3_pointwise\/Relu6:0", + "beta" : 0 + }, + { + "pad_r" : 0, + "fused_relu" : 1, + "fused_tanh" : 0, + "pad_fill_mode" : 0, + "pad_b" : 0, + "pad_l" : 0, + "top" : "relu_MobilenetV1\/MobilenetV1\/Conv2d_4_depthwise\/Relu6:0", + "blob_weights" : 31, + "K" : 128, + "blob_biases" : 29, + "stride_x" : 2, + "name" : "MobilenetV1\/MobilenetV1\/Conv2d_4_depthwise\/depthwise:0", + "has_batch_norm" : 0, + "type" : "convolution", + "n_groups" : 128, + "pad_t" : 0, + "stride_y" : 2, + "has_biases" : 1, + "C" : 128, + "bottom" : "MobilenetV1\/MobilenetV1\/Conv2d_3_pointwise\/Relu6:0", + "weights" : { + + }, + "Nx" : 3, + "pad_mode" : 1, + "pad_value" : 0, + "Ny" : 3, + "n_parallel" : 1 + }, + { + "alpha" : -1, + "bottom" : "relu_MobilenetV1\/MobilenetV1\/Conv2d_4_depthwise\/Relu6:0", + "weights" : { + + }, + "mode" : 6, + "top" : "relu_MobilenetV1\/MobilenetV1\/Conv2d_4_depthwise\/Relu6:0_neg", + "type" : "activation", + "name" : "relu_MobilenetV1\/MobilenetV1\/Conv2d_4_depthwise\/Relu6:0_neg", + "beta" : 0 + }, + { + "alpha" : -6, + "bottom" : "relu_MobilenetV1\/MobilenetV1\/Conv2d_4_depthwise\/Relu6:0_neg", + "operation" : 25, + "weights" : { + + }, + "fused_relu" : 0, + "top" : "relu_MobilenetV1\/MobilenetV1\/Conv2d_4_depthwise\/Relu6:0_clip", + "type" : "elementwise", + "name" : "relu_MobilenetV1\/MobilenetV1\/Conv2d_4_depthwise\/Relu6:0_clip", + "beta" : 0 + }, + { + "alpha" : -1, + "bottom" : "relu_MobilenetV1\/MobilenetV1\/Conv2d_4_depthwise\/Relu6:0_clip", + "weights" : { + + }, + "mode" : 6, + "top" : "MobilenetV1\/MobilenetV1\/Conv2d_4_depthwise\/Relu6:0", + "type" : "activation", + "name" : "MobilenetV1\/MobilenetV1\/Conv2d_4_depthwise\/Relu6:0", + "beta" : 0 + }, + { + "pad_r" : 0, + "fused_relu" : 1, + "fused_tanh" : 0, + "pad_fill_mode" : 0, + "pad_b" : 0, + "pad_l" : 0, + "top" : "relu_MobilenetV1\/MobilenetV1\/Conv2d_4_pointwise\/Relu6:0", + "blob_weights" : 35, + "K" : 128, + "blob_biases" : 33, + "name" : "MobilenetV1\/MobilenetV1\/Conv2d_4_pointwise\/Conv2D:0", + "has_batch_norm" : 0, + "type" : "convolution", + "n_groups" : 1, + "pad_t" : 0, + "has_biases" : 1, + "C" : 256, + "bottom" : "MobilenetV1\/MobilenetV1\/Conv2d_4_depthwise\/Relu6:0", + "weights" : { + + }, + "pad_mode" : 1, + "Nx" : 1, + "pad_value" : 0, + "Ny" : 1, + "n_parallel" : 1 + }, + { + "alpha" : -1, + "bottom" : "relu_MobilenetV1\/MobilenetV1\/Conv2d_4_pointwise\/Relu6:0", + "weights" : { + + }, + "mode" : 6, + "top" : "relu_MobilenetV1\/MobilenetV1\/Conv2d_4_pointwise\/Relu6:0_neg", + "type" : "activation", + "name" : "relu_MobilenetV1\/MobilenetV1\/Conv2d_4_pointwise\/Relu6:0_neg", + "beta" : 0 + }, + { + "alpha" : -6, + "bottom" : "relu_MobilenetV1\/MobilenetV1\/Conv2d_4_pointwise\/Relu6:0_neg", + "operation" : 25, + "weights" : { + + }, + "fused_relu" : 0, + "top" : "relu_MobilenetV1\/MobilenetV1\/Conv2d_4_pointwise\/Relu6:0_clip", + "type" : "elementwise", + "name" : "relu_MobilenetV1\/MobilenetV1\/Conv2d_4_pointwise\/Relu6:0_clip", + "beta" : 0 + }, + { + "alpha" : -1, + "bottom" : "relu_MobilenetV1\/MobilenetV1\/Conv2d_4_pointwise\/Relu6:0_clip", + "weights" : { + + }, + "mode" : 6, + "top" : "MobilenetV1\/MobilenetV1\/Conv2d_4_pointwise\/Relu6:0", + "type" : "activation", + "name" : "MobilenetV1\/MobilenetV1\/Conv2d_4_pointwise\/Relu6:0", + "beta" : 0 + }, + { + "pad_r" : 0, + "fused_relu" : 1, + "fused_tanh" : 0, + "pad_fill_mode" : 0, + "pad_b" : 0, + "pad_l" : 0, + "top" : "relu_MobilenetV1\/MobilenetV1\/Conv2d_5_depthwise\/Relu6:0", + "blob_weights" : 39, + "K" : 256, + "blob_biases" : 37, + "name" : "MobilenetV1\/MobilenetV1\/Conv2d_5_depthwise\/depthwise:0", + "has_batch_norm" : 0, + "type" : "convolution", + "n_groups" : 256, + "pad_t" : 0, + "has_biases" : 1, + "C" : 256, + "bottom" : "MobilenetV1\/MobilenetV1\/Conv2d_4_pointwise\/Relu6:0", + "weights" : { + + }, + "pad_mode" : 1, + "Nx" : 3, + "pad_value" : 0, + "Ny" : 3, + "n_parallel" : 1 + }, + { + "alpha" : -1, + "bottom" : "relu_MobilenetV1\/MobilenetV1\/Conv2d_5_depthwise\/Relu6:0", + "weights" : { + + }, + "mode" : 6, + "top" : "relu_MobilenetV1\/MobilenetV1\/Conv2d_5_depthwise\/Relu6:0_neg", + "type" : "activation", + "name" : "relu_MobilenetV1\/MobilenetV1\/Conv2d_5_depthwise\/Relu6:0_neg", + "beta" : 0 + }, + { + "alpha" : -6, + "bottom" : "relu_MobilenetV1\/MobilenetV1\/Conv2d_5_depthwise\/Relu6:0_neg", + "operation" : 25, + "weights" : { + + }, + "fused_relu" : 0, + "top" : "relu_MobilenetV1\/MobilenetV1\/Conv2d_5_depthwise\/Relu6:0_clip", + "type" : "elementwise", + "name" : "relu_MobilenetV1\/MobilenetV1\/Conv2d_5_depthwise\/Relu6:0_clip", + "beta" : 0 + }, + { + "alpha" : -1, + "bottom" : "relu_MobilenetV1\/MobilenetV1\/Conv2d_5_depthwise\/Relu6:0_clip", + "weights" : { + + }, + "mode" : 6, + "top" : "MobilenetV1\/MobilenetV1\/Conv2d_5_depthwise\/Relu6:0", + "type" : "activation", + "name" : "MobilenetV1\/MobilenetV1\/Conv2d_5_depthwise\/Relu6:0", + "beta" : 0 + }, + { + "pad_r" : 0, + "fused_relu" : 1, + "fused_tanh" : 0, + "pad_fill_mode" : 0, + "pad_b" : 0, + "pad_l" : 0, + "top" : "relu_MobilenetV1\/MobilenetV1\/Conv2d_5_pointwise\/Relu6:0", + "blob_weights" : 43, + "K" : 256, + "blob_biases" : 41, + "name" : "MobilenetV1\/MobilenetV1\/Conv2d_5_pointwise\/Conv2D:0", + "has_batch_norm" : 0, + "type" : "convolution", + "n_groups" : 1, + "pad_t" : 0, + "has_biases" : 1, + "C" : 256, + "bottom" : "MobilenetV1\/MobilenetV1\/Conv2d_5_depthwise\/Relu6:0", + "weights" : { + + }, + "pad_mode" : 1, + "Nx" : 1, + "pad_value" : 0, + "Ny" : 1, + "n_parallel" : 1 + }, + { + "alpha" : -1, + "bottom" : "relu_MobilenetV1\/MobilenetV1\/Conv2d_5_pointwise\/Relu6:0", + "weights" : { + + }, + "mode" : 6, + "top" : "relu_MobilenetV1\/MobilenetV1\/Conv2d_5_pointwise\/Relu6:0_neg", + "type" : "activation", + "name" : "relu_MobilenetV1\/MobilenetV1\/Conv2d_5_pointwise\/Relu6:0_neg", + "beta" : 0 + }, + { + "alpha" : -6, + "bottom" : "relu_MobilenetV1\/MobilenetV1\/Conv2d_5_pointwise\/Relu6:0_neg", + "operation" : 25, + "weights" : { + + }, + "fused_relu" : 0, + "top" : "relu_MobilenetV1\/MobilenetV1\/Conv2d_5_pointwise\/Relu6:0_clip", + "type" : "elementwise", + "name" : "relu_MobilenetV1\/MobilenetV1\/Conv2d_5_pointwise\/Relu6:0_clip", + "beta" : 0 + }, + { + "alpha" : -1, + "bottom" : "relu_MobilenetV1\/MobilenetV1\/Conv2d_5_pointwise\/Relu6:0_clip", + "weights" : { + + }, + "mode" : 6, + "top" : "MobilenetV1\/MobilenetV1\/Conv2d_5_pointwise\/Relu6:0", + "type" : "activation", + "name" : "MobilenetV1\/MobilenetV1\/Conv2d_5_pointwise\/Relu6:0", + "beta" : 0 + }, + { + "pad_r" : 0, + "fused_relu" : 1, + "fused_tanh" : 0, + "pad_fill_mode" : 0, + "pad_b" : 0, + "pad_l" : 0, + "top" : "relu_MobilenetV1\/MobilenetV1\/Conv2d_6_depthwise\/Relu6:0", + "blob_weights" : 47, + "K" : 256, + "blob_biases" : 45, + "stride_x" : 2, + "name" : "MobilenetV1\/MobilenetV1\/Conv2d_6_depthwise\/depthwise:0", + "has_batch_norm" : 0, + "type" : "convolution", + "n_groups" : 256, + "pad_t" : 0, + "stride_y" : 2, + "has_biases" : 1, + "C" : 256, + "bottom" : "MobilenetV1\/MobilenetV1\/Conv2d_5_pointwise\/Relu6:0", + "weights" : { + + }, + "Nx" : 3, + "pad_mode" : 1, + "pad_value" : 0, + "Ny" : 3, + "n_parallel" : 1 + }, + { + "alpha" : -1, + "bottom" : "relu_MobilenetV1\/MobilenetV1\/Conv2d_6_depthwise\/Relu6:0", + "weights" : { + + }, + "mode" : 6, + "top" : "relu_MobilenetV1\/MobilenetV1\/Conv2d_6_depthwise\/Relu6:0_neg", + "type" : "activation", + "name" : "relu_MobilenetV1\/MobilenetV1\/Conv2d_6_depthwise\/Relu6:0_neg", + "beta" : 0 + }, + { + "alpha" : -6, + "bottom" : "relu_MobilenetV1\/MobilenetV1\/Conv2d_6_depthwise\/Relu6:0_neg", + "operation" : 25, + "weights" : { + + }, + "fused_relu" : 0, + "top" : "relu_MobilenetV1\/MobilenetV1\/Conv2d_6_depthwise\/Relu6:0_clip", + "type" : "elementwise", + "name" : "relu_MobilenetV1\/MobilenetV1\/Conv2d_6_depthwise\/Relu6:0_clip", + "beta" : 0 + }, + { + "alpha" : -1, + "bottom" : "relu_MobilenetV1\/MobilenetV1\/Conv2d_6_depthwise\/Relu6:0_clip", + "weights" : { + + }, + "mode" : 6, + "top" : "MobilenetV1\/MobilenetV1\/Conv2d_6_depthwise\/Relu6:0", + "type" : "activation", + "name" : "MobilenetV1\/MobilenetV1\/Conv2d_6_depthwise\/Relu6:0", + "beta" : 0 + }, + { + "pad_r" : 0, + "fused_relu" : 1, + "fused_tanh" : 0, + "pad_fill_mode" : 0, + "pad_b" : 0, + "pad_l" : 0, + "top" : "relu_MobilenetV1\/MobilenetV1\/Conv2d_6_pointwise\/Relu6:0", + "blob_weights" : 51, + "K" : 256, + "blob_biases" : 49, + "name" : "MobilenetV1\/MobilenetV1\/Conv2d_6_pointwise\/Conv2D:0", + "has_batch_norm" : 0, + "type" : "convolution", + "n_groups" : 1, + "pad_t" : 0, + "has_biases" : 1, + "C" : 512, + "bottom" : "MobilenetV1\/MobilenetV1\/Conv2d_6_depthwise\/Relu6:0", + "weights" : { + + }, + "pad_mode" : 1, + "Nx" : 1, + "pad_value" : 0, + "Ny" : 1, + "n_parallel" : 1 + }, + { + "alpha" : -1, + "bottom" : "relu_MobilenetV1\/MobilenetV1\/Conv2d_6_pointwise\/Relu6:0", + "weights" : { + + }, + "mode" : 6, + "top" : "relu_MobilenetV1\/MobilenetV1\/Conv2d_6_pointwise\/Relu6:0_neg", + "type" : "activation", + "name" : "relu_MobilenetV1\/MobilenetV1\/Conv2d_6_pointwise\/Relu6:0_neg", + "beta" : 0 + }, + { + "alpha" : -6, + "bottom" : "relu_MobilenetV1\/MobilenetV1\/Conv2d_6_pointwise\/Relu6:0_neg", + "operation" : 25, + "weights" : { + + }, + "fused_relu" : 0, + "top" : "relu_MobilenetV1\/MobilenetV1\/Conv2d_6_pointwise\/Relu6:0_clip", + "type" : "elementwise", + "name" : "relu_MobilenetV1\/MobilenetV1\/Conv2d_6_pointwise\/Relu6:0_clip", + "beta" : 0 + }, + { + "alpha" : -1, + "bottom" : "relu_MobilenetV1\/MobilenetV1\/Conv2d_6_pointwise\/Relu6:0_clip", + "weights" : { + + }, + "mode" : 6, + "top" : "MobilenetV1\/MobilenetV1\/Conv2d_6_pointwise\/Relu6:0", + "type" : "activation", + "name" : "MobilenetV1\/MobilenetV1\/Conv2d_6_pointwise\/Relu6:0", + "beta" : 0 + }, + { + "pad_r" : 0, + "fused_relu" : 1, + "fused_tanh" : 0, + "pad_fill_mode" : 0, + "pad_b" : 0, + "pad_l" : 0, + "top" : "relu_MobilenetV1\/MobilenetV1\/Conv2d_7_depthwise\/Relu6:0", + "blob_weights" : 55, + "K" : 512, + "blob_biases" : 53, + "name" : "MobilenetV1\/MobilenetV1\/Conv2d_7_depthwise\/depthwise:0", + "has_batch_norm" : 0, + "type" : "convolution", + "n_groups" : 512, + "pad_t" : 0, + "has_biases" : 1, + "C" : 512, + "bottom" : "MobilenetV1\/MobilenetV1\/Conv2d_6_pointwise\/Relu6:0", + "weights" : { + + }, + "pad_mode" : 1, + "Nx" : 3, + "pad_value" : 0, + "Ny" : 3, + "n_parallel" : 1 + }, + { + "alpha" : -1, + "bottom" : "relu_MobilenetV1\/MobilenetV1\/Conv2d_7_depthwise\/Relu6:0", + "weights" : { + + }, + "mode" : 6, + "top" : "relu_MobilenetV1\/MobilenetV1\/Conv2d_7_depthwise\/Relu6:0_neg", + "type" : "activation", + "name" : "relu_MobilenetV1\/MobilenetV1\/Conv2d_7_depthwise\/Relu6:0_neg", + "beta" : 0 + }, + { + "alpha" : -6, + "bottom" : "relu_MobilenetV1\/MobilenetV1\/Conv2d_7_depthwise\/Relu6:0_neg", + "operation" : 25, + "weights" : { + + }, + "fused_relu" : 0, + "top" : "relu_MobilenetV1\/MobilenetV1\/Conv2d_7_depthwise\/Relu6:0_clip", + "type" : "elementwise", + "name" : "relu_MobilenetV1\/MobilenetV1\/Conv2d_7_depthwise\/Relu6:0_clip", + "beta" : 0 + }, + { + "alpha" : -1, + "bottom" : "relu_MobilenetV1\/MobilenetV1\/Conv2d_7_depthwise\/Relu6:0_clip", + "weights" : { + + }, + "mode" : 6, + "top" : "MobilenetV1\/MobilenetV1\/Conv2d_7_depthwise\/Relu6:0", + "type" : "activation", + "name" : "MobilenetV1\/MobilenetV1\/Conv2d_7_depthwise\/Relu6:0", + "beta" : 0 + }, + { + "pad_r" : 0, + "fused_relu" : 1, + "fused_tanh" : 0, + "pad_fill_mode" : 0, + "pad_b" : 0, + "pad_l" : 0, + "top" : "relu_MobilenetV1\/MobilenetV1\/Conv2d_7_pointwise\/Relu6:0", + "blob_weights" : 59, + "K" : 512, + "blob_biases" : 57, + "name" : "MobilenetV1\/MobilenetV1\/Conv2d_7_pointwise\/Conv2D:0", + "has_batch_norm" : 0, + "type" : "convolution", + "n_groups" : 1, + "pad_t" : 0, + "has_biases" : 1, + "C" : 512, + "bottom" : "MobilenetV1\/MobilenetV1\/Conv2d_7_depthwise\/Relu6:0", + "weights" : { + + }, + "pad_mode" : 1, + "Nx" : 1, + "pad_value" : 0, + "Ny" : 1, + "n_parallel" : 1 + }, + { + "alpha" : -1, + "bottom" : "relu_MobilenetV1\/MobilenetV1\/Conv2d_7_pointwise\/Relu6:0", + "weights" : { + + }, + "mode" : 6, + "top" : "relu_MobilenetV1\/MobilenetV1\/Conv2d_7_pointwise\/Relu6:0_neg", + "type" : "activation", + "name" : "relu_MobilenetV1\/MobilenetV1\/Conv2d_7_pointwise\/Relu6:0_neg", + "beta" : 0 + }, + { + "alpha" : -6, + "bottom" : "relu_MobilenetV1\/MobilenetV1\/Conv2d_7_pointwise\/Relu6:0_neg", + "operation" : 25, + "weights" : { + + }, + "fused_relu" : 0, + "top" : "relu_MobilenetV1\/MobilenetV1\/Conv2d_7_pointwise\/Relu6:0_clip", + "type" : "elementwise", + "name" : "relu_MobilenetV1\/MobilenetV1\/Conv2d_7_pointwise\/Relu6:0_clip", + "beta" : 0 + }, + { + "alpha" : -1, + "bottom" : "relu_MobilenetV1\/MobilenetV1\/Conv2d_7_pointwise\/Relu6:0_clip", + "weights" : { + + }, + "mode" : 6, + "top" : "MobilenetV1\/MobilenetV1\/Conv2d_7_pointwise\/Relu6:0", + "type" : "activation", + "name" : "MobilenetV1\/MobilenetV1\/Conv2d_7_pointwise\/Relu6:0", + "beta" : 0 + }, + { + "pad_r" : 0, + "fused_relu" : 1, + "fused_tanh" : 0, + "pad_fill_mode" : 0, + "pad_b" : 0, + "pad_l" : 0, + "top" : "relu_MobilenetV1\/MobilenetV1\/Conv2d_8_depthwise\/Relu6:0", + "blob_weights" : 63, + "K" : 512, + "blob_biases" : 61, + "name" : "MobilenetV1\/MobilenetV1\/Conv2d_8_depthwise\/depthwise:0", + "has_batch_norm" : 0, + "type" : "convolution", + "n_groups" : 512, + "pad_t" : 0, + "has_biases" : 1, + "C" : 512, + "bottom" : "MobilenetV1\/MobilenetV1\/Conv2d_7_pointwise\/Relu6:0", + "weights" : { + + }, + "pad_mode" : 1, + "Nx" : 3, + "pad_value" : 0, + "Ny" : 3, + "n_parallel" : 1 + }, + { + "alpha" : -1, + "bottom" : "relu_MobilenetV1\/MobilenetV1\/Conv2d_8_depthwise\/Relu6:0", + "weights" : { + + }, + "mode" : 6, + "top" : "relu_MobilenetV1\/MobilenetV1\/Conv2d_8_depthwise\/Relu6:0_neg", + "type" : "activation", + "name" : "relu_MobilenetV1\/MobilenetV1\/Conv2d_8_depthwise\/Relu6:0_neg", + "beta" : 0 + }, + { + "alpha" : -6, + "bottom" : "relu_MobilenetV1\/MobilenetV1\/Conv2d_8_depthwise\/Relu6:0_neg", + "operation" : 25, + "weights" : { + + }, + "fused_relu" : 0, + "top" : "relu_MobilenetV1\/MobilenetV1\/Conv2d_8_depthwise\/Relu6:0_clip", + "type" : "elementwise", + "name" : "relu_MobilenetV1\/MobilenetV1\/Conv2d_8_depthwise\/Relu6:0_clip", + "beta" : 0 + }, + { + "alpha" : -1, + "bottom" : "relu_MobilenetV1\/MobilenetV1\/Conv2d_8_depthwise\/Relu6:0_clip", + "weights" : { + + }, + "mode" : 6, + "top" : "MobilenetV1\/MobilenetV1\/Conv2d_8_depthwise\/Relu6:0", + "type" : "activation", + "name" : "MobilenetV1\/MobilenetV1\/Conv2d_8_depthwise\/Relu6:0", + "beta" : 0 + }, + { + "pad_r" : 0, + "fused_relu" : 1, + "fused_tanh" : 0, + "pad_fill_mode" : 0, + "pad_b" : 0, + "pad_l" : 0, + "top" : "relu_MobilenetV1\/MobilenetV1\/Conv2d_8_pointwise\/Relu6:0", + "blob_weights" : 67, + "K" : 512, + "blob_biases" : 65, + "name" : "MobilenetV1\/MobilenetV1\/Conv2d_8_pointwise\/Conv2D:0", + "has_batch_norm" : 0, + "type" : "convolution", + "n_groups" : 1, + "pad_t" : 0, + "has_biases" : 1, + "C" : 512, + "bottom" : "MobilenetV1\/MobilenetV1\/Conv2d_8_depthwise\/Relu6:0", + "weights" : { + + }, + "pad_mode" : 1, + "Nx" : 1, + "pad_value" : 0, + "Ny" : 1, + "n_parallel" : 1 + }, + { + "alpha" : -1, + "bottom" : "relu_MobilenetV1\/MobilenetV1\/Conv2d_8_pointwise\/Relu6:0", + "weights" : { + + }, + "mode" : 6, + "top" : "relu_MobilenetV1\/MobilenetV1\/Conv2d_8_pointwise\/Relu6:0_neg", + "type" : "activation", + "name" : "relu_MobilenetV1\/MobilenetV1\/Conv2d_8_pointwise\/Relu6:0_neg", + "beta" : 0 + }, + { + "alpha" : -6, + "bottom" : "relu_MobilenetV1\/MobilenetV1\/Conv2d_8_pointwise\/Relu6:0_neg", + "operation" : 25, + "weights" : { + + }, + "fused_relu" : 0, + "top" : "relu_MobilenetV1\/MobilenetV1\/Conv2d_8_pointwise\/Relu6:0_clip", + "type" : "elementwise", + "name" : "relu_MobilenetV1\/MobilenetV1\/Conv2d_8_pointwise\/Relu6:0_clip", + "beta" : 0 + }, + { + "alpha" : -1, + "bottom" : "relu_MobilenetV1\/MobilenetV1\/Conv2d_8_pointwise\/Relu6:0_clip", + "weights" : { + + }, + "mode" : 6, + "top" : "MobilenetV1\/MobilenetV1\/Conv2d_8_pointwise\/Relu6:0", + "type" : "activation", + "name" : "MobilenetV1\/MobilenetV1\/Conv2d_8_pointwise\/Relu6:0", + "beta" : 0 + }, + { + "pad_r" : 0, + "fused_relu" : 1, + "fused_tanh" : 0, + "pad_fill_mode" : 0, + "pad_b" : 0, + "pad_l" : 0, + "top" : "relu_MobilenetV1\/MobilenetV1\/Conv2d_9_depthwise\/Relu6:0", + "blob_weights" : 71, + "K" : 512, + "blob_biases" : 69, + "name" : "MobilenetV1\/MobilenetV1\/Conv2d_9_depthwise\/depthwise:0", + "has_batch_norm" : 0, + "type" : "convolution", + "n_groups" : 512, + "pad_t" : 0, + "has_biases" : 1, + "C" : 512, + "bottom" : "MobilenetV1\/MobilenetV1\/Conv2d_8_pointwise\/Relu6:0", + "weights" : { + + }, + "pad_mode" : 1, + "Nx" : 3, + "pad_value" : 0, + "Ny" : 3, + "n_parallel" : 1 + }, + { + "alpha" : -1, + "bottom" : "relu_MobilenetV1\/MobilenetV1\/Conv2d_9_depthwise\/Relu6:0", + "weights" : { + + }, + "mode" : 6, + "top" : "relu_MobilenetV1\/MobilenetV1\/Conv2d_9_depthwise\/Relu6:0_neg", + "type" : "activation", + "name" : "relu_MobilenetV1\/MobilenetV1\/Conv2d_9_depthwise\/Relu6:0_neg", + "beta" : 0 + }, + { + "alpha" : -6, + "bottom" : "relu_MobilenetV1\/MobilenetV1\/Conv2d_9_depthwise\/Relu6:0_neg", + "operation" : 25, + "weights" : { + + }, + "fused_relu" : 0, + "top" : "relu_MobilenetV1\/MobilenetV1\/Conv2d_9_depthwise\/Relu6:0_clip", + "type" : "elementwise", + "name" : "relu_MobilenetV1\/MobilenetV1\/Conv2d_9_depthwise\/Relu6:0_clip", + "beta" : 0 + }, + { + "alpha" : -1, + "bottom" : "relu_MobilenetV1\/MobilenetV1\/Conv2d_9_depthwise\/Relu6:0_clip", + "weights" : { + + }, + "mode" : 6, + "top" : "MobilenetV1\/MobilenetV1\/Conv2d_9_depthwise\/Relu6:0", + "type" : "activation", + "name" : "MobilenetV1\/MobilenetV1\/Conv2d_9_depthwise\/Relu6:0", + "beta" : 0 + }, + { + "pad_r" : 0, + "fused_relu" : 1, + "fused_tanh" : 0, + "pad_fill_mode" : 0, + "pad_b" : 0, + "pad_l" : 0, + "top" : "relu_MobilenetV1\/MobilenetV1\/Conv2d_9_pointwise\/Relu6:0", + "blob_weights" : 75, + "K" : 512, + "blob_biases" : 73, + "name" : "MobilenetV1\/MobilenetV1\/Conv2d_9_pointwise\/Conv2D:0", + "has_batch_norm" : 0, + "type" : "convolution", + "n_groups" : 1, + "pad_t" : 0, + "has_biases" : 1, + "C" : 512, + "bottom" : "MobilenetV1\/MobilenetV1\/Conv2d_9_depthwise\/Relu6:0", + "weights" : { + + }, + "pad_mode" : 1, + "Nx" : 1, + "pad_value" : 0, + "Ny" : 1, + "n_parallel" : 1 + }, + { + "alpha" : -1, + "bottom" : "relu_MobilenetV1\/MobilenetV1\/Conv2d_9_pointwise\/Relu6:0", + "weights" : { + + }, + "mode" : 6, + "top" : "relu_MobilenetV1\/MobilenetV1\/Conv2d_9_pointwise\/Relu6:0_neg", + "type" : "activation", + "name" : "relu_MobilenetV1\/MobilenetV1\/Conv2d_9_pointwise\/Relu6:0_neg", + "beta" : 0 + }, + { + "alpha" : -6, + "bottom" : "relu_MobilenetV1\/MobilenetV1\/Conv2d_9_pointwise\/Relu6:0_neg", + "operation" : 25, + "weights" : { + + }, + "fused_relu" : 0, + "top" : "relu_MobilenetV1\/MobilenetV1\/Conv2d_9_pointwise\/Relu6:0_clip", + "type" : "elementwise", + "name" : "relu_MobilenetV1\/MobilenetV1\/Conv2d_9_pointwise\/Relu6:0_clip", + "beta" : 0 + }, + { + "alpha" : -1, + "bottom" : "relu_MobilenetV1\/MobilenetV1\/Conv2d_9_pointwise\/Relu6:0_clip", + "weights" : { + + }, + "mode" : 6, + "top" : "MobilenetV1\/MobilenetV1\/Conv2d_9_pointwise\/Relu6:0", + "type" : "activation", + "name" : "MobilenetV1\/MobilenetV1\/Conv2d_9_pointwise\/Relu6:0", + "beta" : 0 + }, + { + "pad_r" : 0, + "fused_relu" : 1, + "fused_tanh" : 0, + "pad_fill_mode" : 0, + "pad_b" : 0, + "pad_l" : 0, + "top" : "relu_MobilenetV1\/MobilenetV1\/Conv2d_10_depthwise\/Relu6:0", + "blob_weights" : 79, + "K" : 512, + "blob_biases" : 77, + "name" : "MobilenetV1\/MobilenetV1\/Conv2d_10_depthwise\/depthwise:0", + "has_batch_norm" : 0, + "type" : "convolution", + "n_groups" : 512, + "pad_t" : 0, + "has_biases" : 1, + "C" : 512, + "bottom" : "MobilenetV1\/MobilenetV1\/Conv2d_9_pointwise\/Relu6:0", + "weights" : { + + }, + "pad_mode" : 1, + "Nx" : 3, + "pad_value" : 0, + "Ny" : 3, + "n_parallel" : 1 + }, + { + "alpha" : -1, + "bottom" : "relu_MobilenetV1\/MobilenetV1\/Conv2d_10_depthwise\/Relu6:0", + "weights" : { + + }, + "mode" : 6, + "top" : "relu_MobilenetV1\/MobilenetV1\/Conv2d_10_depthwise\/Relu6:0_neg", + "type" : "activation", + "name" : "relu_MobilenetV1\/MobilenetV1\/Conv2d_10_depthwise\/Relu6:0_neg", + "beta" : 0 + }, + { + "alpha" : -6, + "bottom" : "relu_MobilenetV1\/MobilenetV1\/Conv2d_10_depthwise\/Relu6:0_neg", + "operation" : 25, + "weights" : { + + }, + "fused_relu" : 0, + "top" : "relu_MobilenetV1\/MobilenetV1\/Conv2d_10_depthwise\/Relu6:0_clip", + "type" : "elementwise", + "name" : "relu_MobilenetV1\/MobilenetV1\/Conv2d_10_depthwise\/Relu6:0_clip", + "beta" : 0 + }, + { + "alpha" : -1, + "bottom" : "relu_MobilenetV1\/MobilenetV1\/Conv2d_10_depthwise\/Relu6:0_clip", + "weights" : { + + }, + "mode" : 6, + "top" : "MobilenetV1\/MobilenetV1\/Conv2d_10_depthwise\/Relu6:0", + "type" : "activation", + "name" : "MobilenetV1\/MobilenetV1\/Conv2d_10_depthwise\/Relu6:0", + "beta" : 0 + }, + { + "pad_r" : 0, + "fused_relu" : 1, + "fused_tanh" : 0, + "pad_fill_mode" : 0, + "pad_b" : 0, + "pad_l" : 0, + "top" : "relu_MobilenetV1\/MobilenetV1\/Conv2d_10_pointwise\/Relu6:0", + "blob_weights" : 83, + "K" : 512, + "blob_biases" : 81, + "name" : "MobilenetV1\/MobilenetV1\/Conv2d_10_pointwise\/Conv2D:0", + "has_batch_norm" : 0, + "type" : "convolution", + "n_groups" : 1, + "pad_t" : 0, + "has_biases" : 1, + "C" : 512, + "bottom" : "MobilenetV1\/MobilenetV1\/Conv2d_10_depthwise\/Relu6:0", + "weights" : { + + }, + "pad_mode" : 1, + "Nx" : 1, + "pad_value" : 0, + "Ny" : 1, + "n_parallel" : 1 + }, + { + "alpha" : -1, + "bottom" : "relu_MobilenetV1\/MobilenetV1\/Conv2d_10_pointwise\/Relu6:0", + "weights" : { + + }, + "mode" : 6, + "top" : "relu_MobilenetV1\/MobilenetV1\/Conv2d_10_pointwise\/Relu6:0_neg", + "type" : "activation", + "name" : "relu_MobilenetV1\/MobilenetV1\/Conv2d_10_pointwise\/Relu6:0_neg", + "beta" : 0 + }, + { + "alpha" : -6, + "bottom" : "relu_MobilenetV1\/MobilenetV1\/Conv2d_10_pointwise\/Relu6:0_neg", + "operation" : 25, + "weights" : { + + }, + "fused_relu" : 0, + "top" : "relu_MobilenetV1\/MobilenetV1\/Conv2d_10_pointwise\/Relu6:0_clip", + "type" : "elementwise", + "name" : "relu_MobilenetV1\/MobilenetV1\/Conv2d_10_pointwise\/Relu6:0_clip", + "beta" : 0 + }, + { + "alpha" : -1, + "bottom" : "relu_MobilenetV1\/MobilenetV1\/Conv2d_10_pointwise\/Relu6:0_clip", + "weights" : { + + }, + "mode" : 6, + "top" : "MobilenetV1\/MobilenetV1\/Conv2d_10_pointwise\/Relu6:0", + "type" : "activation", + "name" : "MobilenetV1\/MobilenetV1\/Conv2d_10_pointwise\/Relu6:0", + "beta" : 0 + }, + { + "pad_r" : 0, + "fused_relu" : 1, + "fused_tanh" : 0, + "pad_fill_mode" : 0, + "pad_b" : 0, + "pad_l" : 0, + "top" : "relu_MobilenetV1\/MobilenetV1\/Conv2d_11_depthwise\/Relu6:0", + "blob_weights" : 87, + "K" : 512, + "blob_biases" : 85, + "name" : "MobilenetV1\/MobilenetV1\/Conv2d_11_depthwise\/depthwise:0", + "has_batch_norm" : 0, + "type" : "convolution", + "n_groups" : 512, + "pad_t" : 0, + "has_biases" : 1, + "C" : 512, + "bottom" : "MobilenetV1\/MobilenetV1\/Conv2d_10_pointwise\/Relu6:0", + "weights" : { + + }, + "pad_mode" : 1, + "Nx" : 3, + "pad_value" : 0, + "Ny" : 3, + "n_parallel" : 1 + }, + { + "alpha" : -1, + "bottom" : "relu_MobilenetV1\/MobilenetV1\/Conv2d_11_depthwise\/Relu6:0", + "weights" : { + + }, + "mode" : 6, + "top" : "relu_MobilenetV1\/MobilenetV1\/Conv2d_11_depthwise\/Relu6:0_neg", + "type" : "activation", + "name" : "relu_MobilenetV1\/MobilenetV1\/Conv2d_11_depthwise\/Relu6:0_neg", + "beta" : 0 + }, + { + "alpha" : -6, + "bottom" : "relu_MobilenetV1\/MobilenetV1\/Conv2d_11_depthwise\/Relu6:0_neg", + "operation" : 25, + "weights" : { + + }, + "fused_relu" : 0, + "top" : "relu_MobilenetV1\/MobilenetV1\/Conv2d_11_depthwise\/Relu6:0_clip", + "type" : "elementwise", + "name" : "relu_MobilenetV1\/MobilenetV1\/Conv2d_11_depthwise\/Relu6:0_clip", + "beta" : 0 + }, + { + "alpha" : -1, + "bottom" : "relu_MobilenetV1\/MobilenetV1\/Conv2d_11_depthwise\/Relu6:0_clip", + "weights" : { + + }, + "mode" : 6, + "top" : "MobilenetV1\/MobilenetV1\/Conv2d_11_depthwise\/Relu6:0", + "type" : "activation", + "name" : "MobilenetV1\/MobilenetV1\/Conv2d_11_depthwise\/Relu6:0", + "beta" : 0 + }, + { + "pad_r" : 0, + "fused_relu" : 1, + "fused_tanh" : 0, + "pad_fill_mode" : 0, + "pad_b" : 0, + "pad_l" : 0, + "top" : "relu_MobilenetV1\/MobilenetV1\/Conv2d_11_pointwise\/Relu6:0", + "blob_weights" : 91, + "K" : 512, + "blob_biases" : 89, + "name" : "MobilenetV1\/MobilenetV1\/Conv2d_11_pointwise\/Conv2D:0", + "has_batch_norm" : 0, + "type" : "convolution", + "n_groups" : 1, + "pad_t" : 0, + "has_biases" : 1, + "C" : 512, + "bottom" : "MobilenetV1\/MobilenetV1\/Conv2d_11_depthwise\/Relu6:0", + "weights" : { + + }, + "pad_mode" : 1, + "Nx" : 1, + "pad_value" : 0, + "Ny" : 1, + "n_parallel" : 1 + }, + { + "alpha" : -1, + "bottom" : "relu_MobilenetV1\/MobilenetV1\/Conv2d_11_pointwise\/Relu6:0", + "weights" : { + + }, + "mode" : 6, + "top" : "relu_MobilenetV1\/MobilenetV1\/Conv2d_11_pointwise\/Relu6:0_neg", + "type" : "activation", + "name" : "relu_MobilenetV1\/MobilenetV1\/Conv2d_11_pointwise\/Relu6:0_neg", + "beta" : 0 + }, + { + "alpha" : -6, + "bottom" : "relu_MobilenetV1\/MobilenetV1\/Conv2d_11_pointwise\/Relu6:0_neg", + "operation" : 25, + "weights" : { + + }, + "fused_relu" : 0, + "top" : "relu_MobilenetV1\/MobilenetV1\/Conv2d_11_pointwise\/Relu6:0_clip", + "type" : "elementwise", + "name" : "relu_MobilenetV1\/MobilenetV1\/Conv2d_11_pointwise\/Relu6:0_clip", + "beta" : 0 + }, + { + "alpha" : -1, + "bottom" : "relu_MobilenetV1\/MobilenetV1\/Conv2d_11_pointwise\/Relu6:0_clip", + "weights" : { + + }, + "mode" : 6, + "top" : "MobilenetV1\/MobilenetV1\/Conv2d_11_pointwise\/Relu6:0", + "type" : "activation", + "name" : "MobilenetV1\/MobilenetV1\/Conv2d_11_pointwise\/Relu6:0", + "beta" : 0 + }, + { + "pad_r" : 0, + "fused_relu" : 1, + "fused_tanh" : 0, + "pad_fill_mode" : 0, + "pad_b" : 0, + "pad_l" : 0, + "top" : "relu_MobilenetV1\/MobilenetV1\/Conv2d_12_depthwise\/Relu6:0", + "blob_weights" : 95, + "K" : 512, + "blob_biases" : 93, + "stride_x" : 2, + "name" : "MobilenetV1\/MobilenetV1\/Conv2d_12_depthwise\/depthwise:0", + "has_batch_norm" : 0, + "type" : "convolution", + "n_groups" : 512, + "pad_t" : 0, + "stride_y" : 2, + "has_biases" : 1, + "C" : 512, + "bottom" : "MobilenetV1\/MobilenetV1\/Conv2d_11_pointwise\/Relu6:0", + "weights" : { + + }, + "Nx" : 3, + "pad_mode" : 1, + "pad_value" : 0, + "Ny" : 3, + "n_parallel" : 1 + }, + { + "alpha" : -1, + "bottom" : "relu_MobilenetV1\/MobilenetV1\/Conv2d_12_depthwise\/Relu6:0", + "weights" : { + + }, + "mode" : 6, + "top" : "relu_MobilenetV1\/MobilenetV1\/Conv2d_12_depthwise\/Relu6:0_neg", + "type" : "activation", + "name" : "relu_MobilenetV1\/MobilenetV1\/Conv2d_12_depthwise\/Relu6:0_neg", + "beta" : 0 + }, + { + "alpha" : -6, + "bottom" : "relu_MobilenetV1\/MobilenetV1\/Conv2d_12_depthwise\/Relu6:0_neg", + "operation" : 25, + "weights" : { + + }, + "fused_relu" : 0, + "top" : "relu_MobilenetV1\/MobilenetV1\/Conv2d_12_depthwise\/Relu6:0_clip", + "type" : "elementwise", + "name" : "relu_MobilenetV1\/MobilenetV1\/Conv2d_12_depthwise\/Relu6:0_clip", + "beta" : 0 + }, + { + "alpha" : -1, + "bottom" : "relu_MobilenetV1\/MobilenetV1\/Conv2d_12_depthwise\/Relu6:0_clip", + "weights" : { + + }, + "mode" : 6, + "top" : "MobilenetV1\/MobilenetV1\/Conv2d_12_depthwise\/Relu6:0", + "type" : "activation", + "name" : "MobilenetV1\/MobilenetV1\/Conv2d_12_depthwise\/Relu6:0", + "beta" : 0 + }, + { + "pad_r" : 0, + "fused_relu" : 1, + "fused_tanh" : 0, + "pad_fill_mode" : 0, + "pad_b" : 0, + "pad_l" : 0, + "top" : "relu_MobilenetV1\/MobilenetV1\/Conv2d_12_pointwise\/Relu6:0", + "blob_weights" : 99, + "K" : 512, + "blob_biases" : 97, + "name" : "MobilenetV1\/MobilenetV1\/Conv2d_12_pointwise\/Conv2D:0", + "has_batch_norm" : 0, + "type" : "convolution", + "n_groups" : 1, + "pad_t" : 0, + "has_biases" : 1, + "C" : 1024, + "bottom" : "MobilenetV1\/MobilenetV1\/Conv2d_12_depthwise\/Relu6:0", + "weights" : { + + }, + "pad_mode" : 1, + "Nx" : 1, + "pad_value" : 0, + "Ny" : 1, + "n_parallel" : 1 + }, + { + "alpha" : -1, + "bottom" : "relu_MobilenetV1\/MobilenetV1\/Conv2d_12_pointwise\/Relu6:0", + "weights" : { + + }, + "mode" : 6, + "top" : "relu_MobilenetV1\/MobilenetV1\/Conv2d_12_pointwise\/Relu6:0_neg", + "type" : "activation", + "name" : "relu_MobilenetV1\/MobilenetV1\/Conv2d_12_pointwise\/Relu6:0_neg", + "beta" : 0 + }, + { + "alpha" : -6, + "bottom" : "relu_MobilenetV1\/MobilenetV1\/Conv2d_12_pointwise\/Relu6:0_neg", + "operation" : 25, + "weights" : { + + }, + "fused_relu" : 0, + "top" : "relu_MobilenetV1\/MobilenetV1\/Conv2d_12_pointwise\/Relu6:0_clip", + "type" : "elementwise", + "name" : "relu_MobilenetV1\/MobilenetV1\/Conv2d_12_pointwise\/Relu6:0_clip", + "beta" : 0 + }, + { + "alpha" : -1, + "bottom" : "relu_MobilenetV1\/MobilenetV1\/Conv2d_12_pointwise\/Relu6:0_clip", + "weights" : { + + }, + "mode" : 6, + "top" : "MobilenetV1\/MobilenetV1\/Conv2d_12_pointwise\/Relu6:0", + "type" : "activation", + "name" : "MobilenetV1\/MobilenetV1\/Conv2d_12_pointwise\/Relu6:0", + "beta" : 0 + }, + { + "pad_r" : 0, + "fused_relu" : 1, + "fused_tanh" : 0, + "pad_fill_mode" : 0, + "pad_b" : 0, + "pad_l" : 0, + "top" : "relu_MobilenetV1\/MobilenetV1\/Conv2d_13_depthwise\/Relu6:0", + "blob_weights" : 103, + "K" : 1024, + "blob_biases" : 101, + "name" : "MobilenetV1\/MobilenetV1\/Conv2d_13_depthwise\/depthwise:0", + "has_batch_norm" : 0, + "type" : "convolution", + "n_groups" : 1024, + "pad_t" : 0, + "has_biases" : 1, + "C" : 1024, + "bottom" : "MobilenetV1\/MobilenetV1\/Conv2d_12_pointwise\/Relu6:0", + "weights" : { + + }, + "pad_mode" : 1, + "Nx" : 3, + "pad_value" : 0, + "Ny" : 3, + "n_parallel" : 1 + }, + { + "alpha" : -1, + "bottom" : "relu_MobilenetV1\/MobilenetV1\/Conv2d_13_depthwise\/Relu6:0", + "weights" : { + + }, + "mode" : 6, + "top" : "relu_MobilenetV1\/MobilenetV1\/Conv2d_13_depthwise\/Relu6:0_neg", + "type" : "activation", + "name" : "relu_MobilenetV1\/MobilenetV1\/Conv2d_13_depthwise\/Relu6:0_neg", + "beta" : 0 + }, + { + "alpha" : -6, + "bottom" : "relu_MobilenetV1\/MobilenetV1\/Conv2d_13_depthwise\/Relu6:0_neg", + "operation" : 25, + "weights" : { + + }, + "fused_relu" : 0, + "top" : "relu_MobilenetV1\/MobilenetV1\/Conv2d_13_depthwise\/Relu6:0_clip", + "type" : "elementwise", + "name" : "relu_MobilenetV1\/MobilenetV1\/Conv2d_13_depthwise\/Relu6:0_clip", + "beta" : 0 + }, + { + "alpha" : -1, + "bottom" : "relu_MobilenetV1\/MobilenetV1\/Conv2d_13_depthwise\/Relu6:0_clip", + "weights" : { + + }, + "mode" : 6, + "top" : "MobilenetV1\/MobilenetV1\/Conv2d_13_depthwise\/Relu6:0", + "type" : "activation", + "name" : "MobilenetV1\/MobilenetV1\/Conv2d_13_depthwise\/Relu6:0", + "beta" : 0 + }, + { + "pad_r" : 0, + "fused_relu" : 1, + "fused_tanh" : 0, + "pad_fill_mode" : 0, + "pad_b" : 0, + "pad_l" : 0, + "top" : "relu_MobilenetV1\/MobilenetV1\/Conv2d_13_pointwise\/Relu6:0", + "blob_weights" : 107, + "K" : 1024, + "blob_biases" : 105, + "name" : "MobilenetV1\/MobilenetV1\/Conv2d_13_pointwise\/Conv2D:0", + "has_batch_norm" : 0, + "type" : "convolution", + "n_groups" : 1, + "pad_t" : 0, + "has_biases" : 1, + "C" : 1024, + "bottom" : "MobilenetV1\/MobilenetV1\/Conv2d_13_depthwise\/Relu6:0", + "weights" : { + + }, + "pad_mode" : 1, + "Nx" : 1, + "pad_value" : 0, + "Ny" : 1, + "n_parallel" : 1 + }, + { + "alpha" : -1, + "bottom" : "relu_MobilenetV1\/MobilenetV1\/Conv2d_13_pointwise\/Relu6:0", + "weights" : { + + }, + "mode" : 6, + "top" : "relu_MobilenetV1\/MobilenetV1\/Conv2d_13_pointwise\/Relu6:0_neg", + "type" : "activation", + "name" : "relu_MobilenetV1\/MobilenetV1\/Conv2d_13_pointwise\/Relu6:0_neg", + "beta" : 0 + }, + { + "alpha" : -6, + "bottom" : "relu_MobilenetV1\/MobilenetV1\/Conv2d_13_pointwise\/Relu6:0_neg", + "operation" : 25, + "weights" : { + + }, + "fused_relu" : 0, + "top" : "relu_MobilenetV1\/MobilenetV1\/Conv2d_13_pointwise\/Relu6:0_clip", + "type" : "elementwise", + "name" : "relu_MobilenetV1\/MobilenetV1\/Conv2d_13_pointwise\/Relu6:0_clip", + "beta" : 0 + }, + { + "alpha" : -1, + "bottom" : "relu_MobilenetV1\/MobilenetV1\/Conv2d_13_pointwise\/Relu6:0_clip", + "weights" : { + + }, + "mode" : 6, + "top" : "MobilenetV1\/MobilenetV1\/Conv2d_13_pointwise\/Relu6:0", + "type" : "activation", + "name" : "MobilenetV1\/MobilenetV1\/Conv2d_13_pointwise\/Relu6:0", + "beta" : 0 + }, + { + "pad_value" : 0, + "average_count_exclude_padding" : 1, + "top_shape_style" : 0, + "avg_or_max" : 0, + "stride_x" : 2, + "pad_mode" : 2, + "stride_y" : 2, + "pad_t" : 0, + "weights" : { + + }, + "size_y" : 7, + "type" : "pool", + "pad_r" : 0, + "pad_b" : 0, + "size_x" : 7, + "pad_fill_mode" : 0, + "bottom" : "MobilenetV1\/MobilenetV1\/Conv2d_13_pointwise\/Relu6:0", + "pad_l" : 0, + "name" : "MobilenetV1\/Logits\/AvgPool_1a\/AvgPool:0", + "top" : "MobilenetV1\/Logits\/AvgPool_1a\/AvgPool:0" + }, + { + "pad_r" : 0, + "fused_relu" : 0, + "fused_tanh" : 0, + "pad_fill_mode" : 0, + "pad_b" : 0, + "pad_l" : 0, + "top" : "MobilenetV1\/Logits\/Conv2d_1c_1x1\/BiasAdd:0", + "blob_weights" : 111, + "K" : 1024, + "blob_biases" : 109, + "name" : "MobilenetV1\/Logits\/Conv2d_1c_1x1\/Conv2D:0", + "has_batch_norm" : 0, + "type" : "convolution", + "n_groups" : 1, + "pad_t" : 0, + "has_biases" : 1, + "C" : 1001, + "bottom" : "MobilenetV1\/Logits\/AvgPool_1a\/AvgPool:0", + "weights" : { + + }, + "pad_mode" : 1, + "Nx" : 1, + "pad_value" : 0, + "Ny" : 1, + "n_parallel" : 1 + }, + { + "nB" : 1001, + "top" : "final_retrain_ops\/Wx_plus_b\/add:0", + "has_biases" : 1, + "weights" : { + + }, + "nC" : 7, + "blob_weights" : 115, + "type" : "inner_product", + "has_relu" : 0, + "bottom" : "MobilenetV1\/Logits\/Conv2d_1c_1x1\/BiasAdd:0", + "blob_biases" : 113, + "has_tanh" : 0, + "name" : "final_retrain_ops\/Wx_plus_b\/MatMul", + "has_prelu" : 0 + }, + { + "C" : 2, + "weights" : { + + }, + "top" : "final_result__0", + "type" : "softmax", + "name" : "final_result:0", + "bottom" : "final_retrain_ops\/Wx_plus_b\/add:0" + } + ] +} \ No newline at end of file diff --git a/Playground/MLMOJI.playgroundbook/Contents/PrivateResources/EmojiSketches.mlmodelc/model.espresso.shape b/Playground/MLMOJI.playgroundbook/Contents/PrivateResources/EmojiSketches.mlmodelc/model.espresso.shape new file mode 100644 index 0000000..65c1500 --- /dev/null +++ b/Playground/MLMOJI.playgroundbook/Contents/PrivateResources/EmojiSketches.mlmodelc/model.espresso.shape @@ -0,0 +1,682 @@ +{ + "layer_shapes" : { + "relu_MobilenetV1\/MobilenetV1\/Conv2d_10_depthwise\/Relu6:0_neg" : { + "k" : 512, + "w" : 14, + "n" : 1, + "h" : 14 + }, + "relu_MobilenetV1\/MobilenetV1\/Conv2d_2_pointwise\/Relu6:0_neg" : { + "k" : 128, + "w" : 56, + "n" : 1, + "h" : 56 + }, + "relu_MobilenetV1\/MobilenetV1\/Conv2d_5_pointwise\/Relu6:0_clip" : { + "k" : 256, + "w" : 28, + "n" : 1, + "h" : 28 + }, + "MobilenetV1\/MobilenetV1\/Conv2d_5_depthwise\/Relu6:0" : { + "k" : 256, + "w" : 28, + "n" : 1, + "h" : 28 + }, + "relu_MobilenetV1\/MobilenetV1\/Conv2d_8_pointwise\/Relu6:0_clip" : { + "k" : 512, + "w" : 14, + "n" : 1, + "h" : 14 + }, + "MobilenetV1\/MobilenetV1\/Conv2d_9_pointwise\/Relu6:0" : { + "k" : 512, + "w" : 14, + "n" : 1, + "h" : 14 + }, + "relu_MobilenetV1\/MobilenetV1\/Conv2d_2_depthwise\/Relu6:0_clip" : { + "k" : 64, + "w" : 56, + "n" : 1, + "h" : 56 + }, + "relu_MobilenetV1\/MobilenetV1\/Conv2d_5_depthwise\/Relu6:0" : { + "k" : 256, + "w" : 28, + "n" : 1, + "h" : 28 + }, + "MobilenetV1\/MobilenetV1\/Conv2d_10_depthwise\/Relu6:0" : { + "k" : 512, + "w" : 14, + "n" : 1, + "h" : 14 + }, + "MobilenetV1\/MobilenetV1\/Conv2d_3_pointwise\/Relu6:0" : { + "k" : 128, + "w" : 56, + "n" : 1, + "h" : 56 + }, + "relu_MobilenetV1\/MobilenetV1\/Conv2d_5_depthwise\/Relu6:0_clip" : { + "k" : 256, + "w" : 28, + "n" : 1, + "h" : 28 + }, + "relu_MobilenetV1\/MobilenetV1\/Conv2d_11_pointwise\/Relu6:0_clip" : { + "k" : 512, + "w" : 14, + "n" : 1, + "h" : 14 + }, + "relu_MobilenetV1\/MobilenetV1\/Conv2d_2_pointwise\/Relu6:0" : { + "k" : 128, + "w" : 56, + "n" : 1, + "h" : 56 + }, + "relu_MobilenetV1\/MobilenetV1\/Conv2d_4_depthwise\/Relu6:0_neg" : { + "k" : 128, + "w" : 28, + "n" : 1, + "h" : 28 + }, + "MobilenetV1\/MobilenetV1\/Conv2d_6_pointwise\/Relu6:0" : { + "k" : 512, + "w" : 14, + "n" : 1, + "h" : 14 + }, + "relu_MobilenetV1\/MobilenetV1\/Conv2d_11_pointwise\/Relu6:0_neg" : { + "k" : 512, + "w" : 14, + "n" : 1, + "h" : 14 + }, + "input__0" : { + "k" : 3, + "w" : 224, + "n" : 1, + "h" : 224 + }, + "final_retrain_ops\/Wx_plus_b\/add:0" : { + "k" : 7, + "w" : 1, + "n" : 1, + "h" : 1 + }, + "relu_MobilenetV1\/MobilenetV1\/Conv2d_1_depthwise\/Relu6:0" : { + "k" : 32, + "w" : 112, + "n" : 1, + "h" : 112 + }, + "MobilenetV1\/MobilenetV1\/Conv2d_11_depthwise\/Relu6:0" : { + "k" : 512, + "w" : 14, + "n" : 1, + "h" : 14 + }, + "relu_MobilenetV1\/MobilenetV1\/Conv2d_7_pointwise\/Relu6:0_neg" : { + "k" : 512, + "w" : 14, + "n" : 1, + "h" : 14 + }, + "relu_MobilenetV1\/MobilenetV1\/Conv2d_11_pointwise\/Relu6:0" : { + "k" : 512, + "w" : 14, + "n" : 1, + "h" : 14 + }, + "relu_MobilenetV1\/MobilenetV1\/Conv2d_13_pointwise\/Relu6:0_neg" : { + "k" : 1024, + "w" : 7, + "n" : 1, + "h" : 7 + }, + "relu_MobilenetV1\/MobilenetV1\/Conv2d_4_pointwise\/Relu6:0_neg" : { + "k" : 256, + "w" : 28, + "n" : 1, + "h" : 28 + }, + "relu_MobilenetV1\/MobilenetV1\/Conv2d_3_pointwise\/Relu6:0_clip" : { + "k" : 128, + "w" : 56, + "n" : 1, + "h" : 56 + }, + "relu_MobilenetV1\/MobilenetV1\/Conv2d_11_depthwise\/Relu6:0_clip" : { + "k" : 512, + "w" : 14, + "n" : 1, + "h" : 14 + }, + "relu_MobilenetV1\/MobilenetV1\/Conv2d_0\/Relu6:0_clip" : { + "k" : 32, + "w" : 112, + "n" : 1, + "h" : 112 + }, + "relu_MobilenetV1\/MobilenetV1\/Conv2d_8_depthwise\/Relu6:0_clip" : { + "k" : 512, + "w" : 14, + "n" : 1, + "h" : 14 + }, + "relu_MobilenetV1\/MobilenetV1\/Conv2d_7_pointwise\/Relu6:0" : { + "k" : 512, + "w" : 14, + "n" : 1, + "h" : 14 + }, + "MobilenetV1\/MobilenetV1\/Conv2d_0\/Relu6:0" : { + "k" : 32, + "w" : 112, + "n" : 1, + "h" : 112 + }, + "MobilenetV1\/MobilenetV1\/Conv2d_7_depthwise\/Relu6:0" : { + "k" : 512, + "w" : 14, + "n" : 1, + "h" : 14 + }, + "relu_MobilenetV1\/MobilenetV1\/Conv2d_10_depthwise\/Relu6:0" : { + "k" : 512, + "w" : 14, + "n" : 1, + "h" : 14 + }, + "relu_MobilenetV1\/MobilenetV1\/Conv2d_9_depthwise\/Relu6:0_neg" : { + "k" : 512, + "w" : 14, + "n" : 1, + "h" : 14 + }, + "relu_MobilenetV1\/MobilenetV1\/Conv2d_1_depthwise\/Relu6:0_neg" : { + "k" : 32, + "w" : 112, + "n" : 1, + "h" : 112 + }, + "MobilenetV1\/MobilenetV1\/Conv2d_1_depthwise\/Relu6:0" : { + "k" : 32, + "w" : 112, + "n" : 1, + "h" : 112 + }, + "MobilenetV1\/MobilenetV1\/Conv2d_12_depthwise\/Relu6:0" : { + "k" : 512, + "w" : 7, + "n" : 1, + "h" : 7 + }, + "relu_MobilenetV1\/MobilenetV1\/Conv2d_6_depthwise\/Relu6:0_neg" : { + "k" : 256, + "w" : 14, + "n" : 1, + "h" : 14 + }, + "relu_MobilenetV1\/MobilenetV1\/Conv2d_10_pointwise\/Relu6:0_clip" : { + "k" : 512, + "w" : 14, + "n" : 1, + "h" : 14 + }, + "relu_MobilenetV1\/MobilenetV1\/Conv2d_6_depthwise\/Relu6:0" : { + "k" : 256, + "w" : 14, + "n" : 1, + "h" : 14 + }, + "MobilenetV1\/MobilenetV1\/Conv2d_10_pointwise\/Relu6:0" : { + "k" : 512, + "w" : 14, + "n" : 1, + "h" : 14 + }, + "final_result__0" : { + "k" : 7, + "w" : 1, + "n" : 1, + "h" : 1 + }, + "MobilenetV1\/MobilenetV1\/Conv2d_4_depthwise\/Relu6:0" : { + "k" : 128, + "w" : 28, + "n" : 1, + "h" : 28 + }, + "relu_MobilenetV1\/MobilenetV1\/Conv2d_1_pointwise\/Relu6:0_neg" : { + "k" : 64, + "w" : 112, + "n" : 1, + "h" : 112 + }, + "relu_MobilenetV1\/MobilenetV1\/Conv2d_9_pointwise\/Relu6:0_neg" : { + "k" : 512, + "w" : 14, + "n" : 1, + "h" : 14 + }, + "relu_MobilenetV1\/MobilenetV1\/Conv2d_6_pointwise\/Relu6:0_clip" : { + "k" : 512, + "w" : 14, + "n" : 1, + "h" : 14 + }, + "MobilenetV1\/MobilenetV1\/Conv2d_8_pointwise\/Relu6:0" : { + "k" : 512, + "w" : 14, + "n" : 1, + "h" : 14 + }, + "relu_MobilenetV1\/MobilenetV1\/Conv2d_3_pointwise\/Relu6:0" : { + "k" : 128, + "w" : 56, + "n" : 1, + "h" : 56 + }, + "MobilenetV1\/MobilenetV1\/Conv2d_2_pointwise\/Relu6:0" : { + "k" : 128, + "w" : 56, + "n" : 1, + "h" : 56 + }, + "MobilenetV1\/MobilenetV1\/Conv2d_13_depthwise\/Relu6:0" : { + "k" : 1024, + "w" : 7, + "n" : 1, + "h" : 7 + }, + "relu_MobilenetV1\/MobilenetV1\/Conv2d_3_depthwise\/Relu6:0_clip" : { + "k" : 128, + "w" : 56, + "n" : 1, + "h" : 56 + }, + "relu_MobilenetV1\/MobilenetV1\/Conv2d_13_pointwise\/Relu6:0_clip" : { + "k" : 1024, + "w" : 7, + "n" : 1, + "h" : 7 + }, + "relu_MobilenetV1\/MobilenetV1\/Conv2d_6_depthwise\/Relu6:0_clip" : { + "k" : 256, + "w" : 14, + "n" : 1, + "h" : 14 + }, + "relu_MobilenetV1\/MobilenetV1\/Conv2d_2_depthwise\/Relu6:0" : { + "k" : 64, + "w" : 56, + "n" : 1, + "h" : 56 + }, + "MobilenetV1\/MobilenetV1\/Conv2d_11_pointwise\/Relu6:0" : { + "k" : 512, + "w" : 14, + "n" : 1, + "h" : 14 + }, + "relu_MobilenetV1\/MobilenetV1\/Conv2d_7_depthwise\/Relu6:0" : { + "k" : 512, + "w" : 14, + "n" : 1, + "h" : 14 + }, + "MobilenetV1\/MobilenetV1\/Conv2d_5_pointwise\/Relu6:0" : { + "k" : 256, + "w" : 28, + "n" : 1, + "h" : 28 + }, + "relu_MobilenetV1\/MobilenetV1\/Conv2d_13_depthwise\/Relu6:0" : { + "k" : 1024, + "w" : 7, + "n" : 1, + "h" : 7 + }, + "relu_MobilenetV1\/MobilenetV1\/Conv2d_3_depthwise\/Relu6:0_neg" : { + "k" : 128, + "w" : 56, + "n" : 1, + "h" : 56 + }, + "relu_MobilenetV1\/MobilenetV1\/Conv2d_10_pointwise\/Relu6:0" : { + "k" : 512, + "w" : 14, + "n" : 1, + "h" : 14 + }, + "relu_MobilenetV1\/MobilenetV1\/Conv2d_11_depthwise\/Relu6:0_neg" : { + "k" : 512, + "w" : 14, + "n" : 1, + "h" : 14 + }, + "MobilenetV1\/MobilenetV1\/Conv2d_13_pointwise\/Relu6:0" : { + "k" : 1024, + "w" : 7, + "n" : 1, + "h" : 7 + }, + "relu_MobilenetV1\/MobilenetV1\/Conv2d_8_pointwise\/Relu6:0" : { + "k" : 512, + "w" : 14, + "n" : 1, + "h" : 14 + }, + "relu_MobilenetV1\/MobilenetV1\/Conv2d_6_pointwise\/Relu6:0_neg" : { + "k" : 512, + "w" : 14, + "n" : 1, + "h" : 14 + }, + "relu_MobilenetV1\/MobilenetV1\/Conv2d_9_pointwise\/Relu6:0_clip" : { + "k" : 512, + "w" : 14, + "n" : 1, + "h" : 14 + }, + "relu_MobilenetV1\/MobilenetV1\/Conv2d_1_pointwise\/Relu6:0_clip" : { + "k" : 64, + "w" : 112, + "n" : 1, + "h" : 112 + }, + "MobilenetV1\/MobilenetV1\/Conv2d_12_pointwise\/Relu6:0" : { + "k" : 1024, + "w" : 7, + "n" : 1, + "h" : 7 + }, + "relu_MobilenetV1\/MobilenetV1\/Conv2d_13_depthwise\/Relu6:0_neg" : { + "k" : 1024, + "w" : 7, + "n" : 1, + "h" : 7 + }, + "relu_MobilenetV1\/MobilenetV1\/Conv2d_3_depthwise\/Relu6:0" : { + "k" : 128, + "w" : 56, + "n" : 1, + "h" : 56 + }, + "relu_MobilenetV1\/MobilenetV1\/Conv2d_9_depthwise\/Relu6:0_clip" : { + "k" : 512, + "w" : 14, + "n" : 1, + "h" : 14 + }, + "relu_MobilenetV1\/MobilenetV1\/Conv2d_1_depthwise\/Relu6:0_clip" : { + "k" : 32, + "w" : 112, + "n" : 1, + "h" : 112 + }, + "MobilenetV1\/MobilenetV1\/Conv2d_6_depthwise\/Relu6:0" : { + "k" : 256, + "w" : 14, + "n" : 1, + "h" : 14 + }, + "relu_MobilenetV1\/MobilenetV1\/Conv2d_10_depthwise\/Relu6:0_clip" : { + "k" : 512, + "w" : 14, + "n" : 1, + "h" : 14 + }, + "relu_MobilenetV1\/MobilenetV1\/Conv2d_4_pointwise\/Relu6:0" : { + "k" : 256, + "w" : 28, + "n" : 1, + "h" : 28 + }, + "relu_MobilenetV1\/MobilenetV1\/Conv2d_8_depthwise\/Relu6:0_neg" : { + "k" : 512, + "w" : 14, + "n" : 1, + "h" : 14 + }, + "relu_MobilenetV1\/MobilenetV1\/Conv2d_9_pointwise\/Relu6:0" : { + "k" : 512, + "w" : 14, + "n" : 1, + "h" : 14 + }, + "MobilenetV1\/MobilenetV1\/Conv2d_9_depthwise\/Relu6:0" : { + "k" : 512, + "w" : 14, + "n" : 1, + "h" : 14 + }, + "MobilenetV1\/Logits\/AvgPool_1a\/AvgPool:0" : { + "k" : 1024, + "w" : 1, + "n" : 1, + "h" : 1 + }, + "relu_MobilenetV1\/MobilenetV1\/Conv2d_10_pointwise\/Relu6:0_neg" : { + "k" : 512, + "w" : 14, + "n" : 1, + "h" : 14 + }, + "MobilenetV1\/MobilenetV1\/Conv2d_3_depthwise\/Relu6:0" : { + "k" : 128, + "w" : 56, + "n" : 1, + "h" : 56 + }, + "relu_MobilenetV1\/MobilenetV1\/Conv2d_3_pointwise\/Relu6:0_neg" : { + "k" : 128, + "w" : 56, + "n" : 1, + "h" : 56 + }, + "relu_MobilenetV1\/MobilenetV1\/Conv2d_4_pointwise\/Relu6:0_clip" : { + "k" : 256, + "w" : 28, + "n" : 1, + "h" : 28 + }, + "relu_MobilenetV1\/MobilenetV1\/Conv2d_8_depthwise\/Relu6:0" : { + "k" : 512, + "w" : 14, + "n" : 1, + "h" : 14 + }, + "MobilenetV1\/MobilenetV1\/Conv2d_1_pointwise\/Relu6:0" : { + "k" : 64, + "w" : 112, + "n" : 1, + "h" : 112 + }, + "relu_MobilenetV1\/MobilenetV1\/Conv2d_13_depthwise\/Relu6:0_clip" : { + "k" : 1024, + "w" : 7, + "n" : 1, + "h" : 7 + }, + "relu_MobilenetV1\/MobilenetV1\/Conv2d_7_pointwise\/Relu6:0_clip" : { + "k" : 512, + "w" : 14, + "n" : 1, + "h" : 14 + }, + "relu_MobilenetV1\/MobilenetV1\/Conv2d_12_depthwise\/Relu6:0" : { + "k" : 512, + "w" : 7, + "n" : 1, + "h" : 7 + }, + "relu_MobilenetV1\/MobilenetV1\/Conv2d_12_pointwise\/Relu6:0_neg" : { + "k" : 1024, + "w" : 7, + "n" : 1, + "h" : 7 + }, + "relu_MobilenetV1\/MobilenetV1\/Conv2d_5_pointwise\/Relu6:0" : { + "k" : 256, + "w" : 28, + "n" : 1, + "h" : 28 + }, + "relu_MobilenetV1\/MobilenetV1\/Conv2d_4_depthwise\/Relu6:0_clip" : { + "k" : 128, + "w" : 28, + "n" : 1, + "h" : 28 + }, + "relu_MobilenetV1\/MobilenetV1\/Conv2d_13_pointwise\/Relu6:0" : { + "k" : 1024, + "w" : 7, + "n" : 1, + "h" : 7 + }, + "MobilenetV1\/MobilenetV1\/Conv2d_4_pointwise\/Relu6:0" : { + "k" : 256, + "w" : 28, + "n" : 1, + "h" : 28 + }, + "relu_MobilenetV1\/MobilenetV1\/Conv2d_5_depthwise\/Relu6:0_neg" : { + "k" : 256, + "w" : 28, + "n" : 1, + "h" : 28 + }, + "relu_MobilenetV1\/MobilenetV1\/Conv2d_12_pointwise\/Relu6:0_clip" : { + "k" : 1024, + "w" : 7, + "n" : 1, + "h" : 7 + }, + "relu_MobilenetV1\/MobilenetV1\/Conv2d_4_depthwise\/Relu6:0" : { + "k" : 128, + "w" : 28, + "n" : 1, + "h" : 28 + }, + "relu_MobilenetV1\/MobilenetV1\/Conv2d_2_depthwise\/Relu6:0_neg" : { + "k" : 64, + "w" : 56, + "n" : 1, + "h" : 56 + }, + "MobilenetV1\/MobilenetV1\/Conv2d_7_pointwise\/Relu6:0" : { + "k" : 512, + "w" : 14, + "n" : 1, + "h" : 14 + }, + "relu_MobilenetV1\/MobilenetV1\/Conv2d_8_pointwise\/Relu6:0_neg" : { + "k" : 512, + "w" : 14, + "n" : 1, + "h" : 14 + }, + "relu_MobilenetV1\/MobilenetV1\/Conv2d_5_pointwise\/Relu6:0_neg" : { + "k" : 256, + "w" : 28, + "n" : 1, + "h" : 28 + }, + "relu_MobilenetV1\/MobilenetV1\/Conv2d_2_pointwise\/Relu6:0_clip" : { + "k" : 128, + "w" : 56, + "n" : 1, + "h" : 56 + }, + "relu_MobilenetV1\/MobilenetV1\/Conv2d_0\/Relu6:0" : { + "k" : 32, + "w" : 112, + "n" : 1, + "h" : 112 + }, + "MobilenetV1\/Logits\/Conv2d_1c_1x1\/BiasAdd:0" : { + "k" : 1001, + "w" : 1, + "n" : 1, + "h" : 1 + }, + "relu_MobilenetV1\/MobilenetV1\/Conv2d_7_depthwise\/Relu6:0_clip" : { + "k" : 512, + "w" : 14, + "n" : 1, + "h" : 14 + }, + "relu_MobilenetV1\/MobilenetV1\/Conv2d_12_pointwise\/Relu6:0" : { + "k" : 1024, + "w" : 7, + "n" : 1, + "h" : 7 + }, + "relu_MobilenetV1\/MobilenetV1\/Conv2d_12_depthwise\/Relu6:0_neg" : { + "k" : 512, + "w" : 7, + "n" : 1, + "h" : 7 + }, + "relu_MobilenetV1\/MobilenetV1\/Conv2d_9_depthwise\/Relu6:0" : { + "k" : 512, + "w" : 14, + "n" : 1, + "h" : 14 + }, + "MobilenetV1\/MobilenetV1\/Conv2d_8_depthwise\/Relu6:0" : { + "k" : 512, + "w" : 14, + "n" : 1, + "h" : 14 + }, + "relu_MobilenetV1\/MobilenetV1\/Conv2d_7_depthwise\/Relu6:0_neg" : { + "k" : 512, + "w" : 14, + "n" : 1, + "h" : 14 + }, + "relu_MobilenetV1\/MobilenetV1\/Conv2d_12_depthwise\/Relu6:0_clip" : { + "k" : 512, + "w" : 7, + "n" : 1, + "h" : 7 + }, + "MobilenetV1\/MobilenetV1\/Conv2d_2_depthwise\/Relu6:0" : { + "k" : 64, + "w" : 56, + "n" : 1, + "h" : 56 + }, + "relu_MobilenetV1\/MobilenetV1\/Conv2d_1_pointwise\/Relu6:0" : { + "k" : 64, + "w" : 112, + "n" : 1, + "h" : 112 + }, + "relu_MobilenetV1\/MobilenetV1\/Conv2d_6_pointwise\/Relu6:0" : { + "k" : 512, + "w" : 14, + "n" : 1, + "h" : 14 + }, + "relu_MobilenetV1\/MobilenetV1\/Conv2d_11_depthwise\/Relu6:0" : { + "k" : 512, + "w" : 14, + "n" : 1, + "h" : 14 + }, + "relu_MobilenetV1\/MobilenetV1\/Conv2d_0\/Relu6:0_neg" : { + "k" : 32, + "w" : 112, + "n" : 1, + "h" : 112 + } + } +} \ No newline at end of file diff --git a/Playground/MLMOJI.playgroundbook/Contents/PrivateResources/EmojiSketches.mlmodelc/model.espresso.weights b/Playground/MLMOJI.playgroundbook/Contents/PrivateResources/EmojiSketches.mlmodelc/model.espresso.weights new file mode 100644 index 0000000..e0c9a0c Binary files /dev/null and b/Playground/MLMOJI.playgroundbook/Contents/PrivateResources/EmojiSketches.mlmodelc/model.espresso.weights differ diff --git a/Playground/MLMOJI.playgroundbook/Contents/PrivateResources/EmojiSketches.mlmodelc/model/coremldata.bin b/Playground/MLMOJI.playgroundbook/Contents/PrivateResources/EmojiSketches.mlmodelc/model/coremldata.bin new file mode 100644 index 0000000..e7e00c0 Binary files /dev/null and b/Playground/MLMOJI.playgroundbook/Contents/PrivateResources/EmojiSketches.mlmodelc/model/coremldata.bin differ diff --git a/Playground/MLMOJI.playgroundbook/Contents/PrivateResources/Glossary.plist b/Playground/MLMOJI.playgroundbook/Contents/PrivateResources/Glossary.plist new file mode 100644 index 0000000..dfbbace --- /dev/null +++ b/Playground/MLMOJI.playgroundbook/Contents/PrivateResources/Glossary.plist @@ -0,0 +1,57 @@ + + + + + Terms + + classifier + + Title + classifier + Definition + A classifier is a type of neural network that identifies the sub-categories to which an item belongs to. It is trained using items whose category is already known. They are not limitated to images, and can also be used for text or audio processing. + +For example, it can be used to tag an e-mail as spam, or to tell if a picture of food is a hot dog or not. + FirstUse + + PageReference + MLMOJI/Building a Data Set + Title + Building a Data Set + + + feature + + Title + feature + Definition + A feature (or pattern) is an individual property of an item being observed. When training a deep neural network, we are in fact training the model to learn the distinctive features of each prediction class. + +In image recognition, features can include edges, colors and shapes. + FirstUse + + PageReference + MLMOJI/Building a Data Set + Title + Building a Data Set + + + dnn + + Title + deep neural network + Definition + A neural network is a set of mathematical operations organized as layers. The operations are applied to the input input inside “hidden“ layers. The last layer produces the output. + +A deep neural network is a neural network that contains multiple hidden layers. Therefore, it applies multiple transforms to the input before producing the output. It is used for complex problems, like image classification. + FirstUse + + PageReference + MLMOJI/Introducing MLMOJI + Title + Introducing MLMOJI + + + + + diff --git a/Playground/MLMOJI.playgroundbook/Contents/PrivateResources/KeyboardClear@2x.png b/Playground/MLMOJI.playgroundbook/Contents/PrivateResources/KeyboardClear@2x.png new file mode 100644 index 0000000..58497ec Binary files /dev/null and b/Playground/MLMOJI.playgroundbook/Contents/PrivateResources/KeyboardClear@2x.png differ diff --git a/Playground/MLMOJI.playgroundbook/Contents/Sources/Augmentation/AugmentationFilter.swift b/Playground/MLMOJI.playgroundbook/Contents/Sources/Augmentation/AugmentationFilter.swift new file mode 100644 index 0000000..a52f430 --- /dev/null +++ b/Playground/MLMOJI.playgroundbook/Contents/Sources/Augmentation/AugmentationFilter.swift @@ -0,0 +1,29 @@ +// +// MLMOJI +// +// This file is part of Alexis Aubry's WWDC18 scolarship submission open source project. +// Copyright © 2018 Alexis Aubry. Available under the terms of the MIT License. +// + +import CoreGraphics +import CoreImage + +/** + * The filters that can be performed in a data set augmentation session. + */ + +public enum AugmentationFilter { + + /// Rotates the image by the specified angle. + case rotate(CGFloat) + + /// Translates the image to the given point on the screen. + case translate(CGFloat, CGFloat) + + /// Applies + case scale(CGFloat) + + /// Blurs the image. + case blur + +} diff --git a/Playground/MLMOJI.playgroundbook/Contents/Sources/Augmentation/AugmentationSession.swift b/Playground/MLMOJI.playgroundbook/Contents/Sources/Augmentation/AugmentationSession.swift new file mode 100644 index 0000000..e504832 --- /dev/null +++ b/Playground/MLMOJI.playgroundbook/Contents/Sources/Augmentation/AugmentationSession.swift @@ -0,0 +1,189 @@ +// +// MLMOJI +// +// This file is part of Alexis Aubry's WWDC18 scolarship submission open source project. +// Copyright © 2018 Alexis Aubry. Available under the terms of the MIT License. +// + +import UIKit + +/** + * The delegate of an image augmentation session. + */ + +public protocol AugmentationSessionDelegate: class { + + /** + * Called when the augmentation session added an image. + */ + + func augmentationSession(_ session: AugmentationSession, didCreateImage image: UIImage) + + /** + * Called when the augmentation session finishes. + */ + + func augmentationSessionDidFinish(_ session: AugmentationSession) + +} + +/** + * An object that augments an image. + */ + +public class AugmentationSession: NSObject { + + /// The filters to apply. + public let actions: [AugmentationFilter] + + /// The size of images. + public let imageSize = CGSize(width: 250, height: 250) + + /// The session delegate. + public weak var delegate: AugmentationSessionDelegate? + + /// The background queue where the augmentation work will be executed. + private let workQueue = DispatchQueue(label: "AugmentationSession.Work") + + // MARK: - Initialization + + public init(actions: [AugmentationFilter]) { + self.actions = actions + } + + // MARK: - Filters + + /** + * Starts a session with a delegate and an initial image. + * + * The filters will be applied cumulatively to the initial image. + */ + + public func start(initialImage: CGImage) { + workQueue.async { + self.applyFilters(start: initialImage) + } + } + + /** + * Applies the filter to an image. + */ + + private func applyFilters(start: CGImage) { + + var current = start + + for action in actions { + + let imageBounds = CGRect(origin: .zero, size: self.imageSize) + var outputImage: CGImage + var needsFlip: Bool = false + + switch action { + case .blur: + let ciImage = CIImage(cgImage: current) + + let filter = CIFilter(name: "CIGaussianBlur")! + filter.setValue(5, forKey: "inputRadius") + filter.setValue(ciImage, forKey: kCIInputImageKey) + let filteredImage = filter.outputImage! + + let context = CIContext(options: [kCIContextUseSoftwareRenderer: true]) + outputImage = context.createCGImage(filteredImage, from: ciImage.extent)! + needsFlip = true + + default: + var transform = CGAffineTransform.identity + switch action { + case .rotate(let degreeAngle): + transform = transform + .translatedBy(x: imageBounds.size.width / 2, y: imageBounds.size.height / 2) + .rotated(by: -degreeAngle * .pi / 180) + .translatedBy(x: -imageBounds.size.width / 2, y: -imageBounds.size.height / 2) + + case .scale(let scale): + transform = transform + .translatedBy(x: imageBounds.size.width / 2, y: imageBounds.size.height / 2) + .scaledBy(x: scale / 100, y: scale / 100) + .translatedBy(x: -imageBounds.size.width / 2, y: -imageBounds.size.height / 2) + + case .translate(let x, let y): + transform = transform.translatedBy(x: x, y: -y) + + case .blur: + preconditionFailure("Not reachable") + break + } + + let tranformedImage = UIGraphicsImageRenderer(size: imageBounds.size).image { context in + context.cgContext.concatenate(transform) + context.cgContext.draw(current, in: imageBounds) + } + + outputImage = tranformedImage.cgImage! + } + + let outputImageSize = CGSize(width: outputImage.width, height: outputImage.height) + let scaledRect = scaleRectangle(ofSize: outputImageSize, in: imageBounds) + + let renderedImage = UIGraphicsImageRenderer(size: imageSize).image { context in + UIColor.white.setFill() + context.fill(imageBounds) + context.cgContext.translateBy(x: 0, y: needsFlip ? imageBounds.size.height : 0) + context.cgContext.scaleBy(x: 1, y: needsFlip ? -1 : 1) + let outputImageRect = scaledRect + context.cgContext.draw(outputImage, in: outputImageRect) + } + + current = renderedImage.cgImage! + + DispatchQueue.main.async { + self.delegate?.augmentationSession(self, didCreateImage: renderedImage) + } + + } + + DispatchQueue.main.async { + self.delegate?.augmentationSessionDidFinish(self) + } + + } + + func scaleRectangle(ofSize size: CGSize, in rect: CGRect) -> CGRect { + + let imageWidth = size.width + let imageHeight = size.height + let containerWidth = rect.size.width + let containerHeight = rect.size.height + + // If the image is the same size as the container, return the container unscaled + if (imageWidth == containerWidth) && (imageHeight == containerHeight) { + return rect + } + + // Downscale the image to fit inside the container if needed + + let scale: CGFloat + let scaleX = containerWidth / imageWidth + let scaleY = containerHeight / imageHeight + + if (imageWidth > containerWidth) || (imageHeight > containerHeight) { + scale = min(scaleX, scaleY) + } else { + scale = 1 + } + + let adaptedWidth = imageWidth * scale + let adaptedHeight = imageHeight * scale + + // Center the image in the parent container + + var adaptedRect = CGRect(origin: .zero, size: CGSize(width: adaptedWidth, height: adaptedHeight)) + adaptedRect.origin.x = rect.origin.x + ((containerWidth - adaptedWidth) / 2) + adaptedRect.origin.y = rect.origin.y + ((containerHeight - adaptedHeight) / 2) + + return adaptedRect + + } + +} diff --git a/Playground/MLMOJI.playgroundbook/Contents/Sources/Paper/Exporters/BitmapPaperExporter.swift b/Playground/MLMOJI.playgroundbook/Contents/Sources/Paper/Exporters/BitmapPaperExporter.swift new file mode 100644 index 0000000..8b8715e --- /dev/null +++ b/Playground/MLMOJI.playgroundbook/Contents/Sources/Paper/Exporters/BitmapPaperExporter.swift @@ -0,0 +1,68 @@ +// +// MLMOJI +// +// This file is part of Alexis Aubry's WWDC18 scolarship submission open source project. +// Copyright © 2018 Alexis Aubry. Available under the terms of the MIT License. +// + +import UIKit +import MobileCoreServices + +/** + * Exports papers to image bitmaps. + */ + +public class BitmapPaperExporter: PaperExporter { + + /// The UTI of the image file format to use. + public let fileType: CFString + + /** + * Creates a bitmap exporter. + * + * - parameter size: The size of the image to export. + * - parameter scale: The factor to multiply the image size. + * - parameter fileType: The UTI of the image file format to use (ex: `kUTTypeJPEG`). Must + * conform to `public.image`. + */ + + public init(size: CGSize, scale: CGFloat, fileType: CFString) { + self.size = size + self.fileType = fileType + self.scale = scale + } + + // MARK: - PaperExporter + + public let size: CGSize + public let scale: CGFloat + + public func exportPaper(contents: CGImage) throws -> Data { + + let outputData = NSMutableData() + + guard UTTypeConformsTo(fileType, kUTTypeImage) else { + throw NSError(domain: "FilePaperExporterDomain", code: 2001, userInfo: [ + NSLocalizedDescriptionKey: "The selected type '\(fileType)' is not an image type." + ]) + } + + guard let destination = CGImageDestinationCreateWithData(outputData as CFMutableData, fileType, 1, nil) else { + throw NSError(domain: "FilePaperExporterDomain", code: 2002, userInfo: [ + NSLocalizedDescriptionKey: "Could not create the destination for the image." + ]) + } + + CGImageDestinationAddImage(destination, contents, nil) + + if !CGImageDestinationFinalize(destination) { + throw NSError(domain: "FilePaperExporterDomain", code: 2003, userInfo: [ + NSLocalizedDescriptionKey: "Could not save image as a file." + ]) + } + + return outputData as Data + + } + +} diff --git a/Playground/MLMOJI.playgroundbook/Contents/Sources/Paper/Exporters/CVPixelBufferExporter.swift b/Playground/MLMOJI.playgroundbook/Contents/Sources/Paper/Exporters/CVPixelBufferExporter.swift new file mode 100644 index 0000000..3fb696f --- /dev/null +++ b/Playground/MLMOJI.playgroundbook/Contents/Sources/Paper/Exporters/CVPixelBufferExporter.swift @@ -0,0 +1,73 @@ +// +// MLMOJI +// +// This file is part of Alexis Aubry's WWDC18 scolarship submission open source project. +// Copyright © 2018 Alexis Aubry. Available under the terms of the MIT License. +// + +import UIKit +import CoreVideo + +/** + * Exports images to a CVPixelBuffer. + */ + +public class CVPixelBufferExporter: PaperExporter { + + public init(size: CGSize, scale: CGFloat) { + self.size = size + self.scale = scale + } + + public let size: CGSize + public let scale: CGFloat + + public func exportPaper(contents: CGImage) throws -> CVPixelBuffer { + + // Create an empty image buffer that fits the size of the image + + let imageWidth = Int(size.width * scale) + let imageHeight = Int(size.height * scale) + + var buffer: CVPixelBuffer? = nil + let status = CVPixelBufferCreate(kCFAllocatorDefault, imageWidth, imageHeight, kCVPixelFormatType_32BGRA, nil, &buffer) + + guard status == kCVReturnSuccess else { + throw NSError(domain: "CVPixelBufferExporter", code: 2001, userInfo: [ + NSLocalizedDescriptionKey: "Could not create the pixel buffer for the image (code \(status))." + ]) + } + + guard let pixelBuffer = buffer else { + throw NSError(domain: "CVPixelBufferExporter", code: 2002, userInfo: [ + NSLocalizedDescriptionKey: "Could not create the pixel buffer for the image." + ]) + } + + // Draw the image into the buffer's context + + CVPixelBufferLockBaseAddress(pixelBuffer, []) + + let data = CVPixelBufferGetBaseAddress(pixelBuffer) + let colorSpace = CGColorSpaceCreateDeviceRGB() + let bitmapInfo = CGBitmapInfo(rawValue: CGBitmapInfo.byteOrder32Little.rawValue | CGImageAlphaInfo.premultipliedFirst.rawValue) + let bytesPerRow = CVPixelBufferGetBytesPerRow(pixelBuffer) + + let ctx = CGContext(data: data, width: imageWidth, height: imageHeight, bitsPerComponent: 8, bytesPerRow: bytesPerRow, + space: colorSpace, bitmapInfo: bitmapInfo.rawValue) + + guard let pixelBufferContext = ctx else { + throw NSError(domain: "CVPixelBufferExporter", code: 2003, userInfo: [ + NSLocalizedDescriptionKey: "Could not create the context to draw the image." + ]) + } + + pixelBufferContext.draw(contents, in: CGRect(origin: .zero, size: size)) + + // Return the filled buffer + CVPixelBufferUnlockBaseAddress(pixelBuffer, []) + return pixelBuffer + + } + +} diff --git a/Playground/MLMOJI.playgroundbook/Contents/Sources/Paper/Exporters/PaperExporter.swift b/Playground/MLMOJI.playgroundbook/Contents/Sources/Paper/Exporters/PaperExporter.swift new file mode 100644 index 0000000..3d40ee5 --- /dev/null +++ b/Playground/MLMOJI.playgroundbook/Contents/Sources/Paper/Exporters/PaperExporter.swift @@ -0,0 +1,35 @@ +// +// MLMOJI +// +// This file is part of Alexis Aubry's WWDC18 scolarship submission open source project. +// Copyright © 2018 Alexis Aubry. Available under the terms of the MIT License. +// + +import UIKit + +/** + * You implement this protocol to provide a way to export the sketches in a `Paper` instance. + */ + +public protocol PaperExporter { + + /** + * The type of output values produced by the exporter. Can be any type. + */ + + associatedtype Output + + /// The size of the exported image, in points. + var size: CGSize { get } + + /// The display scale to use to export the image. + var scale: CGFloat { get } + + /** + * Process the contents of the paper, provided as a `CGImage` and convert it to the + * object your exporter provides. + */ + + func exportPaper(contents: CGImage) throws -> Output + +} diff --git a/Playground/MLMOJI.playgroundbook/Contents/Sources/Paper/Paper.swift b/Playground/MLMOJI.playgroundbook/Contents/Sources/Paper/Paper.swift new file mode 100644 index 0000000..f5ed029 --- /dev/null +++ b/Playground/MLMOJI.playgroundbook/Contents/Sources/Paper/Paper.swift @@ -0,0 +1,367 @@ +// +// MLMOJI +// +// This file is part of Alexis Aubry's WWDC18 scolarship submission open source project. +// Copyright © 2018 Alexis Aubry. Available under the terms of the MIT License. +// + +import UIKit + +/** + * An object that receives information about the paper. + */ + +public protocol PaperDelegate: class { + + /** + * Called when the paper starts drawing a stroke. + */ + + func paperDidStartDrawingStroke(_ paper: Paper) + + /** + * Called when the paper did finish drawing and flattening a stroke. + */ + + func paperDidFinishDrawingStroke(_ paper: Paper) + +} + +/** + * A view that renders strokes based on touches. + */ + +public class Paper: UIView { + + /// The object that receives updates about the paper. + public weak var delegate: PaperDelegate? = nil + + // MARK: - Strokes + + /// The currently active stroke. + private var currentStroke: [CGPoint]? + + /// The image containing the previously drawn strokes. + private var strokesBuffer: UIImage? + + /// The color of the strokes. + private let strokeColor: UIColor = .black + + /// The size of the brush. + private let brushSize: CGFloat = 5 + + + // MARK: - Initialization + + public override init(frame: CGRect) { + super.init(frame: frame) + // Redraw the contents of the paper when the screen changes. + self.contentMode = .redraw + } + + public required init?(coder aDecoder: NSCoder) { + super.init(coder: aDecoder) + // Redraw the contents of the paper when the screen changes. + self.contentMode = .redraw + } + +} + +// MARK: - Drawing + +extension Paper { + + /** + * Draws the contents of the paper on the screen. + */ + + public override func draw(_ rect: CGRect) { + guard let context = UIGraphicsGetCurrentContext() else { return } + + if let buffer = strokesBuffer { + + // Scale the image if needed when we draw it for the first time + if rect == bounds { + let container = scaleRectangle(ofSize: buffer.size, in: rect) + buffer.draw(in: container) + } else { + buffer.draw(at: .zero) + } + + } + + // Draw the current unfinished stroke + if let stroke = currentStroke { + draw(stroke, in: context) + } + + } + + /** + * Draws a set of points. + */ + + private func draw(_ stroke: [CGPoint], in context: CGContext) { + + guard let firstPoint = stroke.first else { + return + } + + context.setFillColor(strokeColor.cgColor) + context.setStrokeColor(strokeColor.cgColor) + + // If there is only one point, draw a dot. + if stroke.count == 1 { + context.fillEllipse(in: boundingRect(around: firstPoint)) + return + } + + // If there are multiple points, join them with a set of strokes + let path = UIBezierPath() + path.lineCapStyle = .round + path.lineWidth = brushSize + + path.move(to: stroke.first!) + + for point in stroke.dropFirst() { + path.addLine(to: point) + } + + path.stroke() + + } + + /** + * Draws the current stroke on top of the buffer image, and updates the buffer image to + * include the new stroke. + */ + + private func flatten(in renderer: UIGraphicsImageRenderer, bounds: CGRect) -> UIImage { + + // Draw the image and the current stroke in the context of the screen + + return renderer.image { context in + + UIColor.white.setFill() + context.fill(bounds) + + if let bufferImage = strokesBuffer { + let container = scaleRectangle(ofSize: bufferImage.size, in: bounds) + bufferImage.draw(in: container) + } + + if let stroke = currentStroke { + draw(stroke, in: context.cgContext) + } + + } + + } + +} + +// MARK: - Calculations + +private extension Paper { + + /** + * Calculates a point between the previous point and the new one to achieve a smoother line. + */ + + func continuousPoint(at location: CGPoint) -> CGPoint { + let factor: CGFloat = 0.35 + let previous = currentStroke!.last! + return CGPoint(x: previous.x * (1 - factor) + location.x * factor, + y: previous.y * (1 - factor) + location.y * factor) + } + + /** + * Returns the bounding rectangle around the specified point. + * + * It is a rectangle where the point is the center, and whose size is the size of the brush. + */ + + func boundingRect(around point: CGPoint) -> CGRect { + let inset = brushSize / 2 + return CGRect(x: point.x - inset, y: point.y - inset, width: brushSize, height: brushSize) + } + + /** + * Returns the bounding rectangle around the last `n` last point. + */ + + func boundingRect(aroundLast n: Int) -> CGRect { + guard let lastPoints = currentStroke?.suffix(n) else { + return .zero + } + + var minX = 0.0, minY = 0.0, maxX = 0.0, maxY = 0.0 + + for point in lastPoints { + minX = min(Double(point.x), minX) + minY = min(Double(point.y), minY) + maxX = max(Double(point.x), maxX) + maxY = max(Double(point.y), maxY) + } + + let margins = Double(brushSize) * 2 + + return CGRect(x: minX, y: minY, width: maxX - minX + margins, height: maxY - minY + margins) + .insetBy(dx: -brushSize, dy: -brushSize) + } + + /** + * Downscales the image for the container if needed. + */ + + func scaleRectangle(ofSize size: CGSize, in rect: CGRect) -> CGRect { + + let imageWidth = size.width + let imageHeight = size.height + let containerWidth = rect.size.width + let containerHeight = rect.size.height + + // If the image is the same size as the container, return the container unscaled + if (imageWidth == containerWidth) && (imageHeight == containerHeight) { + return rect + } + + // Downscale the image to fit inside the container if needed + + let scale: CGFloat + let scaleX = containerWidth / imageWidth + let scaleY = containerHeight / imageHeight + + if (imageWidth > containerWidth) || (imageHeight > containerHeight) { + scale = min(scaleX, scaleY) + } else { + scale = 1 + } + + let adaptedWidth = imageWidth * scale + let adaptedHeight = imageHeight * scale + + // Center the image in the parent container + + var adaptedRect = CGRect(origin: .zero, size: CGSize(width: adaptedWidth, height: adaptedHeight)) + adaptedRect.origin.x = rect.origin.x + ((containerWidth - adaptedWidth) / 2) + adaptedRect.origin.y = rect.origin.y + ((containerHeight - adaptedHeight) / 2) + + return adaptedRect + + } + +} + +// MARK: - Touch Handling + +extension Paper { + + public override func touchesBegan(_ touches: Set, with event: UIEvent?) { + super.touchesBegan(touches, with: event) + + guard let location = touches.first?.location(in: self) else { + return + } + + self.currentStroke = [location] + + // Draw the initial point. + let affectedRect = boundingRect(around: location) + setNeedsDisplay(affectedRect) + } + + public override func touchesMoved(_ touches: Set, with event: UIEvent?) { + super.touchesMoved(touches, with: event) + + guard let firstTouch = touches.first else { + return + } + + // Handle touch aggregation + + guard let coalescedTouches = event?.coalescedTouches(for: firstTouch) else { + return + } + + for touch in coalescedTouches { + let location = touch.location(in: self) + let point = continuousPoint(at: location) + currentStroke?.append(point) + } + + delegate?.paperDidStartDrawingStroke(self) + + // Redraw the points that affect the curve + let affectedRect = boundingRect(aroundLast: coalescedTouches.count + 2) + setNeedsDisplay(affectedRect) + + } + + public override func touchesEnded(_ touches: Set, with event: UIEvent?) { + super.touchesEnded(touches, with: event) + + // Flatten the stroke so that it can be scaled as appropriate if the screen size changes + let renderer = UIGraphicsImageRenderer(bounds: bounds) + strokesBuffer = flatten(in: renderer, bounds: bounds) + + // Redraw with the buffer + currentStroke = nil + delegate?.paperDidFinishDrawingStroke(self) + setNeedsDisplay() + } + + public override var canBecomeFirstResponder: Bool { + return true + } + +} + +// MARK: - Interacting with the contents + +extension Paper { + + /** + * Clears the contents of the screen. + */ + + public func clear() { + strokesBuffer = nil + setNeedsDisplay() + } + + /** + * Exports the contents of the paper as an image. + * + * - parameter size: The size of the exported image. + */ + + public func exportImage(size: CGSize) -> CGImage { + + let bounds = CGRect(origin: .zero, size: size) + + let format = UIGraphicsImageRendererFormat(for: traitCollection) + format.opaque = true + format.prefersExtendedRange = false + format.scale = 1 + + let renderer = UIGraphicsImageRenderer(bounds: bounds, format: format) + let finalImage = flatten(in: renderer, bounds: bounds) + + return finalImage.cgImage! + + } + + /** + * Exports the contents of the canvas using the specified exporter. + * + * The strokes will be drawn on a white opaque background. The image will be scaled to + * fit inside the exporter's container bounds, using its `size` property. + */ + + public func export(with exporter: T) throws -> T.Output { + let exportedImage = exportImage(size: exporter.size) + return try exporter.exportPaper(contents: exportedImage) + } + +} diff --git a/Playground/MLMOJI.playgroundbook/Contents/Sources/Prediction/Classes.swift b/Playground/MLMOJI.playgroundbook/Contents/Sources/Prediction/Classes.swift new file mode 100644 index 0000000..3f5fcfb --- /dev/null +++ b/Playground/MLMOJI.playgroundbook/Contents/Sources/Prediction/Classes.swift @@ -0,0 +1,55 @@ +// +// MLMOJI +// +// This file is part of Alexis Aubry's WWDC18 scolarship submission open source project. +// Copyright © 2018 Alexis Aubry. Available under the terms of the MIT License. +// + +import UIKit + +/** + * The classes that the image classifier recognizes. + */ + +public enum Class: String { + + case laugh + case smile + case heart + case checkmark + case croissant + case sun + case cloud + + /// An array containing all the class labels. + public static var allLabels: [Class] { + return [.smile, .laugh, .sun, .checkmark, .croissant, .heart, .cloud] + } + + /// The emoji representation of this label. + public var emojiValue: String { + switch self { + case .laugh: return "😂" + case .smile: return "😊" + case .heart: return "❤️" + case .checkmark: return "✔️" + case .croissant: return "🥐" + case .sun: return "☀️" + case .cloud: return "☁️" + } + } + + /// The color to show for prediction bars. + public var highlightColor: UIColor { + switch self { + case .laugh: return #colorLiteral(red: 0.8274509804, green: 0.3333333333, blue: 0.3607843137, alpha: 1) + case .smile: return #colorLiteral(red: 0.9568627451, green: 0.5529411765, blue: 0.2274509804, alpha: 1) + case .heart: return #colorLiteral(red: 0.9921568627, green: 0.7803921569, blue: 0.3254901961, alpha: 1) + case .checkmark: return #colorLiteral(red: 0.4392156863, green: 0.737254902, blue: 0.3254901961, alpha: 1) + case .croissant: return #colorLiteral(red: 0.1411764706, green: 0.6117647059, blue: 0.8352941176, alpha: 1) + case .sun: return #colorLiteral(red: 0.2196078431, green: 0.08235294118, blue: 0.5725490196, alpha: 1) + case .cloud: return #colorLiteral(red: 0.5058823529, green: 0.1215686275, blue: 0.537254902, alpha: 1) + } + } + +} diff --git a/Playground/MLMOJI.playgroundbook/Contents/Sources/Prediction/EmojiPrediction.swift b/Playground/MLMOJI.playgroundbook/Contents/Sources/Prediction/EmojiPrediction.swift new file mode 100644 index 0000000..87480f2 --- /dev/null +++ b/Playground/MLMOJI.playgroundbook/Contents/Sources/Prediction/EmojiPrediction.swift @@ -0,0 +1,34 @@ +// +// MLMOJI +// +// This file is part of Alexis Aubry's WWDC18 scolarship submission open source project. +// Copyright © 2018 Alexis Aubry. Available under the terms of the MIT License. +// + +import Foundation + +/** + * The result of emoji classification. + */ + +public class EmojiPrediction { + + let output: EmojiSketchesOutput + + init(output: EmojiSketchesOutput) { + self.output = output + } + + // MARK: - Wrappers + + /// The predicted class. + public var classLabel: String { + return output.classLabel + } + + /// The predicted percentage for each class in the `Class` enum. + public var predictions: [String: Double] { + return output.final_result__0 + } + +} diff --git a/Playground/MLMOJI.playgroundbook/Contents/Sources/Prediction/EmojiSketches.swift b/Playground/MLMOJI.playgroundbook/Contents/Sources/Prediction/EmojiSketches.swift new file mode 100644 index 0000000..e0d4a06 --- /dev/null +++ b/Playground/MLMOJI.playgroundbook/Contents/Sources/Prediction/EmojiSketches.swift @@ -0,0 +1,115 @@ +// +// MLMOJI +// +// This file is part of Alexis Aubry's WWDC18 scolarship submission open source project. +// Copyright © 2018 Alexis Aubry. Available under the terms of the MIT License. +// +// This file was automatically generated and should not be edited. +// + +import CoreML + +/// Model Prediction Input Type +@available(macOS 10.13, iOS 11.0, tvOS 11.0, watchOS 4.0, *) +class EmojiSketchesInput : MLFeatureProvider { + + /// input__0 as color (kCVPixelFormatType_32BGRA) image buffer, 224 pixels wide by 224 pixels high + var input__0: CVPixelBuffer + + var featureNames: Set { + get { + return ["input__0"] + } + } + + func featureValue(for featureName: String) -> MLFeatureValue? { + if (featureName == "input__0") { + return MLFeatureValue(pixelBuffer: input__0) + } + return nil + } + + init(input__0: CVPixelBuffer) { + self.input__0 = input__0 + } +} + +/// Model Prediction Output Type +@available(macOS 10.13, iOS 11.0, tvOS 11.0, watchOS 4.0, *) +public class EmojiSketchesOutput : MLFeatureProvider { + + /// final_result__0 as dictionary of strings to doubles + public let final_result__0: [String : Double] + + /// classLabel as string value + public let classLabel: String + + public var featureNames: Set { + get { + return ["final_result__0", "classLabel"] + } + } + + public func featureValue(for featureName: String) -> MLFeatureValue? { + if (featureName == "final_result__0") { + return try! MLFeatureValue(dictionary: final_result__0 as [NSObject : NSNumber]) + } + if (featureName == "classLabel") { + return MLFeatureValue(string: classLabel) + } + return nil + } + + public init(final_result__0: [String : Double], classLabel: String) { + self.final_result__0 = final_result__0 + self.classLabel = classLabel + } +} + +/// Class for model loading and prediction +@available(macOS 10.13, iOS 11.0, tvOS 11.0, watchOS 4.0, *) +class EmojiSketches { + var model: MLModel + + /** + Construct a model with explicit path to mlmodel file + - parameters: + - url: the file url of the model + - throws: an NSError object that describes the problem + */ + init(contentsOf url: URL) throws { + self.model = try MLModel(contentsOf: url) + } + + /// Construct a model that automatically loads the model from the app's bundle + convenience init() { + let bundle = Bundle(for: EmojiSketches.self) + let assetPath = bundle.url(forResource: "EmojiSketches", withExtension:"mlmodelc") + try! self.init(contentsOf: assetPath!) + } + + /** + Make a prediction using the structured interface + - parameters: + - input: the input to the prediction as EmojiSketchesInput + - throws: an NSError object that describes the problem + - returns: the result of the prediction as EmojiSketchesOutput + */ + func prediction(input: EmojiSketchesInput) throws -> EmojiSketchesOutput { + let outFeatures = try model.prediction(from: input) + let result = EmojiSketchesOutput(final_result__0: outFeatures.featureValue(for: "final_result__0")!.dictionaryValue as! [String : Double], classLabel: outFeatures.featureValue(for: "classLabel")!.stringValue) + return result + } + + /** + Make a prediction using the convenience interface + - parameters: + - input__0 as color (kCVPixelFormatType_32BGRA) image buffer, 224 pixels wide by 224 pixels high + - throws: an NSError object that describes the problem + - returns: the result of the prediction as EmojiSketchesOutput + */ + func prediction(input__0: CVPixelBuffer) throws -> EmojiSketchesOutput { + let input_ = EmojiSketchesInput(input__0: input__0) + return try self.prediction(input: input_) + } +} diff --git a/Playground/MLMOJI.playgroundbook/Contents/Sources/Prediction/PredictionSession.swift b/Playground/MLMOJI.playgroundbook/Contents/Sources/Prediction/PredictionSession.swift new file mode 100644 index 0000000..fb9b4bb --- /dev/null +++ b/Playground/MLMOJI.playgroundbook/Contents/Sources/Prediction/PredictionSession.swift @@ -0,0 +1,67 @@ +// +// MLMOJI +// +// This file is part of Alexis Aubry's WWDC18 scolarship submission open source project. +// Copyright © 2018 Alexis Aubry. Available under the terms of the MIT License. +// + +import Foundation +import CoreVideo + +/** + * Provides Core ML predictions for the EmojiSketches model. + */ + +public class PredictionSession { + + /// The delegate to notify with updates. + public weak var delegate: PredictionSessionDelegate? = nil + + /// The size of input images. + public let inputSize = CGSize(width: 224, height: 224) + + /// The emoji prediction model. + private let model: EmojiSketches + + /// The operation queue where operations will be executed. + private let workQueue = DispatchQueue(label: "PredictionManager", qos: .userInitiated) + + /** + * Creates a new session. + */ + + public init() { + let url = Bundle.main.url(forResource: "EmojiSketches", withExtension: "mlmodelc")! + model = try! EmojiSketches(contentsOf: url) + } + + // MARK: - Getting Predictions + + /** + * Requests a prediction for the specified image buffer. The results will be provided + * to the session's delegate when available. + */ + + public func requestPrediction(for buffer: CVPixelBuffer) { + + let predictionWorkItem = DispatchWorkItem { + + do { + let output = try self.model.prediction(input__0: buffer) + let prediction = EmojiPrediction(output: output) + DispatchQueue.main.async { + self.delegate?.predictionSession(self, didUpdatePrediction: prediction) + } + } catch { + DispatchQueue.main.async { + self.delegate?.predictionSession(self, didFailToProvidePredictionWith: error as NSError) + } + } + + } + + workQueue.async(execute: predictionWorkItem) + + } + +} diff --git a/Playground/MLMOJI.playgroundbook/Contents/Sources/Prediction/PredictionSessionDelegate.swift b/Playground/MLMOJI.playgroundbook/Contents/Sources/Prediction/PredictionSessionDelegate.swift new file mode 100644 index 0000000..f77dad4 --- /dev/null +++ b/Playground/MLMOJI.playgroundbook/Contents/Sources/Prediction/PredictionSessionDelegate.swift @@ -0,0 +1,32 @@ +// +// MLMOJI +// +// This file is part of Alexis Aubry's WWDC18 scolarship submission open source project. +// Copyright © 2018 Alexis Aubry. Available under the terms of the MIT License. +// + +import Foundation + +/** + * An object that receives notifications from a prediction session. + */ + +public protocol PredictionSessionDelegate: class { + + /** + * The session has finished predicting the emoji for the current buffer. + * + * Always called on the main thread. + */ + + func predictionSession(_ session: PredictionSession, didUpdatePrediction prediction: EmojiPrediction) + + /** + * The session failed to provide a prediction for the current buffer, and returned an error. + * + * Always called on the main thread. + */ + + func predictionSession(_ session: PredictionSession, didFailToProvidePredictionWith error: NSError) + +} diff --git a/Playground/MLMOJI.playgroundbook/Contents/Sources/Utilities/Animation.swift b/Playground/MLMOJI.playgroundbook/Contents/Sources/Utilities/Animation.swift new file mode 100644 index 0000000..69cdd01 --- /dev/null +++ b/Playground/MLMOJI.playgroundbook/Contents/Sources/Utilities/Animation.swift @@ -0,0 +1,106 @@ +// +// MLMOJI +// +// This file is part of Alexis Aubry's WWDC18 scolarship submission open source project. +// Copyright © 2018 Alexis Aubry. Available under the terms of the MIT License. +// + +import UIKit + +/** + * The types of animation that can be queued. + */ + +public enum AnimationType { + + /// A UIView transition of the specified duration and options. + case transition(Double, UIViewAnimationOptions) + + /// A UIView property animation of the specified duration and curve. + case property(Double, UIViewAnimationCurve) + + /// A change that is not animated. + case notAnimated + +} + +/** + * Describes a view change. + */ + +public class Animation { + + /// The changes to send to the view. + let changes: () -> Void + + /// The type of animation to perform. + let type: AnimationType + + public init(type: AnimationType, changes: @escaping () -> Void) { + self.changes = changes + self.type = type + } + +} + +// MARK: - Enqueuing Changes + +extension UIView { + + /** + * Enqueues an animation. + */ + + public func enqueueChanges(animation: AnimationType, changes: @escaping () -> Void, completion: ((Bool) -> Void)? = nil) { + + setNeedsLayout() + + let layoutCompletion: (Bool) -> Void = { + self.setNeedsLayout() + completion?($0) + } + + switch animation { + case .notAnimated: + changes() + layoutCompletion(true) + + case let .property(duration, curve): + let animator = UIViewPropertyAnimator(duration: duration, curve: curve, animations: changes) + animator.addCompletion { layoutCompletion($0 == .end) } + animator.startAnimation() + + case let .transition(duration, options): + UIView.transition(with: self, duration: duration, options: options, animations: changes, completion: layoutCompletion) + } + + } + + /** + * Performs multiple animations one after the other. + */ + + public func enqueueChanges(_ animations: [Animation]) { + + guard let currentAnimation = animations.first else { + return + } + + let nextIndex = animations.index(after: animations.startIndex) + + enqueueChanges(animation: currentAnimation.type, changes: currentAnimation.changes) { finished in + + let next = Array(animations.suffix(from: nextIndex)) + + guard finished else { + next.forEach { $0.changes() } + return + } + + self.enqueueChanges(next) + + } + + } + +} diff --git a/Playground/MLMOJI.playgroundbook/Contents/Sources/Utilities/AutoLayout+Pin.swift b/Playground/MLMOJI.playgroundbook/Contents/Sources/Utilities/AutoLayout+Pin.swift new file mode 100644 index 0000000..28aa31a --- /dev/null +++ b/Playground/MLMOJI.playgroundbook/Contents/Sources/Utilities/AutoLayout+Pin.swift @@ -0,0 +1,24 @@ +// +// MLMOJI +// +// This file is part of Alexis Aubry's WWDC18 scolarship submission open source project. +// Copyright © 2018 Alexis Aubry. Available under the terms of the MIT License. +// + +import UIKit + +extension UIView { + + /** + * Pins the edges of the reciever to those of the specified view. + */ + + public func pinEdges(to container: UIView) { + self.translatesAutoresizingMaskIntoConstraints = false + self.leadingAnchor.constraint(equalTo: container.leadingAnchor).isActive = true + self.trailingAnchor.constraint(equalTo: container.trailingAnchor).isActive = true + self.topAnchor.constraint(equalTo: container.topAnchor).isActive = true + self.bottomAnchor.constraint(equalTo: container.bottomAnchor).isActive = true + } + +} diff --git a/Playground/MLMOJI.playgroundbook/Contents/Sources/Utilities/Messages.swift b/Playground/MLMOJI.playgroundbook/Contents/Sources/Utilities/Messages.swift new file mode 100644 index 0000000..58ecd17 --- /dev/null +++ b/Playground/MLMOJI.playgroundbook/Contents/Sources/Utilities/Messages.swift @@ -0,0 +1,42 @@ +// +// MLMOJI +// +// This file is part of Alexis Aubry's WWDC18 scolarship submission open source project. +// Copyright © 2018 Alexis Aubry. Available under the terms of the MIT License. +// + +import Foundation +import PlaygroundSupport + +/** + * The types of messages sent to the playground. + */ + +public enum MessageType { + + /// The message describes an event. + public static let event: String = "Event" +} + +/** + * The events that can happen in the app. + */ + +public enum Event: String { + + /// The plaground page requested that the predictions be started. + case startPredictions = "StartPredictions" + +} + +extension PlaygroundRemoteLiveViewProxy { + + /** + * Sends an event to the proxy. + */ + + public func sendEvent(_ event: Event) { + self.send(.dictionary([MessageType.event: PlaygroundValue.string(event.rawValue)])) + } + +} diff --git a/Playground/MLMOJI.playgroundbook/Contents/Sources/Utilities/PlaygroundPage+Tools.swift b/Playground/MLMOJI.playgroundbook/Contents/Sources/Utilities/PlaygroundPage+Tools.swift new file mode 100644 index 0000000..7c3b2cd --- /dev/null +++ b/Playground/MLMOJI.playgroundbook/Contents/Sources/Utilities/PlaygroundPage+Tools.swift @@ -0,0 +1,21 @@ +// +// MLMOJI +// +// This file is part of Alexis Aubry's WWDC18 scolarship submission open source project. +// Copyright © 2018 Alexis Aubry. Available under the terms of the MIT License. +// + +import Foundation +import PlaygroundSupport + +extension PlaygroundPage { + + /** + * The object that receives message from the playground page. + */ + + public var proxy: PlaygroundRemoteLiveViewProxy? { + return liveView as? PlaygroundRemoteLiveViewProxy + } + +} diff --git a/Playground/MLMOJI.playgroundbook/Contents/Sources/Utilities/PlaygroundViewController.swift b/Playground/MLMOJI.playgroundbook/Contents/Sources/Utilities/PlaygroundViewController.swift new file mode 100644 index 0000000..26f1b64 --- /dev/null +++ b/Playground/MLMOJI.playgroundbook/Contents/Sources/Utilities/PlaygroundViewController.swift @@ -0,0 +1,54 @@ +// +// MLMOJI +// +// This file is part of Alexis Aubry's WWDC18 scolarship submission open source project. +// Copyright © 2018 Alexis Aubry. Available under the terms of the MIT License. +// + +import UIKit +import PlaygroundSupport + +// MARK: PlaygroundContainable + +/** + * A view controller that can be contained inside a playground's safe area. + */ + +public protocol PlaygroundContainable: class { + var containerLayoutGuide: UILayoutGuide! { get set } + func configureConstraints() +} + +// MARK: - PlaygroundViewController + +/** + * A view controller that displays a child view controller within the playground safe area. + */ + +open class PlaygroundViewController: UIViewController, PlaygroundLiveViewSafeAreaContainer { + + /// The child view controller displayed in the playground. + public let child: T + + // MARK: Initialization + + public init(child: T) { + self.child = child + super.init(nibName: nil, bundle: nil) + } + + public required init?(coder aDecoder: NSCoder) { + fatalError("init(coder:) was not implemented.") + } + + override open func viewDidLoad() { + super.viewDidLoad() + child.containerLayoutGuide = liveViewSafeAreaGuide + addChildViewController(child) + view.addSubview(child.view) + child.configureConstraints() + child.view.pinEdges(to: view) + child.didMove(toParentViewController: self) + } + +} diff --git a/Playground/PlaygroundStub/PlaygroundStub.xcodeproj/project.pbxproj b/Playground/PlaygroundStub/PlaygroundStub.xcodeproj/project.pbxproj new file mode 100644 index 0000000..663c444 --- /dev/null +++ b/Playground/PlaygroundStub/PlaygroundStub.xcodeproj/project.pbxproj @@ -0,0 +1,573 @@ +// !$*UTF8*$! +{ + archiveVersion = 1; + classes = { + }; + objectVersion = 48; + objects = { + +/* Begin PBXBuildFile section */ + 552FFBCB2072051300B0E10E /* AugmentedSet@2x.png in Resources */ = {isa = PBXBuildFile; fileRef = 552FFBCA2072051200B0E10E /* AugmentedSet@2x.png */; }; + 552FFBCD20722C0B00B0E10E /* PredictionStubViewController.swift in Sources */ = {isa = PBXBuildFile; fileRef = 552FFBCC20722C0B00B0E10E /* PredictionStubViewController.swift */; }; + 552FFBCF20722CE600B0E10E /* AugmentationStubViewController.swift in Sources */ = {isa = PBXBuildFile; fileRef = 552FFBCE20722CE600B0E10E /* AugmentationStubViewController.swift */; }; + 5551CD46206946C200CEBB77 /* AppDelegate.swift in Sources */ = {isa = PBXBuildFile; fileRef = 5551CD45206946C200CEBB77 /* AppDelegate.swift */; }; + 5551CD4B206946C200CEBB77 /* Main.storyboard in Resources */ = {isa = PBXBuildFile; fileRef = 5551CD49206946C200CEBB77 /* Main.storyboard */; }; + 5551CD4D206946C200CEBB77 /* Assets.xcassets in Resources */ = {isa = PBXBuildFile; fileRef = 5551CD4C206946C200CEBB77 /* Assets.xcassets */; }; + 5551CD50206946C200CEBB77 /* LaunchScreen.storyboard in Resources */ = {isa = PBXBuildFile; fileRef = 5551CD4E206946C200CEBB77 /* LaunchScreen.storyboard */; }; + 5551CD622069475E00CEBB77 /* Paper.swift in Sources */ = {isa = PBXBuildFile; fileRef = 5551CD592069475E00CEBB77 /* Paper.swift */; }; + 5551CD632069475E00CEBB77 /* BitmapPaperExporter.swift in Sources */ = {isa = PBXBuildFile; fileRef = 5551CD5B2069475E00CEBB77 /* BitmapPaperExporter.swift */; }; + 5551CD642069475E00CEBB77 /* PaperExporter.swift in Sources */ = {isa = PBXBuildFile; fileRef = 5551CD5C2069475E00CEBB77 /* PaperExporter.swift */; }; + 5551CD652069475E00CEBB77 /* CVPixelBufferExporter.swift in Sources */ = {isa = PBXBuildFile; fileRef = 5551CD5D2069475E00CEBB77 /* CVPixelBufferExporter.swift */; }; + 5551CD682069475E00CEBB77 /* Classes.swift in Sources */ = {isa = PBXBuildFile; fileRef = 5551CD612069475E00CEBB77 /* Classes.swift */; }; + 555C3443206F689300F74F6B /* AugmentationSession.swift in Sources */ = {isa = PBXBuildFile; fileRef = 555C3442206F689200F74F6B /* AugmentationSession.swift */; }; + 556155F820722E2600AFA4C8 /* EmojiPrediction.swift in Sources */ = {isa = PBXBuildFile; fileRef = 556155F720722E2600AFA4C8 /* EmojiPrediction.swift */; }; + 556262C42070CDB60066DF0E /* AugmentationFilter.swift in Sources */ = {isa = PBXBuildFile; fileRef = 556262C32070CDB60066DF0E /* AugmentationFilter.swift */; }; + 55B661232070DB2F006B964D /* Manifest.plist in Resources */ = {isa = PBXBuildFile; fileRef = 55B6611B2070DB2F006B964D /* Manifest.plist */; }; + 55B661272070DB2F006B964D /* ImageCollectionViewCell.swift in Sources */ = {isa = PBXBuildFile; fileRef = 55B661212070DB2F006B964D /* ImageCollectionViewCell.swift */; }; + 55B661282070DB2F006B964D /* AugmentViewController.swift in Sources */ = {isa = PBXBuildFile; fileRef = 55B661222070DB2F006B964D /* AugmentViewController.swift */; }; + 55B661292070DC63006B964D /* PredictionResultNode.swift in Sources */ = {isa = PBXBuildFile; fileRef = 55B661152070DB19006B964D /* PredictionResultNode.swift */; }; + 55B6612A2070DC63006B964D /* PredictionResultsContainer.swift in Sources */ = {isa = PBXBuildFile; fileRef = 55B661172070DB19006B964D /* PredictionResultsContainer.swift */; }; + 55B6612B2070DC63006B964D /* PredictionViewController.swift in Sources */ = {isa = PBXBuildFile; fileRef = 55B661182070DB19006B964D /* PredictionViewController.swift */; }; + 55B6613620712245006B964D /* Manifest.plist in Resources */ = {isa = PBXBuildFile; fileRef = 55B6612D20712245006B964D /* Manifest.plist */; }; + 55FA911A206C0F6F007FCE6A /* PredictionSession.swift in Sources */ = {isa = PBXBuildFile; fileRef = 55FA9119206C0F6F007FCE6A /* PredictionSession.swift */; }; + 55FA9126206C11E8007FCE6A /* EmojiSketches.mlmodel in Sources */ = {isa = PBXBuildFile; fileRef = 55FA9125206C11E8007FCE6A /* EmojiSketches.mlmodel */; }; + 55FA9128206D2E04007FCE6A /* PredictionSessionDelegate.swift in Sources */ = {isa = PBXBuildFile; fileRef = 55FA9127206D2E04007FCE6A /* PredictionSessionDelegate.swift */; }; + 55FA912B206D3250007FCE6A /* AutoLayout+Pin.swift in Sources */ = {isa = PBXBuildFile; fileRef = 55FA912A206D3250007FCE6A /* AutoLayout+Pin.swift */; }; + 55FA9137206E02A4007FCE6A /* Animation.swift in Sources */ = {isa = PBXBuildFile; fileRef = 55FA9136206E02A4007FCE6A /* Animation.swift */; }; +/* End PBXBuildFile section */ + +/* Begin PBXFileReference section */ + 552FFBCA2072051200B0E10E /* AugmentedSet@2x.png */ = {isa = PBXFileReference; lastKnownFileType = image.png; path = "AugmentedSet@2x.png"; sourceTree = ""; }; + 552FFBCC20722C0B00B0E10E /* PredictionStubViewController.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = PredictionStubViewController.swift; sourceTree = ""; }; + 552FFBCE20722CE600B0E10E /* AugmentationStubViewController.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = AugmentationStubViewController.swift; sourceTree = ""; }; + 5551CD42206946C200CEBB77 /* PlaygroundStub.app */ = {isa = PBXFileReference; explicitFileType = wrapper.application; includeInIndex = 0; path = PlaygroundStub.app; sourceTree = BUILT_PRODUCTS_DIR; }; + 5551CD45206946C200CEBB77 /* AppDelegate.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = AppDelegate.swift; sourceTree = ""; }; + 5551CD4A206946C200CEBB77 /* Base */ = {isa = PBXFileReference; lastKnownFileType = file.storyboard; name = Base; path = Base.lproj/Main.storyboard; sourceTree = ""; }; + 5551CD4C206946C200CEBB77 /* Assets.xcassets */ = {isa = PBXFileReference; lastKnownFileType = folder.assetcatalog; path = Assets.xcassets; sourceTree = ""; }; + 5551CD4F206946C200CEBB77 /* Base */ = {isa = PBXFileReference; lastKnownFileType = file.storyboard; name = Base; path = Base.lproj/LaunchScreen.storyboard; sourceTree = ""; }; + 5551CD51206946C200CEBB77 /* Info.plist */ = {isa = PBXFileReference; lastKnownFileType = text.plist.xml; path = Info.plist; sourceTree = ""; }; + 5551CD592069475E00CEBB77 /* Paper.swift */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.swift; path = Paper.swift; sourceTree = ""; }; + 5551CD5B2069475E00CEBB77 /* BitmapPaperExporter.swift */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.swift; path = BitmapPaperExporter.swift; sourceTree = ""; }; + 5551CD5C2069475E00CEBB77 /* PaperExporter.swift */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.swift; path = PaperExporter.swift; sourceTree = ""; }; + 5551CD5D2069475E00CEBB77 /* CVPixelBufferExporter.swift */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.swift; path = CVPixelBufferExporter.swift; sourceTree = ""; }; + 5551CD612069475E00CEBB77 /* Classes.swift */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.swift; path = Classes.swift; sourceTree = ""; }; + 555C3442206F689200F74F6B /* AugmentationSession.swift */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.swift; path = AugmentationSession.swift; sourceTree = ""; }; + 556155F720722E2600AFA4C8 /* EmojiPrediction.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = EmojiPrediction.swift; sourceTree = ""; }; + 556262C32070CDB60066DF0E /* AugmentationFilter.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = AugmentationFilter.swift; sourceTree = ""; }; + 55B6610F2070DB19006B964D /* Manifest.plist */ = {isa = PBXFileReference; lastKnownFileType = text.plist.xml; path = Manifest.plist; sourceTree = ""; }; + 55B661112070DB19006B964D /* EmojiStrokes@2x.png */ = {isa = PBXFileReference; lastKnownFileType = image.png; path = "EmojiStrokes@2x.png"; sourceTree = ""; }; + 55B661122070DB19006B964D /* SupportedEmoji@2x.png */ = {isa = PBXFileReference; lastKnownFileType = image.png; path = "SupportedEmoji@2x.png"; sourceTree = ""; }; + 55B661132070DB19006B964D /* Contents.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = Contents.swift; sourceTree = ""; }; + 55B661152070DB19006B964D /* PredictionResultNode.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = PredictionResultNode.swift; sourceTree = ""; }; + 55B661162070DB19006B964D /* PredictionViewController+Playground.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = "PredictionViewController+Playground.swift"; sourceTree = ""; }; + 55B661172070DB19006B964D /* PredictionResultsContainer.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = PredictionResultsContainer.swift; sourceTree = ""; }; + 55B661182070DB19006B964D /* PredictionViewController.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = PredictionViewController.swift; sourceTree = ""; }; + 55B661192070DB19006B964D /* LiveView.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = LiveView.swift; sourceTree = ""; }; + 55B6611B2070DB2F006B964D /* Manifest.plist */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = text.plist.xml; path = Manifest.plist; sourceTree = ""; }; + 55B6611E2070DB2F006B964D /* Contents.swift */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.swift; path = Contents.swift; sourceTree = ""; }; + 55B661202070DB2F006B964D /* AugmentViewController+Playground.swift */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.swift; path = "AugmentViewController+Playground.swift"; sourceTree = ""; }; + 55B661212070DB2F006B964D /* ImageCollectionViewCell.swift */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.swift; path = ImageCollectionViewCell.swift; sourceTree = ""; }; + 55B661222070DB2F006B964D /* AugmentViewController.swift */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.swift; path = AugmentViewController.swift; sourceTree = ""; }; + 55B6612D20712245006B964D /* Manifest.plist */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = text.plist.xml; path = Manifest.plist; sourceTree = ""; }; + 55B6613020712245006B964D /* Contents.swift */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.swift; path = Contents.swift; sourceTree = ""; }; + 55B6613520712245006B964D /* LiveView.swift */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.swift; path = LiveView.swift; sourceTree = ""; }; + 55FA9117206C0F3E007FCE6A /* EmojiSketches.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = EmojiSketches.swift; sourceTree = ""; }; + 55FA9119206C0F6F007FCE6A /* PredictionSession.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = PredictionSession.swift; sourceTree = ""; }; + 55FA9125206C11E8007FCE6A /* EmojiSketches.mlmodel */ = {isa = PBXFileReference; lastKnownFileType = file.mlmodel; name = EmojiSketches.mlmodel; path = "../../../Data-Model/EmojiSketches.mlmodel"; sourceTree = ""; }; + 55FA9127206D2E04007FCE6A /* PredictionSessionDelegate.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = PredictionSessionDelegate.swift; sourceTree = ""; }; + 55FA912A206D3250007FCE6A /* AutoLayout+Pin.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = "AutoLayout+Pin.swift"; sourceTree = ""; }; + 55FA9136206E02A4007FCE6A /* Animation.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = Animation.swift; sourceTree = ""; }; +/* End PBXFileReference section */ + +/* Begin PBXFrameworksBuildPhase section */ + 5551CD3F206946C200CEBB77 /* Frameworks */ = { + isa = PBXFrameworksBuildPhase; + buildActionMask = 2147483647; + files = ( + ); + runOnlyForDeploymentPostprocessing = 0; + }; +/* End PBXFrameworksBuildPhase section */ + +/* Begin PBXGroup section */ + 552FFBC92072051200B0E10E /* PrivateResources */ = { + isa = PBXGroup; + children = ( + 552FFBCA2072051200B0E10E /* AugmentedSet@2x.png */, + ); + path = PrivateResources; + sourceTree = ""; + }; + 552FFBD020722DC400B0E10E /* Supporting */ = { + isa = PBXGroup; + children = ( + 5551CD49206946C200CEBB77 /* Main.storyboard */, + 5551CD4C206946C200CEBB77 /* Assets.xcassets */, + 5551CD4E206946C200CEBB77 /* LaunchScreen.storyboard */, + 5551CD51206946C200CEBB77 /* Info.plist */, + 5551CD45206946C200CEBB77 /* AppDelegate.swift */, + ); + name = Supporting; + sourceTree = ""; + }; + 5551CD39206946C200CEBB77 = { + isa = PBXGroup; + children = ( + 5551CD44206946C200CEBB77 /* PlaygroundStub */, + 5551CD43206946C200CEBB77 /* Products */, + ); + sourceTree = ""; + }; + 5551CD43206946C200CEBB77 /* Products */ = { + isa = PBXGroup; + children = ( + 5551CD42206946C200CEBB77 /* PlaygroundStub.app */, + ); + name = Products; + sourceTree = ""; + }; + 5551CD44206946C200CEBB77 /* PlaygroundStub */ = { + isa = PBXGroup; + children = ( + 552FFBCC20722C0B00B0E10E /* PredictionStubViewController.swift */, + 552FFBCE20722CE600B0E10E /* AugmentationStubViewController.swift */, + 552FFBD020722DC400B0E10E /* Supporting */, + 5551CD71206AF26C00CEBB77 /* Resources */, + 55B6610D2070DB02006B964D /* Chapters */, + 5551CD572069475E00CEBB77 /* Sources */, + ); + path = PlaygroundStub; + sourceTree = ""; + }; + 5551CD572069475E00CEBB77 /* Sources */ = { + isa = PBXGroup; + children = ( + 555C3441206F689200F74F6B /* Augmentation */, + 5551CD582069475E00CEBB77 /* Paper */, + 55FA9116206C0F00007FCE6A /* Prediction */, + 55FA9129206D323C007FCE6A /* Utilities */, + ); + name = Sources; + path = ../../MLMOJI.playgroundbook/Contents/Sources; + sourceTree = ""; + }; + 5551CD582069475E00CEBB77 /* Paper */ = { + isa = PBXGroup; + children = ( + 5551CD592069475E00CEBB77 /* Paper.swift */, + 5551CD5A2069475E00CEBB77 /* Exporters */, + ); + path = Paper; + sourceTree = ""; + }; + 5551CD5A2069475E00CEBB77 /* Exporters */ = { + isa = PBXGroup; + children = ( + 5551CD5C2069475E00CEBB77 /* PaperExporter.swift */, + 5551CD5B2069475E00CEBB77 /* BitmapPaperExporter.swift */, + 5551CD5D2069475E00CEBB77 /* CVPixelBufferExporter.swift */, + ); + path = Exporters; + sourceTree = ""; + }; + 5551CD71206AF26C00CEBB77 /* Resources */ = { + isa = PBXGroup; + children = ( + 55FA9125206C11E8007FCE6A /* EmojiSketches.mlmodel */, + ); + name = Resources; + sourceTree = ""; + }; + 555C3441206F689200F74F6B /* Augmentation */ = { + isa = PBXGroup; + children = ( + 556262C32070CDB60066DF0E /* AugmentationFilter.swift */, + 555C3442206F689200F74F6B /* AugmentationSession.swift */, + ); + path = Augmentation; + sourceTree = ""; + }; + 55B6610D2070DB02006B964D /* Chapters */ = { + isa = PBXGroup; + children = ( + 55B6610E2070DB19006B964D /* Page1.playgroundpage */, + 55B6611A2070DB2F006B964D /* Page2.playgroundpage */, + 55B6612C20712245006B964D /* Page3.playgroundpage */, + ); + name = Chapters; + sourceTree = ""; + }; + 55B6610E2070DB19006B964D /* Page1.playgroundpage */ = { + isa = PBXGroup; + children = ( + 55B661132070DB19006B964D /* Contents.swift */, + 55B661192070DB19006B964D /* LiveView.swift */, + 55B6610F2070DB19006B964D /* Manifest.plist */, + 55B661102070DB19006B964D /* PrivateResources */, + 55B661142070DB19006B964D /* Sources */, + ); + name = Page1.playgroundpage; + path = ../../MLMOJI.playgroundbook/Contents/Chapters/Chapter1.playgroundchapter/Pages/Page1.playgroundpage; + sourceTree = ""; + }; + 55B661102070DB19006B964D /* PrivateResources */ = { + isa = PBXGroup; + children = ( + 55B661112070DB19006B964D /* EmojiStrokes@2x.png */, + 55B661122070DB19006B964D /* SupportedEmoji@2x.png */, + ); + path = PrivateResources; + sourceTree = ""; + }; + 55B661142070DB19006B964D /* Sources */ = { + isa = PBXGroup; + children = ( + 55B661152070DB19006B964D /* PredictionResultNode.swift */, + 55B661172070DB19006B964D /* PredictionResultsContainer.swift */, + 55B661182070DB19006B964D /* PredictionViewController.swift */, + 55B661162070DB19006B964D /* PredictionViewController+Playground.swift */, + ); + path = Sources; + sourceTree = ""; + }; + 55B6611A2070DB2F006B964D /* Page2.playgroundpage */ = { + isa = PBXGroup; + children = ( + 55B6611E2070DB2F006B964D /* Contents.swift */, + 55B6611B2070DB2F006B964D /* Manifest.plist */, + 552FFBC92072051200B0E10E /* PrivateResources */, + 55B6611F2070DB2F006B964D /* Sources */, + ); + name = Page2.playgroundpage; + path = ../../MLMOJI.playgroundbook/Contents/Chapters/Chapter1.playgroundchapter/Pages/Page2.playgroundpage; + sourceTree = ""; + }; + 55B6611F2070DB2F006B964D /* Sources */ = { + isa = PBXGroup; + children = ( + 55B661222070DB2F006B964D /* AugmentViewController.swift */, + 55B661212070DB2F006B964D /* ImageCollectionViewCell.swift */, + 55B661202070DB2F006B964D /* AugmentViewController+Playground.swift */, + ); + path = Sources; + sourceTree = ""; + }; + 55B6612C20712245006B964D /* Page3.playgroundpage */ = { + isa = PBXGroup; + children = ( + 55B6613020712245006B964D /* Contents.swift */, + 55B6613520712245006B964D /* LiveView.swift */, + 55B6612D20712245006B964D /* Manifest.plist */, + ); + name = Page3.playgroundpage; + path = ../../MLMOJI.playgroundbook/Contents/Chapters/Chapter1.playgroundchapter/Pages/Page3.playgroundpage; + sourceTree = ""; + }; + 55FA9116206C0F00007FCE6A /* Prediction */ = { + isa = PBXGroup; + children = ( + 5551CD612069475E00CEBB77 /* Classes.swift */, + 55FA9117206C0F3E007FCE6A /* EmojiSketches.swift */, + 55FA9119206C0F6F007FCE6A /* PredictionSession.swift */, + 55FA9127206D2E04007FCE6A /* PredictionSessionDelegate.swift */, + 556155F720722E2600AFA4C8 /* EmojiPrediction.swift */, + ); + path = Prediction; + sourceTree = ""; + }; + 55FA9129206D323C007FCE6A /* Utilities */ = { + isa = PBXGroup; + children = ( + 55FA9136206E02A4007FCE6A /* Animation.swift */, + 55FA912A206D3250007FCE6A /* AutoLayout+Pin.swift */, + ); + path = Utilities; + sourceTree = ""; + }; +/* End PBXGroup section */ + +/* Begin PBXNativeTarget section */ + 5551CD41206946C200CEBB77 /* PlaygroundStub */ = { + isa = PBXNativeTarget; + buildConfigurationList = 5551CD54206946C200CEBB77 /* Build configuration list for PBXNativeTarget "PlaygroundStub" */; + buildPhases = ( + 5551CD3E206946C200CEBB77 /* Sources */, + 5551CD3F206946C200CEBB77 /* Frameworks */, + 5551CD40206946C200CEBB77 /* Resources */, + ); + buildRules = ( + ); + dependencies = ( + ); + name = PlaygroundStub; + productName = PlaygroundStub; + productReference = 5551CD42206946C200CEBB77 /* PlaygroundStub.app */; + productType = "com.apple.product-type.application"; + }; +/* End PBXNativeTarget section */ + +/* Begin PBXProject section */ + 5551CD3A206946C200CEBB77 /* Project object */ = { + isa = PBXProject; + attributes = { + LastSwiftUpdateCheck = 0920; + LastUpgradeCheck = 0930; + ORGANIZATIONNAME = "Alexis Aubry"; + TargetAttributes = { + 5551CD41206946C200CEBB77 = { + CreatedOnToolsVersion = 9.2; + ProvisioningStyle = Automatic; + }; + }; + }; + buildConfigurationList = 5551CD3D206946C200CEBB77 /* Build configuration list for PBXProject "PlaygroundStub" */; + compatibilityVersion = "Xcode 8.0"; + developmentRegion = en; + hasScannedForEncodings = 0; + knownRegions = ( + en, + Base, + ); + mainGroup = 5551CD39206946C200CEBB77; + productRefGroup = 5551CD43206946C200CEBB77 /* Products */; + projectDirPath = ""; + projectRoot = ""; + targets = ( + 5551CD41206946C200CEBB77 /* PlaygroundStub */, + ); + }; +/* End PBXProject section */ + +/* Begin PBXResourcesBuildPhase section */ + 5551CD40206946C200CEBB77 /* Resources */ = { + isa = PBXResourcesBuildPhase; + buildActionMask = 2147483647; + files = ( + 5551CD50206946C200CEBB77 /* LaunchScreen.storyboard in Resources */, + 55B6613620712245006B964D /* Manifest.plist in Resources */, + 5551CD4D206946C200CEBB77 /* Assets.xcassets in Resources */, + 5551CD4B206946C200CEBB77 /* Main.storyboard in Resources */, + 552FFBCB2072051300B0E10E /* AugmentedSet@2x.png in Resources */, + 55B661232070DB2F006B964D /* Manifest.plist in Resources */, + ); + runOnlyForDeploymentPostprocessing = 0; + }; +/* End PBXResourcesBuildPhase section */ + +/* Begin PBXSourcesBuildPhase section */ + 5551CD3E206946C200CEBB77 /* Sources */ = { + isa = PBXSourcesBuildPhase; + buildActionMask = 2147483647; + files = ( + 55B661282070DB2F006B964D /* AugmentViewController.swift in Sources */, + 55B661272070DB2F006B964D /* ImageCollectionViewCell.swift in Sources */, + 556155F820722E2600AFA4C8 /* EmojiPrediction.swift in Sources */, + 55FA912B206D3250007FCE6A /* AutoLayout+Pin.swift in Sources */, + 55B6612A2070DC63006B964D /* PredictionResultsContainer.swift in Sources */, + 552FFBCF20722CE600B0E10E /* AugmentationStubViewController.swift in Sources */, + 55FA9126206C11E8007FCE6A /* EmojiSketches.mlmodel in Sources */, + 5551CD46206946C200CEBB77 /* AppDelegate.swift in Sources */, + 556262C42070CDB60066DF0E /* AugmentationFilter.swift in Sources */, + 5551CD652069475E00CEBB77 /* CVPixelBufferExporter.swift in Sources */, + 555C3443206F689300F74F6B /* AugmentationSession.swift in Sources */, + 5551CD632069475E00CEBB77 /* BitmapPaperExporter.swift in Sources */, + 5551CD682069475E00CEBB77 /* Classes.swift in Sources */, + 55FA911A206C0F6F007FCE6A /* PredictionSession.swift in Sources */, + 5551CD622069475E00CEBB77 /* Paper.swift in Sources */, + 5551CD642069475E00CEBB77 /* PaperExporter.swift in Sources */, + 552FFBCD20722C0B00B0E10E /* PredictionStubViewController.swift in Sources */, + 55FA9137206E02A4007FCE6A /* Animation.swift in Sources */, + 55B6612B2070DC63006B964D /* PredictionViewController.swift in Sources */, + 55FA9128206D2E04007FCE6A /* PredictionSessionDelegate.swift in Sources */, + 55B661292070DC63006B964D /* PredictionResultNode.swift in Sources */, + ); + runOnlyForDeploymentPostprocessing = 0; + }; +/* End PBXSourcesBuildPhase section */ + +/* Begin PBXVariantGroup section */ + 5551CD49206946C200CEBB77 /* Main.storyboard */ = { + isa = PBXVariantGroup; + children = ( + 5551CD4A206946C200CEBB77 /* Base */, + ); + name = Main.storyboard; + sourceTree = ""; + }; + 5551CD4E206946C200CEBB77 /* LaunchScreen.storyboard */ = { + isa = PBXVariantGroup; + children = ( + 5551CD4F206946C200CEBB77 /* Base */, + ); + name = LaunchScreen.storyboard; + sourceTree = ""; + }; +/* End PBXVariantGroup section */ + +/* Begin XCBuildConfiguration section */ + 5551CD52206946C200CEBB77 /* Debug */ = { + isa = XCBuildConfiguration; + buildSettings = { + ALWAYS_SEARCH_USER_PATHS = NO; + CLANG_ANALYZER_NONNULL = YES; + CLANG_ANALYZER_NUMBER_OBJECT_CONVERSION = YES_AGGRESSIVE; + CLANG_CXX_LANGUAGE_STANDARD = "gnu++14"; + CLANG_CXX_LIBRARY = "libc++"; + CLANG_ENABLE_MODULES = YES; + CLANG_ENABLE_OBJC_ARC = YES; + CLANG_WARN_BLOCK_CAPTURE_AUTORELEASING = YES; + CLANG_WARN_BOOL_CONVERSION = YES; + CLANG_WARN_COMMA = YES; + CLANG_WARN_CONSTANT_CONVERSION = YES; + CLANG_WARN_DEPRECATED_OBJC_IMPLEMENTATIONS = YES; + CLANG_WARN_DIRECT_OBJC_ISA_USAGE = YES_ERROR; + CLANG_WARN_DOCUMENTATION_COMMENTS = YES; + CLANG_WARN_EMPTY_BODY = YES; + CLANG_WARN_ENUM_CONVERSION = YES; + CLANG_WARN_INFINITE_RECURSION = YES; + CLANG_WARN_INT_CONVERSION = YES; + CLANG_WARN_NON_LITERAL_NULL_CONVERSION = YES; + CLANG_WARN_OBJC_IMPLICIT_RETAIN_SELF = YES; + CLANG_WARN_OBJC_LITERAL_CONVERSION = YES; + CLANG_WARN_OBJC_ROOT_CLASS = YES_ERROR; + CLANG_WARN_RANGE_LOOP_ANALYSIS = YES; + CLANG_WARN_STRICT_PROTOTYPES = YES; + CLANG_WARN_SUSPICIOUS_MOVE = YES; + CLANG_WARN_UNGUARDED_AVAILABILITY = YES_AGGRESSIVE; + CLANG_WARN_UNREACHABLE_CODE = YES; + CLANG_WARN__DUPLICATE_METHOD_MATCH = YES; + CODE_SIGN_IDENTITY = "iPhone Developer"; + COPY_PHASE_STRIP = NO; + DEBUG_INFORMATION_FORMAT = dwarf; + ENABLE_STRICT_OBJC_MSGSEND = YES; + ENABLE_TESTABILITY = YES; + GCC_C_LANGUAGE_STANDARD = gnu11; + GCC_DYNAMIC_NO_PIC = NO; + GCC_NO_COMMON_BLOCKS = YES; + GCC_OPTIMIZATION_LEVEL = 0; + GCC_PREPROCESSOR_DEFINITIONS = ( + "DEBUG=1", + "$(inherited)", + ); + GCC_WARN_64_TO_32_BIT_CONVERSION = YES; + GCC_WARN_ABOUT_RETURN_TYPE = YES_ERROR; + GCC_WARN_UNDECLARED_SELECTOR = YES; + GCC_WARN_UNINITIALIZED_AUTOS = YES_AGGRESSIVE; + GCC_WARN_UNUSED_FUNCTION = YES; + GCC_WARN_UNUSED_VARIABLE = YES; + IPHONEOS_DEPLOYMENT_TARGET = 11.2; + MTL_ENABLE_DEBUG_INFO = YES; + ONLY_ACTIVE_ARCH = YES; + SDKROOT = iphoneos; + SWIFT_ACTIVE_COMPILATION_CONDITIONS = DEBUG; + SWIFT_OPTIMIZATION_LEVEL = "-Onone"; + }; + name = Debug; + }; + 5551CD53206946C200CEBB77 /* Release */ = { + isa = XCBuildConfiguration; + buildSettings = { + ALWAYS_SEARCH_USER_PATHS = NO; + CLANG_ANALYZER_NONNULL = YES; + CLANG_ANALYZER_NUMBER_OBJECT_CONVERSION = YES_AGGRESSIVE; + CLANG_CXX_LANGUAGE_STANDARD = "gnu++14"; + CLANG_CXX_LIBRARY = "libc++"; + CLANG_ENABLE_MODULES = YES; + CLANG_ENABLE_OBJC_ARC = YES; + CLANG_WARN_BLOCK_CAPTURE_AUTORELEASING = YES; + CLANG_WARN_BOOL_CONVERSION = YES; + CLANG_WARN_COMMA = YES; + CLANG_WARN_CONSTANT_CONVERSION = YES; + CLANG_WARN_DEPRECATED_OBJC_IMPLEMENTATIONS = YES; + CLANG_WARN_DIRECT_OBJC_ISA_USAGE = YES_ERROR; + CLANG_WARN_DOCUMENTATION_COMMENTS = YES; + CLANG_WARN_EMPTY_BODY = YES; + CLANG_WARN_ENUM_CONVERSION = YES; + CLANG_WARN_INFINITE_RECURSION = YES; + CLANG_WARN_INT_CONVERSION = YES; + CLANG_WARN_NON_LITERAL_NULL_CONVERSION = YES; + CLANG_WARN_OBJC_IMPLICIT_RETAIN_SELF = YES; + CLANG_WARN_OBJC_LITERAL_CONVERSION = YES; + CLANG_WARN_OBJC_ROOT_CLASS = YES_ERROR; + CLANG_WARN_RANGE_LOOP_ANALYSIS = YES; + CLANG_WARN_STRICT_PROTOTYPES = YES; + CLANG_WARN_SUSPICIOUS_MOVE = YES; + CLANG_WARN_UNGUARDED_AVAILABILITY = YES_AGGRESSIVE; + CLANG_WARN_UNREACHABLE_CODE = YES; + CLANG_WARN__DUPLICATE_METHOD_MATCH = YES; + CODE_SIGN_IDENTITY = "iPhone Developer"; + COPY_PHASE_STRIP = NO; + DEBUG_INFORMATION_FORMAT = "dwarf-with-dsym"; + ENABLE_NS_ASSERTIONS = NO; + ENABLE_STRICT_OBJC_MSGSEND = YES; + GCC_C_LANGUAGE_STANDARD = gnu11; + GCC_NO_COMMON_BLOCKS = YES; + GCC_WARN_64_TO_32_BIT_CONVERSION = YES; + GCC_WARN_ABOUT_RETURN_TYPE = YES_ERROR; + GCC_WARN_UNDECLARED_SELECTOR = YES; + GCC_WARN_UNINITIALIZED_AUTOS = YES_AGGRESSIVE; + GCC_WARN_UNUSED_FUNCTION = YES; + GCC_WARN_UNUSED_VARIABLE = YES; + IPHONEOS_DEPLOYMENT_TARGET = 11.2; + MTL_ENABLE_DEBUG_INFO = NO; + SDKROOT = iphoneos; + SWIFT_OPTIMIZATION_LEVEL = "-Owholemodule"; + VALIDATE_PRODUCT = YES; + }; + name = Release; + }; + 5551CD55206946C200CEBB77 /* Debug */ = { + isa = XCBuildConfiguration; + buildSettings = { + ASSETCATALOG_COMPILER_APPICON_NAME = AppIcon; + CODE_SIGN_STYLE = Automatic; + DEVELOPMENT_TEAM = 75E8K39Z38; + INFOPLIST_FILE = PlaygroundStub/Info.plist; + LD_RUNPATH_SEARCH_PATHS = "$(inherited) @executable_path/Frameworks"; + PRODUCT_BUNDLE_IDENTIFIER = fr.alexaubry.wwdc18.PlaygroundStub; + PRODUCT_NAME = "$(TARGET_NAME)"; + SWIFT_VERSION = 4.0; + TARGETED_DEVICE_FAMILY = "1,2"; + }; + name = Debug; + }; + 5551CD56206946C200CEBB77 /* Release */ = { + isa = XCBuildConfiguration; + buildSettings = { + ASSETCATALOG_COMPILER_APPICON_NAME = AppIcon; + CODE_SIGN_STYLE = Automatic; + DEVELOPMENT_TEAM = 75E8K39Z38; + INFOPLIST_FILE = PlaygroundStub/Info.plist; + LD_RUNPATH_SEARCH_PATHS = "$(inherited) @executable_path/Frameworks"; + PRODUCT_BUNDLE_IDENTIFIER = fr.alexaubry.wwdc18.PlaygroundStub; + PRODUCT_NAME = "$(TARGET_NAME)"; + SWIFT_VERSION = 4.0; + TARGETED_DEVICE_FAMILY = "1,2"; + }; + name = Release; + }; +/* End XCBuildConfiguration section */ + +/* Begin XCConfigurationList section */ + 5551CD3D206946C200CEBB77 /* Build configuration list for PBXProject "PlaygroundStub" */ = { + isa = XCConfigurationList; + buildConfigurations = ( + 5551CD52206946C200CEBB77 /* Debug */, + 5551CD53206946C200CEBB77 /* Release */, + ); + defaultConfigurationIsVisible = 0; + defaultConfigurationName = Release; + }; + 5551CD54206946C200CEBB77 /* Build configuration list for PBXNativeTarget "PlaygroundStub" */ = { + isa = XCConfigurationList; + buildConfigurations = ( + 5551CD55206946C200CEBB77 /* Debug */, + 5551CD56206946C200CEBB77 /* Release */, + ); + defaultConfigurationIsVisible = 0; + defaultConfigurationName = Release; + }; +/* End XCConfigurationList section */ + }; + rootObject = 5551CD3A206946C200CEBB77 /* Project object */; +} diff --git a/Playground/PlaygroundStub/PlaygroundStub.xcodeproj/project.xcworkspace/contents.xcworkspacedata b/Playground/PlaygroundStub/PlaygroundStub.xcodeproj/project.xcworkspace/contents.xcworkspacedata new file mode 100644 index 0000000..e336d4b --- /dev/null +++ b/Playground/PlaygroundStub/PlaygroundStub.xcodeproj/project.xcworkspace/contents.xcworkspacedata @@ -0,0 +1,7 @@ + + + + + diff --git a/Playground/PlaygroundStub/PlaygroundStub/AppDelegate.swift b/Playground/PlaygroundStub/PlaygroundStub/AppDelegate.swift new file mode 100644 index 0000000..7d6ceb0 --- /dev/null +++ b/Playground/PlaygroundStub/PlaygroundStub/AppDelegate.swift @@ -0,0 +1,20 @@ +// +// MLMOJI +// +// This file is part of Alexis Aubry's WWDC18 scolarship submission open source project. +// Copyright © 2018 Alexis Aubry. Available under the terms of the MIT License. +// + +import UIKit + +@UIApplicationMain +class AppDelegate: UIResponder, UIApplicationDelegate { + + var window: UIWindow? + + func application(_ application: UIApplication, didFinishLaunchingWithOptions launchOptions: [UIApplicationLaunchOptionsKey: Any]?) -> Bool { + return true + } + +} + diff --git a/Playground/PlaygroundStub/PlaygroundStub/Assets.xcassets/AppIcon.appiconset/Contents.json b/Playground/PlaygroundStub/PlaygroundStub/Assets.xcassets/AppIcon.appiconset/Contents.json new file mode 100644 index 0000000..d8db8d6 --- /dev/null +++ b/Playground/PlaygroundStub/PlaygroundStub/Assets.xcassets/AppIcon.appiconset/Contents.json @@ -0,0 +1,98 @@ +{ + "images" : [ + { + "idiom" : "iphone", + "size" : "20x20", + "scale" : "2x" + }, + { + "idiom" : "iphone", + "size" : "20x20", + "scale" : "3x" + }, + { + "idiom" : "iphone", + "size" : "29x29", + "scale" : "2x" + }, + { + "idiom" : "iphone", + "size" : "29x29", + "scale" : "3x" + }, + { + "idiom" : "iphone", + "size" : "40x40", + "scale" : "2x" + }, + { + "idiom" : "iphone", + "size" : "40x40", + "scale" : "3x" + }, + { + "idiom" : "iphone", + "size" : "60x60", + "scale" : "2x" + }, + { + "idiom" : "iphone", + "size" : "60x60", + "scale" : "3x" + }, + { + "idiom" : "ipad", + "size" : "20x20", + "scale" : "1x" + }, + { + "idiom" : "ipad", + "size" : "20x20", + "scale" : "2x" + }, + { + "idiom" : "ipad", + "size" : "29x29", + "scale" : "1x" + }, + { + "idiom" : "ipad", + "size" : "29x29", + "scale" : "2x" + }, + { + "idiom" : "ipad", + "size" : "40x40", + "scale" : "1x" + }, + { + "idiom" : "ipad", + "size" : "40x40", + "scale" : "2x" + }, + { + "idiom" : "ipad", + "size" : "76x76", + "scale" : "1x" + }, + { + "idiom" : "ipad", + "size" : "76x76", + "scale" : "2x" + }, + { + "idiom" : "ipad", + "size" : "83.5x83.5", + "scale" : "2x" + }, + { + "idiom" : "ios-marketing", + "size" : "1024x1024", + "scale" : "1x" + } + ], + "info" : { + "version" : 1, + "author" : "xcode" + } +} \ No newline at end of file diff --git a/Playground/PlaygroundStub/PlaygroundStub/Assets.xcassets/ArtificalBackground.imageset/ArtificialBackground.jpeg b/Playground/PlaygroundStub/PlaygroundStub/Assets.xcassets/ArtificalBackground.imageset/ArtificialBackground.jpeg new file mode 100644 index 0000000..1abd4df Binary files /dev/null and b/Playground/PlaygroundStub/PlaygroundStub/Assets.xcassets/ArtificalBackground.imageset/ArtificialBackground.jpeg differ diff --git a/Playground/PlaygroundStub/PlaygroundStub/Assets.xcassets/ArtificalBackground.imageset/Contents.json b/Playground/PlaygroundStub/PlaygroundStub/Assets.xcassets/ArtificalBackground.imageset/Contents.json new file mode 100644 index 0000000..7898dee --- /dev/null +++ b/Playground/PlaygroundStub/PlaygroundStub/Assets.xcassets/ArtificalBackground.imageset/Contents.json @@ -0,0 +1,21 @@ +{ + "images" : [ + { + "idiom" : "universal", + "scale" : "1x" + }, + { + "idiom" : "universal", + "filename" : "ArtificialBackground.jpeg", + "scale" : "2x" + }, + { + "idiom" : "universal", + "scale" : "3x" + } + ], + "info" : { + "version" : 1, + "author" : "xcode" + } +} \ No newline at end of file diff --git a/Playground/PlaygroundStub/PlaygroundStub/Assets.xcassets/Contents.json b/Playground/PlaygroundStub/PlaygroundStub/Assets.xcassets/Contents.json new file mode 100644 index 0000000..da4a164 --- /dev/null +++ b/Playground/PlaygroundStub/PlaygroundStub/Assets.xcassets/Contents.json @@ -0,0 +1,6 @@ +{ + "info" : { + "version" : 1, + "author" : "xcode" + } +} \ No newline at end of file diff --git a/Playground/PlaygroundStub/PlaygroundStub/Assets.xcassets/KeyboardClear.imageset/Contents.json b/Playground/PlaygroundStub/PlaygroundStub/Assets.xcassets/KeyboardClear.imageset/Contents.json new file mode 100644 index 0000000..7329471 --- /dev/null +++ b/Playground/PlaygroundStub/PlaygroundStub/Assets.xcassets/KeyboardClear.imageset/Contents.json @@ -0,0 +1,21 @@ +{ + "images" : [ + { + "idiom" : "universal", + "scale" : "1x" + }, + { + "idiom" : "universal", + "filename" : "KeyboardClear@2x.png", + "scale" : "2x" + }, + { + "idiom" : "universal", + "scale" : "3x" + } + ], + "info" : { + "version" : 1, + "author" : "xcode" + } +} \ No newline at end of file diff --git a/Playground/PlaygroundStub/PlaygroundStub/Assets.xcassets/KeyboardClear.imageset/KeyboardClear@2x.png b/Playground/PlaygroundStub/PlaygroundStub/Assets.xcassets/KeyboardClear.imageset/KeyboardClear@2x.png new file mode 100644 index 0000000..58497ec Binary files /dev/null and b/Playground/PlaygroundStub/PlaygroundStub/Assets.xcassets/KeyboardClear.imageset/KeyboardClear@2x.png differ diff --git a/Playground/PlaygroundStub/PlaygroundStub/AugmentationStubViewController.swift b/Playground/PlaygroundStub/PlaygroundStub/AugmentationStubViewController.swift new file mode 100644 index 0000000..a036ae3 --- /dev/null +++ b/Playground/PlaygroundStub/PlaygroundStub/AugmentationStubViewController.swift @@ -0,0 +1,31 @@ +// +// MLMOJI +// +// This file is part of Alexis Aubry's WWDC18 scolarship submission open source project. +// Copyright © 2018 Alexis Aubry. Available under the terms of the MIT License. +// + +import UIKit + +class AugmentationStubViewController: UIViewController { + + let filters: [AugmentationFilter] = [ + .translate(-15, 69), .rotate(45), .rotate(18), .scale(150), .blur + ] + + override func viewDidLoad() { + super.viewDidLoad() + view.backgroundColor = .black + + let augmentationVC = AugmentViewController(filters: filters) + augmentationVC.containerLayoutGuide = view.safeAreaLayoutGuide + + addChildViewController(augmentationVC) + view.addSubview(augmentationVC.view) + augmentationVC.configureConstraints() + + view.backgroundColor = UIColor(patternImage: #imageLiteral(resourceName: "ArtificalBackground")) + augmentationVC.view.pinEdges(to: self.view) + } + +} diff --git a/Playground/PlaygroundStub/PlaygroundStub/Base.lproj/LaunchScreen.storyboard b/Playground/PlaygroundStub/PlaygroundStub/Base.lproj/LaunchScreen.storyboard new file mode 100644 index 0000000..f83f6fd --- /dev/null +++ b/Playground/PlaygroundStub/PlaygroundStub/Base.lproj/LaunchScreen.storyboard @@ -0,0 +1,25 @@ + + + + + + + + + + + + + + + + + + + + + + + + + diff --git a/Playground/PlaygroundStub/PlaygroundStub/Base.lproj/Main.storyboard b/Playground/PlaygroundStub/PlaygroundStub/Base.lproj/Main.storyboard new file mode 100644 index 0000000..e5d7ad3 --- /dev/null +++ b/Playground/PlaygroundStub/PlaygroundStub/Base.lproj/Main.storyboard @@ -0,0 +1,104 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + diff --git a/Playground/PlaygroundStub/PlaygroundStub/EmojiSketches.mlmodelc/coremldata.bin b/Playground/PlaygroundStub/PlaygroundStub/EmojiSketches.mlmodelc/coremldata.bin new file mode 100644 index 0000000..61114e9 Binary files /dev/null and b/Playground/PlaygroundStub/PlaygroundStub/EmojiSketches.mlmodelc/coremldata.bin differ diff --git a/Playground/PlaygroundStub/PlaygroundStub/EmojiSketches.mlmodelc/model.espresso.net b/Playground/PlaygroundStub/PlaygroundStub/EmojiSketches.mlmodelc/model.espresso.net new file mode 100644 index 0000000..5d2e3a0 --- /dev/null +++ b/Playground/PlaygroundStub/PlaygroundStub/EmojiSketches.mlmodelc/model.espresso.net @@ -0,0 +1,1852 @@ +{ + "storage" : "model.espresso.weights", + "properties" : { + + }, + "format_version" : 200, + "layers" : [ + { + "pad_r" : 0, + "fused_relu" : 1, + "fused_tanh" : 0, + "pad_fill_mode" : 0, + "pad_b" : 0, + "pad_l" : 0, + "top" : "relu_MobilenetV1\/MobilenetV1\/Conv2d_0\/Relu6:0", + "blob_weights" : 3, + "K" : 3, + "blob_biases" : 1, + "stride_x" : 2, + "name" : "MobilenetV1\/MobilenetV1\/Conv2d_0\/Conv2D:0", + "has_batch_norm" : 0, + "type" : "convolution", + "n_groups" : 1, + "pad_t" : 0, + "stride_y" : 2, + "has_biases" : 1, + "C" : 32, + "bottom" : "input__0", + "weights" : { + + }, + "Nx" : 3, + "pad_mode" : 1, + "pad_value" : 0, + "Ny" : 3, + "n_parallel" : 1 + }, + { + "alpha" : -1, + "bottom" : "relu_MobilenetV1\/MobilenetV1\/Conv2d_0\/Relu6:0", + "weights" : { + + }, + "mode" : 6, + "top" : "relu_MobilenetV1\/MobilenetV1\/Conv2d_0\/Relu6:0_neg", + "type" : "activation", + "name" : "relu_MobilenetV1\/MobilenetV1\/Conv2d_0\/Relu6:0_neg", + "beta" : 0 + }, + { + "alpha" : -6, + "bottom" : "relu_MobilenetV1\/MobilenetV1\/Conv2d_0\/Relu6:0_neg", + "operation" : 25, + "weights" : { + + }, + "fused_relu" : 0, + "top" : "relu_MobilenetV1\/MobilenetV1\/Conv2d_0\/Relu6:0_clip", + "type" : "elementwise", + "name" : "relu_MobilenetV1\/MobilenetV1\/Conv2d_0\/Relu6:0_clip", + "beta" : 0 + }, + { + "alpha" : -1, + "bottom" : "relu_MobilenetV1\/MobilenetV1\/Conv2d_0\/Relu6:0_clip", + "weights" : { + + }, + "mode" : 6, + "top" : "MobilenetV1\/MobilenetV1\/Conv2d_0\/Relu6:0", + "type" : "activation", + "name" : "MobilenetV1\/MobilenetV1\/Conv2d_0\/Relu6:0", + "beta" : 0 + }, + { + "pad_r" : 0, + "fused_relu" : 1, + "fused_tanh" : 0, + "pad_fill_mode" : 0, + "pad_b" : 0, + "pad_l" : 0, + "top" : "relu_MobilenetV1\/MobilenetV1\/Conv2d_1_depthwise\/Relu6:0", + "blob_weights" : 7, + "K" : 32, + "blob_biases" : 5, + "name" : "MobilenetV1\/MobilenetV1\/Conv2d_1_depthwise\/depthwise:0", + "has_batch_norm" : 0, + "type" : "convolution", + "n_groups" : 32, + "pad_t" : 0, + "has_biases" : 1, + "C" : 32, + "bottom" : "MobilenetV1\/MobilenetV1\/Conv2d_0\/Relu6:0", + "weights" : { + + }, + "pad_mode" : 1, + "Nx" : 3, + "pad_value" : 0, + "Ny" : 3, + "n_parallel" : 1 + }, + { + "alpha" : -1, + "bottom" : "relu_MobilenetV1\/MobilenetV1\/Conv2d_1_depthwise\/Relu6:0", + "weights" : { + + }, + "mode" : 6, + "top" : "relu_MobilenetV1\/MobilenetV1\/Conv2d_1_depthwise\/Relu6:0_neg", + "type" : "activation", + "name" : "relu_MobilenetV1\/MobilenetV1\/Conv2d_1_depthwise\/Relu6:0_neg", + "beta" : 0 + }, + { + "alpha" : -6, + "bottom" : "relu_MobilenetV1\/MobilenetV1\/Conv2d_1_depthwise\/Relu6:0_neg", + "operation" : 25, + "weights" : { + + }, + "fused_relu" : 0, + "top" : "relu_MobilenetV1\/MobilenetV1\/Conv2d_1_depthwise\/Relu6:0_clip", + "type" : "elementwise", + "name" : "relu_MobilenetV1\/MobilenetV1\/Conv2d_1_depthwise\/Relu6:0_clip", + "beta" : 0 + }, + { + "alpha" : -1, + "bottom" : "relu_MobilenetV1\/MobilenetV1\/Conv2d_1_depthwise\/Relu6:0_clip", + "weights" : { + + }, + "mode" : 6, + "top" : "MobilenetV1\/MobilenetV1\/Conv2d_1_depthwise\/Relu6:0", + "type" : "activation", + "name" : "MobilenetV1\/MobilenetV1\/Conv2d_1_depthwise\/Relu6:0", + "beta" : 0 + }, + { + "pad_r" : 0, + "fused_relu" : 1, + "fused_tanh" : 0, + "pad_fill_mode" : 0, + "pad_b" : 0, + "pad_l" : 0, + "top" : "relu_MobilenetV1\/MobilenetV1\/Conv2d_1_pointwise\/Relu6:0", + "blob_weights" : 11, + "K" : 32, + "blob_biases" : 9, + "name" : "MobilenetV1\/MobilenetV1\/Conv2d_1_pointwise\/Conv2D:0", + "has_batch_norm" : 0, + "type" : "convolution", + "n_groups" : 1, + "pad_t" : 0, + "has_biases" : 1, + "C" : 64, + "bottom" : "MobilenetV1\/MobilenetV1\/Conv2d_1_depthwise\/Relu6:0", + "weights" : { + + }, + "pad_mode" : 1, + "Nx" : 1, + "pad_value" : 0, + "Ny" : 1, + "n_parallel" : 1 + }, + { + "alpha" : -1, + "bottom" : "relu_MobilenetV1\/MobilenetV1\/Conv2d_1_pointwise\/Relu6:0", + "weights" : { + + }, + "mode" : 6, + "top" : "relu_MobilenetV1\/MobilenetV1\/Conv2d_1_pointwise\/Relu6:0_neg", + "type" : "activation", + "name" : "relu_MobilenetV1\/MobilenetV1\/Conv2d_1_pointwise\/Relu6:0_neg", + "beta" : 0 + }, + { + "alpha" : -6, + "bottom" : "relu_MobilenetV1\/MobilenetV1\/Conv2d_1_pointwise\/Relu6:0_neg", + "operation" : 25, + "weights" : { + + }, + "fused_relu" : 0, + "top" : "relu_MobilenetV1\/MobilenetV1\/Conv2d_1_pointwise\/Relu6:0_clip", + "type" : "elementwise", + "name" : "relu_MobilenetV1\/MobilenetV1\/Conv2d_1_pointwise\/Relu6:0_clip", + "beta" : 0 + }, + { + "alpha" : -1, + "bottom" : "relu_MobilenetV1\/MobilenetV1\/Conv2d_1_pointwise\/Relu6:0_clip", + "weights" : { + + }, + "mode" : 6, + "top" : "MobilenetV1\/MobilenetV1\/Conv2d_1_pointwise\/Relu6:0", + "type" : "activation", + "name" : "MobilenetV1\/MobilenetV1\/Conv2d_1_pointwise\/Relu6:0", + "beta" : 0 + }, + { + "pad_r" : 0, + "fused_relu" : 1, + "fused_tanh" : 0, + "pad_fill_mode" : 0, + "pad_b" : 0, + "pad_l" : 0, + "top" : "relu_MobilenetV1\/MobilenetV1\/Conv2d_2_depthwise\/Relu6:0", + "blob_weights" : 15, + "K" : 64, + "blob_biases" : 13, + "stride_x" : 2, + "name" : "MobilenetV1\/MobilenetV1\/Conv2d_2_depthwise\/depthwise:0", + "has_batch_norm" : 0, + "type" : "convolution", + "n_groups" : 64, + "pad_t" : 0, + "stride_y" : 2, + "has_biases" : 1, + "C" : 64, + "bottom" : "MobilenetV1\/MobilenetV1\/Conv2d_1_pointwise\/Relu6:0", + "weights" : { + + }, + "Nx" : 3, + "pad_mode" : 1, + "pad_value" : 0, + "Ny" : 3, + "n_parallel" : 1 + }, + { + "alpha" : -1, + "bottom" : "relu_MobilenetV1\/MobilenetV1\/Conv2d_2_depthwise\/Relu6:0", + "weights" : { + + }, + "mode" : 6, + "top" : "relu_MobilenetV1\/MobilenetV1\/Conv2d_2_depthwise\/Relu6:0_neg", + "type" : "activation", + "name" : "relu_MobilenetV1\/MobilenetV1\/Conv2d_2_depthwise\/Relu6:0_neg", + "beta" : 0 + }, + { + "alpha" : -6, + "bottom" : "relu_MobilenetV1\/MobilenetV1\/Conv2d_2_depthwise\/Relu6:0_neg", + "operation" : 25, + "weights" : { + + }, + "fused_relu" : 0, + "top" : "relu_MobilenetV1\/MobilenetV1\/Conv2d_2_depthwise\/Relu6:0_clip", + "type" : "elementwise", + "name" : "relu_MobilenetV1\/MobilenetV1\/Conv2d_2_depthwise\/Relu6:0_clip", + "beta" : 0 + }, + { + "alpha" : -1, + "bottom" : "relu_MobilenetV1\/MobilenetV1\/Conv2d_2_depthwise\/Relu6:0_clip", + "weights" : { + + }, + "mode" : 6, + "top" : "MobilenetV1\/MobilenetV1\/Conv2d_2_depthwise\/Relu6:0", + "type" : "activation", + "name" : "MobilenetV1\/MobilenetV1\/Conv2d_2_depthwise\/Relu6:0", + "beta" : 0 + }, + { + "pad_r" : 0, + "fused_relu" : 1, + "fused_tanh" : 0, + "pad_fill_mode" : 0, + "pad_b" : 0, + "pad_l" : 0, + "top" : "relu_MobilenetV1\/MobilenetV1\/Conv2d_2_pointwise\/Relu6:0", + "blob_weights" : 19, + "K" : 64, + "blob_biases" : 17, + "name" : "MobilenetV1\/MobilenetV1\/Conv2d_2_pointwise\/Conv2D:0", + "has_batch_norm" : 0, + "type" : "convolution", + "n_groups" : 1, + "pad_t" : 0, + "has_biases" : 1, + "C" : 128, + "bottom" : "MobilenetV1\/MobilenetV1\/Conv2d_2_depthwise\/Relu6:0", + "weights" : { + + }, + "pad_mode" : 1, + "Nx" : 1, + "pad_value" : 0, + "Ny" : 1, + "n_parallel" : 1 + }, + { + "alpha" : -1, + "bottom" : "relu_MobilenetV1\/MobilenetV1\/Conv2d_2_pointwise\/Relu6:0", + "weights" : { + + }, + "mode" : 6, + "top" : "relu_MobilenetV1\/MobilenetV1\/Conv2d_2_pointwise\/Relu6:0_neg", + "type" : "activation", + "name" : "relu_MobilenetV1\/MobilenetV1\/Conv2d_2_pointwise\/Relu6:0_neg", + "beta" : 0 + }, + { + "alpha" : -6, + "bottom" : "relu_MobilenetV1\/MobilenetV1\/Conv2d_2_pointwise\/Relu6:0_neg", + "operation" : 25, + "weights" : { + + }, + "fused_relu" : 0, + "top" : "relu_MobilenetV1\/MobilenetV1\/Conv2d_2_pointwise\/Relu6:0_clip", + "type" : "elementwise", + "name" : "relu_MobilenetV1\/MobilenetV1\/Conv2d_2_pointwise\/Relu6:0_clip", + "beta" : 0 + }, + { + "alpha" : -1, + "bottom" : "relu_MobilenetV1\/MobilenetV1\/Conv2d_2_pointwise\/Relu6:0_clip", + "weights" : { + + }, + "mode" : 6, + "top" : "MobilenetV1\/MobilenetV1\/Conv2d_2_pointwise\/Relu6:0", + "type" : "activation", + "name" : "MobilenetV1\/MobilenetV1\/Conv2d_2_pointwise\/Relu6:0", + "beta" : 0 + }, + { + "pad_r" : 0, + "fused_relu" : 1, + "fused_tanh" : 0, + "pad_fill_mode" : 0, + "pad_b" : 0, + "pad_l" : 0, + "top" : "relu_MobilenetV1\/MobilenetV1\/Conv2d_3_depthwise\/Relu6:0", + "blob_weights" : 23, + "K" : 128, + "blob_biases" : 21, + "name" : "MobilenetV1\/MobilenetV1\/Conv2d_3_depthwise\/depthwise:0", + "has_batch_norm" : 0, + "type" : "convolution", + "n_groups" : 128, + "pad_t" : 0, + "has_biases" : 1, + "C" : 128, + "bottom" : "MobilenetV1\/MobilenetV1\/Conv2d_2_pointwise\/Relu6:0", + "weights" : { + + }, + "pad_mode" : 1, + "Nx" : 3, + "pad_value" : 0, + "Ny" : 3, + "n_parallel" : 1 + }, + { + "alpha" : -1, + "bottom" : "relu_MobilenetV1\/MobilenetV1\/Conv2d_3_depthwise\/Relu6:0", + "weights" : { + + }, + "mode" : 6, + "top" : "relu_MobilenetV1\/MobilenetV1\/Conv2d_3_depthwise\/Relu6:0_neg", + "type" : "activation", + "name" : "relu_MobilenetV1\/MobilenetV1\/Conv2d_3_depthwise\/Relu6:0_neg", + "beta" : 0 + }, + { + "alpha" : -6, + "bottom" : "relu_MobilenetV1\/MobilenetV1\/Conv2d_3_depthwise\/Relu6:0_neg", + "operation" : 25, + "weights" : { + + }, + "fused_relu" : 0, + "top" : "relu_MobilenetV1\/MobilenetV1\/Conv2d_3_depthwise\/Relu6:0_clip", + "type" : "elementwise", + "name" : "relu_MobilenetV1\/MobilenetV1\/Conv2d_3_depthwise\/Relu6:0_clip", + "beta" : 0 + }, + { + "alpha" : -1, + "bottom" : "relu_MobilenetV1\/MobilenetV1\/Conv2d_3_depthwise\/Relu6:0_clip", + "weights" : { + + }, + "mode" : 6, + "top" : "MobilenetV1\/MobilenetV1\/Conv2d_3_depthwise\/Relu6:0", + "type" : "activation", + "name" : "MobilenetV1\/MobilenetV1\/Conv2d_3_depthwise\/Relu6:0", + "beta" : 0 + }, + { + "pad_r" : 0, + "fused_relu" : 1, + "fused_tanh" : 0, + "pad_fill_mode" : 0, + "pad_b" : 0, + "pad_l" : 0, + "top" : "relu_MobilenetV1\/MobilenetV1\/Conv2d_3_pointwise\/Relu6:0", + "blob_weights" : 27, + "K" : 128, + "blob_biases" : 25, + "name" : "MobilenetV1\/MobilenetV1\/Conv2d_3_pointwise\/Conv2D:0", + "has_batch_norm" : 0, + "type" : "convolution", + "n_groups" : 1, + "pad_t" : 0, + "has_biases" : 1, + "C" : 128, + "bottom" : "MobilenetV1\/MobilenetV1\/Conv2d_3_depthwise\/Relu6:0", + "weights" : { + + }, + "pad_mode" : 1, + "Nx" : 1, + "pad_value" : 0, + "Ny" : 1, + "n_parallel" : 1 + }, + { + "alpha" : -1, + "bottom" : "relu_MobilenetV1\/MobilenetV1\/Conv2d_3_pointwise\/Relu6:0", + "weights" : { + + }, + "mode" : 6, + "top" : "relu_MobilenetV1\/MobilenetV1\/Conv2d_3_pointwise\/Relu6:0_neg", + "type" : "activation", + "name" : "relu_MobilenetV1\/MobilenetV1\/Conv2d_3_pointwise\/Relu6:0_neg", + "beta" : 0 + }, + { + "alpha" : -6, + "bottom" : "relu_MobilenetV1\/MobilenetV1\/Conv2d_3_pointwise\/Relu6:0_neg", + "operation" : 25, + "weights" : { + + }, + "fused_relu" : 0, + "top" : "relu_MobilenetV1\/MobilenetV1\/Conv2d_3_pointwise\/Relu6:0_clip", + "type" : "elementwise", + "name" : "relu_MobilenetV1\/MobilenetV1\/Conv2d_3_pointwise\/Relu6:0_clip", + "beta" : 0 + }, + { + "alpha" : -1, + "bottom" : "relu_MobilenetV1\/MobilenetV1\/Conv2d_3_pointwise\/Relu6:0_clip", + "weights" : { + + }, + "mode" : 6, + "top" : "MobilenetV1\/MobilenetV1\/Conv2d_3_pointwise\/Relu6:0", + "type" : "activation", + "name" : "MobilenetV1\/MobilenetV1\/Conv2d_3_pointwise\/Relu6:0", + "beta" : 0 + }, + { + "pad_r" : 0, + "fused_relu" : 1, + "fused_tanh" : 0, + "pad_fill_mode" : 0, + "pad_b" : 0, + "pad_l" : 0, + "top" : "relu_MobilenetV1\/MobilenetV1\/Conv2d_4_depthwise\/Relu6:0", + "blob_weights" : 31, + "K" : 128, + "blob_biases" : 29, + "stride_x" : 2, + "name" : "MobilenetV1\/MobilenetV1\/Conv2d_4_depthwise\/depthwise:0", + "has_batch_norm" : 0, + "type" : "convolution", + "n_groups" : 128, + "pad_t" : 0, + "stride_y" : 2, + "has_biases" : 1, + "C" : 128, + "bottom" : "MobilenetV1\/MobilenetV1\/Conv2d_3_pointwise\/Relu6:0", + "weights" : { + + }, + "Nx" : 3, + "pad_mode" : 1, + "pad_value" : 0, + "Ny" : 3, + "n_parallel" : 1 + }, + { + "alpha" : -1, + "bottom" : "relu_MobilenetV1\/MobilenetV1\/Conv2d_4_depthwise\/Relu6:0", + "weights" : { + + }, + "mode" : 6, + "top" : "relu_MobilenetV1\/MobilenetV1\/Conv2d_4_depthwise\/Relu6:0_neg", + "type" : "activation", + "name" : "relu_MobilenetV1\/MobilenetV1\/Conv2d_4_depthwise\/Relu6:0_neg", + "beta" : 0 + }, + { + "alpha" : -6, + "bottom" : "relu_MobilenetV1\/MobilenetV1\/Conv2d_4_depthwise\/Relu6:0_neg", + "operation" : 25, + "weights" : { + + }, + "fused_relu" : 0, + "top" : "relu_MobilenetV1\/MobilenetV1\/Conv2d_4_depthwise\/Relu6:0_clip", + "type" : "elementwise", + "name" : "relu_MobilenetV1\/MobilenetV1\/Conv2d_4_depthwise\/Relu6:0_clip", + "beta" : 0 + }, + { + "alpha" : -1, + "bottom" : "relu_MobilenetV1\/MobilenetV1\/Conv2d_4_depthwise\/Relu6:0_clip", + "weights" : { + + }, + "mode" : 6, + "top" : "MobilenetV1\/MobilenetV1\/Conv2d_4_depthwise\/Relu6:0", + "type" : "activation", + "name" : "MobilenetV1\/MobilenetV1\/Conv2d_4_depthwise\/Relu6:0", + "beta" : 0 + }, + { + "pad_r" : 0, + "fused_relu" : 1, + "fused_tanh" : 0, + "pad_fill_mode" : 0, + "pad_b" : 0, + "pad_l" : 0, + "top" : "relu_MobilenetV1\/MobilenetV1\/Conv2d_4_pointwise\/Relu6:0", + "blob_weights" : 35, + "K" : 128, + "blob_biases" : 33, + "name" : "MobilenetV1\/MobilenetV1\/Conv2d_4_pointwise\/Conv2D:0", + "has_batch_norm" : 0, + "type" : "convolution", + "n_groups" : 1, + "pad_t" : 0, + "has_biases" : 1, + "C" : 256, + "bottom" : "MobilenetV1\/MobilenetV1\/Conv2d_4_depthwise\/Relu6:0", + "weights" : { + + }, + "pad_mode" : 1, + "Nx" : 1, + "pad_value" : 0, + "Ny" : 1, + "n_parallel" : 1 + }, + { + "alpha" : -1, + "bottom" : "relu_MobilenetV1\/MobilenetV1\/Conv2d_4_pointwise\/Relu6:0", + "weights" : { + + }, + "mode" : 6, + "top" : "relu_MobilenetV1\/MobilenetV1\/Conv2d_4_pointwise\/Relu6:0_neg", + "type" : "activation", + "name" : "relu_MobilenetV1\/MobilenetV1\/Conv2d_4_pointwise\/Relu6:0_neg", + "beta" : 0 + }, + { + "alpha" : -6, + "bottom" : "relu_MobilenetV1\/MobilenetV1\/Conv2d_4_pointwise\/Relu6:0_neg", + "operation" : 25, + "weights" : { + + }, + "fused_relu" : 0, + "top" : "relu_MobilenetV1\/MobilenetV1\/Conv2d_4_pointwise\/Relu6:0_clip", + "type" : "elementwise", + "name" : "relu_MobilenetV1\/MobilenetV1\/Conv2d_4_pointwise\/Relu6:0_clip", + "beta" : 0 + }, + { + "alpha" : -1, + "bottom" : "relu_MobilenetV1\/MobilenetV1\/Conv2d_4_pointwise\/Relu6:0_clip", + "weights" : { + + }, + "mode" : 6, + "top" : "MobilenetV1\/MobilenetV1\/Conv2d_4_pointwise\/Relu6:0", + "type" : "activation", + "name" : "MobilenetV1\/MobilenetV1\/Conv2d_4_pointwise\/Relu6:0", + "beta" : 0 + }, + { + "pad_r" : 0, + "fused_relu" : 1, + "fused_tanh" : 0, + "pad_fill_mode" : 0, + "pad_b" : 0, + "pad_l" : 0, + "top" : "relu_MobilenetV1\/MobilenetV1\/Conv2d_5_depthwise\/Relu6:0", + "blob_weights" : 39, + "K" : 256, + "blob_biases" : 37, + "name" : "MobilenetV1\/MobilenetV1\/Conv2d_5_depthwise\/depthwise:0", + "has_batch_norm" : 0, + "type" : "convolution", + "n_groups" : 256, + "pad_t" : 0, + "has_biases" : 1, + "C" : 256, + "bottom" : "MobilenetV1\/MobilenetV1\/Conv2d_4_pointwise\/Relu6:0", + "weights" : { + + }, + "pad_mode" : 1, + "Nx" : 3, + "pad_value" : 0, + "Ny" : 3, + "n_parallel" : 1 + }, + { + "alpha" : -1, + "bottom" : "relu_MobilenetV1\/MobilenetV1\/Conv2d_5_depthwise\/Relu6:0", + "weights" : { + + }, + "mode" : 6, + "top" : "relu_MobilenetV1\/MobilenetV1\/Conv2d_5_depthwise\/Relu6:0_neg", + "type" : "activation", + "name" : "relu_MobilenetV1\/MobilenetV1\/Conv2d_5_depthwise\/Relu6:0_neg", + "beta" : 0 + }, + { + "alpha" : -6, + "bottom" : "relu_MobilenetV1\/MobilenetV1\/Conv2d_5_depthwise\/Relu6:0_neg", + "operation" : 25, + "weights" : { + + }, + "fused_relu" : 0, + "top" : "relu_MobilenetV1\/MobilenetV1\/Conv2d_5_depthwise\/Relu6:0_clip", + "type" : "elementwise", + "name" : "relu_MobilenetV1\/MobilenetV1\/Conv2d_5_depthwise\/Relu6:0_clip", + "beta" : 0 + }, + { + "alpha" : -1, + "bottom" : "relu_MobilenetV1\/MobilenetV1\/Conv2d_5_depthwise\/Relu6:0_clip", + "weights" : { + + }, + "mode" : 6, + "top" : "MobilenetV1\/MobilenetV1\/Conv2d_5_depthwise\/Relu6:0", + "type" : "activation", + "name" : "MobilenetV1\/MobilenetV1\/Conv2d_5_depthwise\/Relu6:0", + "beta" : 0 + }, + { + "pad_r" : 0, + "fused_relu" : 1, + "fused_tanh" : 0, + "pad_fill_mode" : 0, + "pad_b" : 0, + "pad_l" : 0, + "top" : "relu_MobilenetV1\/MobilenetV1\/Conv2d_5_pointwise\/Relu6:0", + "blob_weights" : 43, + "K" : 256, + "blob_biases" : 41, + "name" : "MobilenetV1\/MobilenetV1\/Conv2d_5_pointwise\/Conv2D:0", + "has_batch_norm" : 0, + "type" : "convolution", + "n_groups" : 1, + "pad_t" : 0, + "has_biases" : 1, + "C" : 256, + "bottom" : "MobilenetV1\/MobilenetV1\/Conv2d_5_depthwise\/Relu6:0", + "weights" : { + + }, + "pad_mode" : 1, + "Nx" : 1, + "pad_value" : 0, + "Ny" : 1, + "n_parallel" : 1 + }, + { + "alpha" : -1, + "bottom" : "relu_MobilenetV1\/MobilenetV1\/Conv2d_5_pointwise\/Relu6:0", + "weights" : { + + }, + "mode" : 6, + "top" : "relu_MobilenetV1\/MobilenetV1\/Conv2d_5_pointwise\/Relu6:0_neg", + "type" : "activation", + "name" : "relu_MobilenetV1\/MobilenetV1\/Conv2d_5_pointwise\/Relu6:0_neg", + "beta" : 0 + }, + { + "alpha" : -6, + "bottom" : "relu_MobilenetV1\/MobilenetV1\/Conv2d_5_pointwise\/Relu6:0_neg", + "operation" : 25, + "weights" : { + + }, + "fused_relu" : 0, + "top" : "relu_MobilenetV1\/MobilenetV1\/Conv2d_5_pointwise\/Relu6:0_clip", + "type" : "elementwise", + "name" : "relu_MobilenetV1\/MobilenetV1\/Conv2d_5_pointwise\/Relu6:0_clip", + "beta" : 0 + }, + { + "alpha" : -1, + "bottom" : "relu_MobilenetV1\/MobilenetV1\/Conv2d_5_pointwise\/Relu6:0_clip", + "weights" : { + + }, + "mode" : 6, + "top" : "MobilenetV1\/MobilenetV1\/Conv2d_5_pointwise\/Relu6:0", + "type" : "activation", + "name" : "MobilenetV1\/MobilenetV1\/Conv2d_5_pointwise\/Relu6:0", + "beta" : 0 + }, + { + "pad_r" : 0, + "fused_relu" : 1, + "fused_tanh" : 0, + "pad_fill_mode" : 0, + "pad_b" : 0, + "pad_l" : 0, + "top" : "relu_MobilenetV1\/MobilenetV1\/Conv2d_6_depthwise\/Relu6:0", + "blob_weights" : 47, + "K" : 256, + "blob_biases" : 45, + "stride_x" : 2, + "name" : "MobilenetV1\/MobilenetV1\/Conv2d_6_depthwise\/depthwise:0", + "has_batch_norm" : 0, + "type" : "convolution", + "n_groups" : 256, + "pad_t" : 0, + "stride_y" : 2, + "has_biases" : 1, + "C" : 256, + "bottom" : "MobilenetV1\/MobilenetV1\/Conv2d_5_pointwise\/Relu6:0", + "weights" : { + + }, + "Nx" : 3, + "pad_mode" : 1, + "pad_value" : 0, + "Ny" : 3, + "n_parallel" : 1 + }, + { + "alpha" : -1, + "bottom" : "relu_MobilenetV1\/MobilenetV1\/Conv2d_6_depthwise\/Relu6:0", + "weights" : { + + }, + "mode" : 6, + "top" : "relu_MobilenetV1\/MobilenetV1\/Conv2d_6_depthwise\/Relu6:0_neg", + "type" : "activation", + "name" : "relu_MobilenetV1\/MobilenetV1\/Conv2d_6_depthwise\/Relu6:0_neg", + "beta" : 0 + }, + { + "alpha" : -6, + "bottom" : "relu_MobilenetV1\/MobilenetV1\/Conv2d_6_depthwise\/Relu6:0_neg", + "operation" : 25, + "weights" : { + + }, + "fused_relu" : 0, + "top" : "relu_MobilenetV1\/MobilenetV1\/Conv2d_6_depthwise\/Relu6:0_clip", + "type" : "elementwise", + "name" : "relu_MobilenetV1\/MobilenetV1\/Conv2d_6_depthwise\/Relu6:0_clip", + "beta" : 0 + }, + { + "alpha" : -1, + "bottom" : "relu_MobilenetV1\/MobilenetV1\/Conv2d_6_depthwise\/Relu6:0_clip", + "weights" : { + + }, + "mode" : 6, + "top" : "MobilenetV1\/MobilenetV1\/Conv2d_6_depthwise\/Relu6:0", + "type" : "activation", + "name" : "MobilenetV1\/MobilenetV1\/Conv2d_6_depthwise\/Relu6:0", + "beta" : 0 + }, + { + "pad_r" : 0, + "fused_relu" : 1, + "fused_tanh" : 0, + "pad_fill_mode" : 0, + "pad_b" : 0, + "pad_l" : 0, + "top" : "relu_MobilenetV1\/MobilenetV1\/Conv2d_6_pointwise\/Relu6:0", + "blob_weights" : 51, + "K" : 256, + "blob_biases" : 49, + "name" : "MobilenetV1\/MobilenetV1\/Conv2d_6_pointwise\/Conv2D:0", + "has_batch_norm" : 0, + "type" : "convolution", + "n_groups" : 1, + "pad_t" : 0, + "has_biases" : 1, + "C" : 512, + "bottom" : "MobilenetV1\/MobilenetV1\/Conv2d_6_depthwise\/Relu6:0", + "weights" : { + + }, + "pad_mode" : 1, + "Nx" : 1, + "pad_value" : 0, + "Ny" : 1, + "n_parallel" : 1 + }, + { + "alpha" : -1, + "bottom" : "relu_MobilenetV1\/MobilenetV1\/Conv2d_6_pointwise\/Relu6:0", + "weights" : { + + }, + "mode" : 6, + "top" : "relu_MobilenetV1\/MobilenetV1\/Conv2d_6_pointwise\/Relu6:0_neg", + "type" : "activation", + "name" : "relu_MobilenetV1\/MobilenetV1\/Conv2d_6_pointwise\/Relu6:0_neg", + "beta" : 0 + }, + { + "alpha" : -6, + "bottom" : "relu_MobilenetV1\/MobilenetV1\/Conv2d_6_pointwise\/Relu6:0_neg", + "operation" : 25, + "weights" : { + + }, + "fused_relu" : 0, + "top" : "relu_MobilenetV1\/MobilenetV1\/Conv2d_6_pointwise\/Relu6:0_clip", + "type" : "elementwise", + "name" : "relu_MobilenetV1\/MobilenetV1\/Conv2d_6_pointwise\/Relu6:0_clip", + "beta" : 0 + }, + { + "alpha" : -1, + "bottom" : "relu_MobilenetV1\/MobilenetV1\/Conv2d_6_pointwise\/Relu6:0_clip", + "weights" : { + + }, + "mode" : 6, + "top" : "MobilenetV1\/MobilenetV1\/Conv2d_6_pointwise\/Relu6:0", + "type" : "activation", + "name" : "MobilenetV1\/MobilenetV1\/Conv2d_6_pointwise\/Relu6:0", + "beta" : 0 + }, + { + "pad_r" : 0, + "fused_relu" : 1, + "fused_tanh" : 0, + "pad_fill_mode" : 0, + "pad_b" : 0, + "pad_l" : 0, + "top" : "relu_MobilenetV1\/MobilenetV1\/Conv2d_7_depthwise\/Relu6:0", + "blob_weights" : 55, + "K" : 512, + "blob_biases" : 53, + "name" : "MobilenetV1\/MobilenetV1\/Conv2d_7_depthwise\/depthwise:0", + "has_batch_norm" : 0, + "type" : "convolution", + "n_groups" : 512, + "pad_t" : 0, + "has_biases" : 1, + "C" : 512, + "bottom" : "MobilenetV1\/MobilenetV1\/Conv2d_6_pointwise\/Relu6:0", + "weights" : { + + }, + "pad_mode" : 1, + "Nx" : 3, + "pad_value" : 0, + "Ny" : 3, + "n_parallel" : 1 + }, + { + "alpha" : -1, + "bottom" : "relu_MobilenetV1\/MobilenetV1\/Conv2d_7_depthwise\/Relu6:0", + "weights" : { + + }, + "mode" : 6, + "top" : "relu_MobilenetV1\/MobilenetV1\/Conv2d_7_depthwise\/Relu6:0_neg", + "type" : "activation", + "name" : "relu_MobilenetV1\/MobilenetV1\/Conv2d_7_depthwise\/Relu6:0_neg", + "beta" : 0 + }, + { + "alpha" : -6, + "bottom" : "relu_MobilenetV1\/MobilenetV1\/Conv2d_7_depthwise\/Relu6:0_neg", + "operation" : 25, + "weights" : { + + }, + "fused_relu" : 0, + "top" : "relu_MobilenetV1\/MobilenetV1\/Conv2d_7_depthwise\/Relu6:0_clip", + "type" : "elementwise", + "name" : "relu_MobilenetV1\/MobilenetV1\/Conv2d_7_depthwise\/Relu6:0_clip", + "beta" : 0 + }, + { + "alpha" : -1, + "bottom" : "relu_MobilenetV1\/MobilenetV1\/Conv2d_7_depthwise\/Relu6:0_clip", + "weights" : { + + }, + "mode" : 6, + "top" : "MobilenetV1\/MobilenetV1\/Conv2d_7_depthwise\/Relu6:0", + "type" : "activation", + "name" : "MobilenetV1\/MobilenetV1\/Conv2d_7_depthwise\/Relu6:0", + "beta" : 0 + }, + { + "pad_r" : 0, + "fused_relu" : 1, + "fused_tanh" : 0, + "pad_fill_mode" : 0, + "pad_b" : 0, + "pad_l" : 0, + "top" : "relu_MobilenetV1\/MobilenetV1\/Conv2d_7_pointwise\/Relu6:0", + "blob_weights" : 59, + "K" : 512, + "blob_biases" : 57, + "name" : "MobilenetV1\/MobilenetV1\/Conv2d_7_pointwise\/Conv2D:0", + "has_batch_norm" : 0, + "type" : "convolution", + "n_groups" : 1, + "pad_t" : 0, + "has_biases" : 1, + "C" : 512, + "bottom" : "MobilenetV1\/MobilenetV1\/Conv2d_7_depthwise\/Relu6:0", + "weights" : { + + }, + "pad_mode" : 1, + "Nx" : 1, + "pad_value" : 0, + "Ny" : 1, + "n_parallel" : 1 + }, + { + "alpha" : -1, + "bottom" : "relu_MobilenetV1\/MobilenetV1\/Conv2d_7_pointwise\/Relu6:0", + "weights" : { + + }, + "mode" : 6, + "top" : "relu_MobilenetV1\/MobilenetV1\/Conv2d_7_pointwise\/Relu6:0_neg", + "type" : "activation", + "name" : "relu_MobilenetV1\/MobilenetV1\/Conv2d_7_pointwise\/Relu6:0_neg", + "beta" : 0 + }, + { + "alpha" : -6, + "bottom" : "relu_MobilenetV1\/MobilenetV1\/Conv2d_7_pointwise\/Relu6:0_neg", + "operation" : 25, + "weights" : { + + }, + "fused_relu" : 0, + "top" : "relu_MobilenetV1\/MobilenetV1\/Conv2d_7_pointwise\/Relu6:0_clip", + "type" : "elementwise", + "name" : "relu_MobilenetV1\/MobilenetV1\/Conv2d_7_pointwise\/Relu6:0_clip", + "beta" : 0 + }, + { + "alpha" : -1, + "bottom" : "relu_MobilenetV1\/MobilenetV1\/Conv2d_7_pointwise\/Relu6:0_clip", + "weights" : { + + }, + "mode" : 6, + "top" : "MobilenetV1\/MobilenetV1\/Conv2d_7_pointwise\/Relu6:0", + "type" : "activation", + "name" : "MobilenetV1\/MobilenetV1\/Conv2d_7_pointwise\/Relu6:0", + "beta" : 0 + }, + { + "pad_r" : 0, + "fused_relu" : 1, + "fused_tanh" : 0, + "pad_fill_mode" : 0, + "pad_b" : 0, + "pad_l" : 0, + "top" : "relu_MobilenetV1\/MobilenetV1\/Conv2d_8_depthwise\/Relu6:0", + "blob_weights" : 63, + "K" : 512, + "blob_biases" : 61, + "name" : "MobilenetV1\/MobilenetV1\/Conv2d_8_depthwise\/depthwise:0", + "has_batch_norm" : 0, + "type" : "convolution", + "n_groups" : 512, + "pad_t" : 0, + "has_biases" : 1, + "C" : 512, + "bottom" : "MobilenetV1\/MobilenetV1\/Conv2d_7_pointwise\/Relu6:0", + "weights" : { + + }, + "pad_mode" : 1, + "Nx" : 3, + "pad_value" : 0, + "Ny" : 3, + "n_parallel" : 1 + }, + { + "alpha" : -1, + "bottom" : "relu_MobilenetV1\/MobilenetV1\/Conv2d_8_depthwise\/Relu6:0", + "weights" : { + + }, + "mode" : 6, + "top" : "relu_MobilenetV1\/MobilenetV1\/Conv2d_8_depthwise\/Relu6:0_neg", + "type" : "activation", + "name" : "relu_MobilenetV1\/MobilenetV1\/Conv2d_8_depthwise\/Relu6:0_neg", + "beta" : 0 + }, + { + "alpha" : -6, + "bottom" : "relu_MobilenetV1\/MobilenetV1\/Conv2d_8_depthwise\/Relu6:0_neg", + "operation" : 25, + "weights" : { + + }, + "fused_relu" : 0, + "top" : "relu_MobilenetV1\/MobilenetV1\/Conv2d_8_depthwise\/Relu6:0_clip", + "type" : "elementwise", + "name" : "relu_MobilenetV1\/MobilenetV1\/Conv2d_8_depthwise\/Relu6:0_clip", + "beta" : 0 + }, + { + "alpha" : -1, + "bottom" : "relu_MobilenetV1\/MobilenetV1\/Conv2d_8_depthwise\/Relu6:0_clip", + "weights" : { + + }, + "mode" : 6, + "top" : "MobilenetV1\/MobilenetV1\/Conv2d_8_depthwise\/Relu6:0", + "type" : "activation", + "name" : "MobilenetV1\/MobilenetV1\/Conv2d_8_depthwise\/Relu6:0", + "beta" : 0 + }, + { + "pad_r" : 0, + "fused_relu" : 1, + "fused_tanh" : 0, + "pad_fill_mode" : 0, + "pad_b" : 0, + "pad_l" : 0, + "top" : "relu_MobilenetV1\/MobilenetV1\/Conv2d_8_pointwise\/Relu6:0", + "blob_weights" : 67, + "K" : 512, + "blob_biases" : 65, + "name" : "MobilenetV1\/MobilenetV1\/Conv2d_8_pointwise\/Conv2D:0", + "has_batch_norm" : 0, + "type" : "convolution", + "n_groups" : 1, + "pad_t" : 0, + "has_biases" : 1, + "C" : 512, + "bottom" : "MobilenetV1\/MobilenetV1\/Conv2d_8_depthwise\/Relu6:0", + "weights" : { + + }, + "pad_mode" : 1, + "Nx" : 1, + "pad_value" : 0, + "Ny" : 1, + "n_parallel" : 1 + }, + { + "alpha" : -1, + "bottom" : "relu_MobilenetV1\/MobilenetV1\/Conv2d_8_pointwise\/Relu6:0", + "weights" : { + + }, + "mode" : 6, + "top" : "relu_MobilenetV1\/MobilenetV1\/Conv2d_8_pointwise\/Relu6:0_neg", + "type" : "activation", + "name" : "relu_MobilenetV1\/MobilenetV1\/Conv2d_8_pointwise\/Relu6:0_neg", + "beta" : 0 + }, + { + "alpha" : -6, + "bottom" : "relu_MobilenetV1\/MobilenetV1\/Conv2d_8_pointwise\/Relu6:0_neg", + "operation" : 25, + "weights" : { + + }, + "fused_relu" : 0, + "top" : "relu_MobilenetV1\/MobilenetV1\/Conv2d_8_pointwise\/Relu6:0_clip", + "type" : "elementwise", + "name" : "relu_MobilenetV1\/MobilenetV1\/Conv2d_8_pointwise\/Relu6:0_clip", + "beta" : 0 + }, + { + "alpha" : -1, + "bottom" : "relu_MobilenetV1\/MobilenetV1\/Conv2d_8_pointwise\/Relu6:0_clip", + "weights" : { + + }, + "mode" : 6, + "top" : "MobilenetV1\/MobilenetV1\/Conv2d_8_pointwise\/Relu6:0", + "type" : "activation", + "name" : "MobilenetV1\/MobilenetV1\/Conv2d_8_pointwise\/Relu6:0", + "beta" : 0 + }, + { + "pad_r" : 0, + "fused_relu" : 1, + "fused_tanh" : 0, + "pad_fill_mode" : 0, + "pad_b" : 0, + "pad_l" : 0, + "top" : "relu_MobilenetV1\/MobilenetV1\/Conv2d_9_depthwise\/Relu6:0", + "blob_weights" : 71, + "K" : 512, + "blob_biases" : 69, + "name" : "MobilenetV1\/MobilenetV1\/Conv2d_9_depthwise\/depthwise:0", + "has_batch_norm" : 0, + "type" : "convolution", + "n_groups" : 512, + "pad_t" : 0, + "has_biases" : 1, + "C" : 512, + "bottom" : "MobilenetV1\/MobilenetV1\/Conv2d_8_pointwise\/Relu6:0", + "weights" : { + + }, + "pad_mode" : 1, + "Nx" : 3, + "pad_value" : 0, + "Ny" : 3, + "n_parallel" : 1 + }, + { + "alpha" : -1, + "bottom" : "relu_MobilenetV1\/MobilenetV1\/Conv2d_9_depthwise\/Relu6:0", + "weights" : { + + }, + "mode" : 6, + "top" : "relu_MobilenetV1\/MobilenetV1\/Conv2d_9_depthwise\/Relu6:0_neg", + "type" : "activation", + "name" : "relu_MobilenetV1\/MobilenetV1\/Conv2d_9_depthwise\/Relu6:0_neg", + "beta" : 0 + }, + { + "alpha" : -6, + "bottom" : "relu_MobilenetV1\/MobilenetV1\/Conv2d_9_depthwise\/Relu6:0_neg", + "operation" : 25, + "weights" : { + + }, + "fused_relu" : 0, + "top" : "relu_MobilenetV1\/MobilenetV1\/Conv2d_9_depthwise\/Relu6:0_clip", + "type" : "elementwise", + "name" : "relu_MobilenetV1\/MobilenetV1\/Conv2d_9_depthwise\/Relu6:0_clip", + "beta" : 0 + }, + { + "alpha" : -1, + "bottom" : "relu_MobilenetV1\/MobilenetV1\/Conv2d_9_depthwise\/Relu6:0_clip", + "weights" : { + + }, + "mode" : 6, + "top" : "MobilenetV1\/MobilenetV1\/Conv2d_9_depthwise\/Relu6:0", + "type" : "activation", + "name" : "MobilenetV1\/MobilenetV1\/Conv2d_9_depthwise\/Relu6:0", + "beta" : 0 + }, + { + "pad_r" : 0, + "fused_relu" : 1, + "fused_tanh" : 0, + "pad_fill_mode" : 0, + "pad_b" : 0, + "pad_l" : 0, + "top" : "relu_MobilenetV1\/MobilenetV1\/Conv2d_9_pointwise\/Relu6:0", + "blob_weights" : 75, + "K" : 512, + "blob_biases" : 73, + "name" : "MobilenetV1\/MobilenetV1\/Conv2d_9_pointwise\/Conv2D:0", + "has_batch_norm" : 0, + "type" : "convolution", + "n_groups" : 1, + "pad_t" : 0, + "has_biases" : 1, + "C" : 512, + "bottom" : "MobilenetV1\/MobilenetV1\/Conv2d_9_depthwise\/Relu6:0", + "weights" : { + + }, + "pad_mode" : 1, + "Nx" : 1, + "pad_value" : 0, + "Ny" : 1, + "n_parallel" : 1 + }, + { + "alpha" : -1, + "bottom" : "relu_MobilenetV1\/MobilenetV1\/Conv2d_9_pointwise\/Relu6:0", + "weights" : { + + }, + "mode" : 6, + "top" : "relu_MobilenetV1\/MobilenetV1\/Conv2d_9_pointwise\/Relu6:0_neg", + "type" : "activation", + "name" : "relu_MobilenetV1\/MobilenetV1\/Conv2d_9_pointwise\/Relu6:0_neg", + "beta" : 0 + }, + { + "alpha" : -6, + "bottom" : "relu_MobilenetV1\/MobilenetV1\/Conv2d_9_pointwise\/Relu6:0_neg", + "operation" : 25, + "weights" : { + + }, + "fused_relu" : 0, + "top" : "relu_MobilenetV1\/MobilenetV1\/Conv2d_9_pointwise\/Relu6:0_clip", + "type" : "elementwise", + "name" : "relu_MobilenetV1\/MobilenetV1\/Conv2d_9_pointwise\/Relu6:0_clip", + "beta" : 0 + }, + { + "alpha" : -1, + "bottom" : "relu_MobilenetV1\/MobilenetV1\/Conv2d_9_pointwise\/Relu6:0_clip", + "weights" : { + + }, + "mode" : 6, + "top" : "MobilenetV1\/MobilenetV1\/Conv2d_9_pointwise\/Relu6:0", + "type" : "activation", + "name" : "MobilenetV1\/MobilenetV1\/Conv2d_9_pointwise\/Relu6:0", + "beta" : 0 + }, + { + "pad_r" : 0, + "fused_relu" : 1, + "fused_tanh" : 0, + "pad_fill_mode" : 0, + "pad_b" : 0, + "pad_l" : 0, + "top" : "relu_MobilenetV1\/MobilenetV1\/Conv2d_10_depthwise\/Relu6:0", + "blob_weights" : 79, + "K" : 512, + "blob_biases" : 77, + "name" : "MobilenetV1\/MobilenetV1\/Conv2d_10_depthwise\/depthwise:0", + "has_batch_norm" : 0, + "type" : "convolution", + "n_groups" : 512, + "pad_t" : 0, + "has_biases" : 1, + "C" : 512, + "bottom" : "MobilenetV1\/MobilenetV1\/Conv2d_9_pointwise\/Relu6:0", + "weights" : { + + }, + "pad_mode" : 1, + "Nx" : 3, + "pad_value" : 0, + "Ny" : 3, + "n_parallel" : 1 + }, + { + "alpha" : -1, + "bottom" : "relu_MobilenetV1\/MobilenetV1\/Conv2d_10_depthwise\/Relu6:0", + "weights" : { + + }, + "mode" : 6, + "top" : "relu_MobilenetV1\/MobilenetV1\/Conv2d_10_depthwise\/Relu6:0_neg", + "type" : "activation", + "name" : "relu_MobilenetV1\/MobilenetV1\/Conv2d_10_depthwise\/Relu6:0_neg", + "beta" : 0 + }, + { + "alpha" : -6, + "bottom" : "relu_MobilenetV1\/MobilenetV1\/Conv2d_10_depthwise\/Relu6:0_neg", + "operation" : 25, + "weights" : { + + }, + "fused_relu" : 0, + "top" : "relu_MobilenetV1\/MobilenetV1\/Conv2d_10_depthwise\/Relu6:0_clip", + "type" : "elementwise", + "name" : "relu_MobilenetV1\/MobilenetV1\/Conv2d_10_depthwise\/Relu6:0_clip", + "beta" : 0 + }, + { + "alpha" : -1, + "bottom" : "relu_MobilenetV1\/MobilenetV1\/Conv2d_10_depthwise\/Relu6:0_clip", + "weights" : { + + }, + "mode" : 6, + "top" : "MobilenetV1\/MobilenetV1\/Conv2d_10_depthwise\/Relu6:0", + "type" : "activation", + "name" : "MobilenetV1\/MobilenetV1\/Conv2d_10_depthwise\/Relu6:0", + "beta" : 0 + }, + { + "pad_r" : 0, + "fused_relu" : 1, + "fused_tanh" : 0, + "pad_fill_mode" : 0, + "pad_b" : 0, + "pad_l" : 0, + "top" : "relu_MobilenetV1\/MobilenetV1\/Conv2d_10_pointwise\/Relu6:0", + "blob_weights" : 83, + "K" : 512, + "blob_biases" : 81, + "name" : "MobilenetV1\/MobilenetV1\/Conv2d_10_pointwise\/Conv2D:0", + "has_batch_norm" : 0, + "type" : "convolution", + "n_groups" : 1, + "pad_t" : 0, + "has_biases" : 1, + "C" : 512, + "bottom" : "MobilenetV1\/MobilenetV1\/Conv2d_10_depthwise\/Relu6:0", + "weights" : { + + }, + "pad_mode" : 1, + "Nx" : 1, + "pad_value" : 0, + "Ny" : 1, + "n_parallel" : 1 + }, + { + "alpha" : -1, + "bottom" : "relu_MobilenetV1\/MobilenetV1\/Conv2d_10_pointwise\/Relu6:0", + "weights" : { + + }, + "mode" : 6, + "top" : "relu_MobilenetV1\/MobilenetV1\/Conv2d_10_pointwise\/Relu6:0_neg", + "type" : "activation", + "name" : "relu_MobilenetV1\/MobilenetV1\/Conv2d_10_pointwise\/Relu6:0_neg", + "beta" : 0 + }, + { + "alpha" : -6, + "bottom" : "relu_MobilenetV1\/MobilenetV1\/Conv2d_10_pointwise\/Relu6:0_neg", + "operation" : 25, + "weights" : { + + }, + "fused_relu" : 0, + "top" : "relu_MobilenetV1\/MobilenetV1\/Conv2d_10_pointwise\/Relu6:0_clip", + "type" : "elementwise", + "name" : "relu_MobilenetV1\/MobilenetV1\/Conv2d_10_pointwise\/Relu6:0_clip", + "beta" : 0 + }, + { + "alpha" : -1, + "bottom" : "relu_MobilenetV1\/MobilenetV1\/Conv2d_10_pointwise\/Relu6:0_clip", + "weights" : { + + }, + "mode" : 6, + "top" : "MobilenetV1\/MobilenetV1\/Conv2d_10_pointwise\/Relu6:0", + "type" : "activation", + "name" : "MobilenetV1\/MobilenetV1\/Conv2d_10_pointwise\/Relu6:0", + "beta" : 0 + }, + { + "pad_r" : 0, + "fused_relu" : 1, + "fused_tanh" : 0, + "pad_fill_mode" : 0, + "pad_b" : 0, + "pad_l" : 0, + "top" : "relu_MobilenetV1\/MobilenetV1\/Conv2d_11_depthwise\/Relu6:0", + "blob_weights" : 87, + "K" : 512, + "blob_biases" : 85, + "name" : "MobilenetV1\/MobilenetV1\/Conv2d_11_depthwise\/depthwise:0", + "has_batch_norm" : 0, + "type" : "convolution", + "n_groups" : 512, + "pad_t" : 0, + "has_biases" : 1, + "C" : 512, + "bottom" : "MobilenetV1\/MobilenetV1\/Conv2d_10_pointwise\/Relu6:0", + "weights" : { + + }, + "pad_mode" : 1, + "Nx" : 3, + "pad_value" : 0, + "Ny" : 3, + "n_parallel" : 1 + }, + { + "alpha" : -1, + "bottom" : "relu_MobilenetV1\/MobilenetV1\/Conv2d_11_depthwise\/Relu6:0", + "weights" : { + + }, + "mode" : 6, + "top" : "relu_MobilenetV1\/MobilenetV1\/Conv2d_11_depthwise\/Relu6:0_neg", + "type" : "activation", + "name" : "relu_MobilenetV1\/MobilenetV1\/Conv2d_11_depthwise\/Relu6:0_neg", + "beta" : 0 + }, + { + "alpha" : -6, + "bottom" : "relu_MobilenetV1\/MobilenetV1\/Conv2d_11_depthwise\/Relu6:0_neg", + "operation" : 25, + "weights" : { + + }, + "fused_relu" : 0, + "top" : "relu_MobilenetV1\/MobilenetV1\/Conv2d_11_depthwise\/Relu6:0_clip", + "type" : "elementwise", + "name" : "relu_MobilenetV1\/MobilenetV1\/Conv2d_11_depthwise\/Relu6:0_clip", + "beta" : 0 + }, + { + "alpha" : -1, + "bottom" : "relu_MobilenetV1\/MobilenetV1\/Conv2d_11_depthwise\/Relu6:0_clip", + "weights" : { + + }, + "mode" : 6, + "top" : "MobilenetV1\/MobilenetV1\/Conv2d_11_depthwise\/Relu6:0", + "type" : "activation", + "name" : "MobilenetV1\/MobilenetV1\/Conv2d_11_depthwise\/Relu6:0", + "beta" : 0 + }, + { + "pad_r" : 0, + "fused_relu" : 1, + "fused_tanh" : 0, + "pad_fill_mode" : 0, + "pad_b" : 0, + "pad_l" : 0, + "top" : "relu_MobilenetV1\/MobilenetV1\/Conv2d_11_pointwise\/Relu6:0", + "blob_weights" : 91, + "K" : 512, + "blob_biases" : 89, + "name" : "MobilenetV1\/MobilenetV1\/Conv2d_11_pointwise\/Conv2D:0", + "has_batch_norm" : 0, + "type" : "convolution", + "n_groups" : 1, + "pad_t" : 0, + "has_biases" : 1, + "C" : 512, + "bottom" : "MobilenetV1\/MobilenetV1\/Conv2d_11_depthwise\/Relu6:0", + "weights" : { + + }, + "pad_mode" : 1, + "Nx" : 1, + "pad_value" : 0, + "Ny" : 1, + "n_parallel" : 1 + }, + { + "alpha" : -1, + "bottom" : "relu_MobilenetV1\/MobilenetV1\/Conv2d_11_pointwise\/Relu6:0", + "weights" : { + + }, + "mode" : 6, + "top" : "relu_MobilenetV1\/MobilenetV1\/Conv2d_11_pointwise\/Relu6:0_neg", + "type" : "activation", + "name" : "relu_MobilenetV1\/MobilenetV1\/Conv2d_11_pointwise\/Relu6:0_neg", + "beta" : 0 + }, + { + "alpha" : -6, + "bottom" : "relu_MobilenetV1\/MobilenetV1\/Conv2d_11_pointwise\/Relu6:0_neg", + "operation" : 25, + "weights" : { + + }, + "fused_relu" : 0, + "top" : "relu_MobilenetV1\/MobilenetV1\/Conv2d_11_pointwise\/Relu6:0_clip", + "type" : "elementwise", + "name" : "relu_MobilenetV1\/MobilenetV1\/Conv2d_11_pointwise\/Relu6:0_clip", + "beta" : 0 + }, + { + "alpha" : -1, + "bottom" : "relu_MobilenetV1\/MobilenetV1\/Conv2d_11_pointwise\/Relu6:0_clip", + "weights" : { + + }, + "mode" : 6, + "top" : "MobilenetV1\/MobilenetV1\/Conv2d_11_pointwise\/Relu6:0", + "type" : "activation", + "name" : "MobilenetV1\/MobilenetV1\/Conv2d_11_pointwise\/Relu6:0", + "beta" : 0 + }, + { + "pad_r" : 0, + "fused_relu" : 1, + "fused_tanh" : 0, + "pad_fill_mode" : 0, + "pad_b" : 0, + "pad_l" : 0, + "top" : "relu_MobilenetV1\/MobilenetV1\/Conv2d_12_depthwise\/Relu6:0", + "blob_weights" : 95, + "K" : 512, + "blob_biases" : 93, + "stride_x" : 2, + "name" : "MobilenetV1\/MobilenetV1\/Conv2d_12_depthwise\/depthwise:0", + "has_batch_norm" : 0, + "type" : "convolution", + "n_groups" : 512, + "pad_t" : 0, + "stride_y" : 2, + "has_biases" : 1, + "C" : 512, + "bottom" : "MobilenetV1\/MobilenetV1\/Conv2d_11_pointwise\/Relu6:0", + "weights" : { + + }, + "Nx" : 3, + "pad_mode" : 1, + "pad_value" : 0, + "Ny" : 3, + "n_parallel" : 1 + }, + { + "alpha" : -1, + "bottom" : "relu_MobilenetV1\/MobilenetV1\/Conv2d_12_depthwise\/Relu6:0", + "weights" : { + + }, + "mode" : 6, + "top" : "relu_MobilenetV1\/MobilenetV1\/Conv2d_12_depthwise\/Relu6:0_neg", + "type" : "activation", + "name" : "relu_MobilenetV1\/MobilenetV1\/Conv2d_12_depthwise\/Relu6:0_neg", + "beta" : 0 + }, + { + "alpha" : -6, + "bottom" : "relu_MobilenetV1\/MobilenetV1\/Conv2d_12_depthwise\/Relu6:0_neg", + "operation" : 25, + "weights" : { + + }, + "fused_relu" : 0, + "top" : "relu_MobilenetV1\/MobilenetV1\/Conv2d_12_depthwise\/Relu6:0_clip", + "type" : "elementwise", + "name" : "relu_MobilenetV1\/MobilenetV1\/Conv2d_12_depthwise\/Relu6:0_clip", + "beta" : 0 + }, + { + "alpha" : -1, + "bottom" : "relu_MobilenetV1\/MobilenetV1\/Conv2d_12_depthwise\/Relu6:0_clip", + "weights" : { + + }, + "mode" : 6, + "top" : "MobilenetV1\/MobilenetV1\/Conv2d_12_depthwise\/Relu6:0", + "type" : "activation", + "name" : "MobilenetV1\/MobilenetV1\/Conv2d_12_depthwise\/Relu6:0", + "beta" : 0 + }, + { + "pad_r" : 0, + "fused_relu" : 1, + "fused_tanh" : 0, + "pad_fill_mode" : 0, + "pad_b" : 0, + "pad_l" : 0, + "top" : "relu_MobilenetV1\/MobilenetV1\/Conv2d_12_pointwise\/Relu6:0", + "blob_weights" : 99, + "K" : 512, + "blob_biases" : 97, + "name" : "MobilenetV1\/MobilenetV1\/Conv2d_12_pointwise\/Conv2D:0", + "has_batch_norm" : 0, + "type" : "convolution", + "n_groups" : 1, + "pad_t" : 0, + "has_biases" : 1, + "C" : 1024, + "bottom" : "MobilenetV1\/MobilenetV1\/Conv2d_12_depthwise\/Relu6:0", + "weights" : { + + }, + "pad_mode" : 1, + "Nx" : 1, + "pad_value" : 0, + "Ny" : 1, + "n_parallel" : 1 + }, + { + "alpha" : -1, + "bottom" : "relu_MobilenetV1\/MobilenetV1\/Conv2d_12_pointwise\/Relu6:0", + "weights" : { + + }, + "mode" : 6, + "top" : "relu_MobilenetV1\/MobilenetV1\/Conv2d_12_pointwise\/Relu6:0_neg", + "type" : "activation", + "name" : "relu_MobilenetV1\/MobilenetV1\/Conv2d_12_pointwise\/Relu6:0_neg", + "beta" : 0 + }, + { + "alpha" : -6, + "bottom" : "relu_MobilenetV1\/MobilenetV1\/Conv2d_12_pointwise\/Relu6:0_neg", + "operation" : 25, + "weights" : { + + }, + "fused_relu" : 0, + "top" : "relu_MobilenetV1\/MobilenetV1\/Conv2d_12_pointwise\/Relu6:0_clip", + "type" : "elementwise", + "name" : "relu_MobilenetV1\/MobilenetV1\/Conv2d_12_pointwise\/Relu6:0_clip", + "beta" : 0 + }, + { + "alpha" : -1, + "bottom" : "relu_MobilenetV1\/MobilenetV1\/Conv2d_12_pointwise\/Relu6:0_clip", + "weights" : { + + }, + "mode" : 6, + "top" : "MobilenetV1\/MobilenetV1\/Conv2d_12_pointwise\/Relu6:0", + "type" : "activation", + "name" : "MobilenetV1\/MobilenetV1\/Conv2d_12_pointwise\/Relu6:0", + "beta" : 0 + }, + { + "pad_r" : 0, + "fused_relu" : 1, + "fused_tanh" : 0, + "pad_fill_mode" : 0, + "pad_b" : 0, + "pad_l" : 0, + "top" : "relu_MobilenetV1\/MobilenetV1\/Conv2d_13_depthwise\/Relu6:0", + "blob_weights" : 103, + "K" : 1024, + "blob_biases" : 101, + "name" : "MobilenetV1\/MobilenetV1\/Conv2d_13_depthwise\/depthwise:0", + "has_batch_norm" : 0, + "type" : "convolution", + "n_groups" : 1024, + "pad_t" : 0, + "has_biases" : 1, + "C" : 1024, + "bottom" : "MobilenetV1\/MobilenetV1\/Conv2d_12_pointwise\/Relu6:0", + "weights" : { + + }, + "pad_mode" : 1, + "Nx" : 3, + "pad_value" : 0, + "Ny" : 3, + "n_parallel" : 1 + }, + { + "alpha" : -1, + "bottom" : "relu_MobilenetV1\/MobilenetV1\/Conv2d_13_depthwise\/Relu6:0", + "weights" : { + + }, + "mode" : 6, + "top" : "relu_MobilenetV1\/MobilenetV1\/Conv2d_13_depthwise\/Relu6:0_neg", + "type" : "activation", + "name" : "relu_MobilenetV1\/MobilenetV1\/Conv2d_13_depthwise\/Relu6:0_neg", + "beta" : 0 + }, + { + "alpha" : -6, + "bottom" : "relu_MobilenetV1\/MobilenetV1\/Conv2d_13_depthwise\/Relu6:0_neg", + "operation" : 25, + "weights" : { + + }, + "fused_relu" : 0, + "top" : "relu_MobilenetV1\/MobilenetV1\/Conv2d_13_depthwise\/Relu6:0_clip", + "type" : "elementwise", + "name" : "relu_MobilenetV1\/MobilenetV1\/Conv2d_13_depthwise\/Relu6:0_clip", + "beta" : 0 + }, + { + "alpha" : -1, + "bottom" : "relu_MobilenetV1\/MobilenetV1\/Conv2d_13_depthwise\/Relu6:0_clip", + "weights" : { + + }, + "mode" : 6, + "top" : "MobilenetV1\/MobilenetV1\/Conv2d_13_depthwise\/Relu6:0", + "type" : "activation", + "name" : "MobilenetV1\/MobilenetV1\/Conv2d_13_depthwise\/Relu6:0", + "beta" : 0 + }, + { + "pad_r" : 0, + "fused_relu" : 1, + "fused_tanh" : 0, + "pad_fill_mode" : 0, + "pad_b" : 0, + "pad_l" : 0, + "top" : "relu_MobilenetV1\/MobilenetV1\/Conv2d_13_pointwise\/Relu6:0", + "blob_weights" : 107, + "K" : 1024, + "blob_biases" : 105, + "name" : "MobilenetV1\/MobilenetV1\/Conv2d_13_pointwise\/Conv2D:0", + "has_batch_norm" : 0, + "type" : "convolution", + "n_groups" : 1, + "pad_t" : 0, + "has_biases" : 1, + "C" : 1024, + "bottom" : "MobilenetV1\/MobilenetV1\/Conv2d_13_depthwise\/Relu6:0", + "weights" : { + + }, + "pad_mode" : 1, + "Nx" : 1, + "pad_value" : 0, + "Ny" : 1, + "n_parallel" : 1 + }, + { + "alpha" : -1, + "bottom" : "relu_MobilenetV1\/MobilenetV1\/Conv2d_13_pointwise\/Relu6:0", + "weights" : { + + }, + "mode" : 6, + "top" : "relu_MobilenetV1\/MobilenetV1\/Conv2d_13_pointwise\/Relu6:0_neg", + "type" : "activation", + "name" : "relu_MobilenetV1\/MobilenetV1\/Conv2d_13_pointwise\/Relu6:0_neg", + "beta" : 0 + }, + { + "alpha" : -6, + "bottom" : "relu_MobilenetV1\/MobilenetV1\/Conv2d_13_pointwise\/Relu6:0_neg", + "operation" : 25, + "weights" : { + + }, + "fused_relu" : 0, + "top" : "relu_MobilenetV1\/MobilenetV1\/Conv2d_13_pointwise\/Relu6:0_clip", + "type" : "elementwise", + "name" : "relu_MobilenetV1\/MobilenetV1\/Conv2d_13_pointwise\/Relu6:0_clip", + "beta" : 0 + }, + { + "alpha" : -1, + "bottom" : "relu_MobilenetV1\/MobilenetV1\/Conv2d_13_pointwise\/Relu6:0_clip", + "weights" : { + + }, + "mode" : 6, + "top" : "MobilenetV1\/MobilenetV1\/Conv2d_13_pointwise\/Relu6:0", + "type" : "activation", + "name" : "MobilenetV1\/MobilenetV1\/Conv2d_13_pointwise\/Relu6:0", + "beta" : 0 + }, + { + "pad_value" : 0, + "average_count_exclude_padding" : 1, + "top_shape_style" : 0, + "avg_or_max" : 0, + "stride_x" : 2, + "pad_mode" : 2, + "stride_y" : 2, + "pad_t" : 0, + "weights" : { + + }, + "size_y" : 7, + "type" : "pool", + "pad_r" : 0, + "pad_b" : 0, + "size_x" : 7, + "pad_fill_mode" : 0, + "bottom" : "MobilenetV1\/MobilenetV1\/Conv2d_13_pointwise\/Relu6:0", + "pad_l" : 0, + "name" : "MobilenetV1\/Logits\/AvgPool_1a\/AvgPool:0", + "top" : "MobilenetV1\/Logits\/AvgPool_1a\/AvgPool:0" + }, + { + "pad_r" : 0, + "fused_relu" : 0, + "fused_tanh" : 0, + "pad_fill_mode" : 0, + "pad_b" : 0, + "pad_l" : 0, + "top" : "MobilenetV1\/Logits\/Conv2d_1c_1x1\/BiasAdd:0", + "blob_weights" : 111, + "K" : 1024, + "blob_biases" : 109, + "name" : "MobilenetV1\/Logits\/Conv2d_1c_1x1\/Conv2D:0", + "has_batch_norm" : 0, + "type" : "convolution", + "n_groups" : 1, + "pad_t" : 0, + "has_biases" : 1, + "C" : 1001, + "bottom" : "MobilenetV1\/Logits\/AvgPool_1a\/AvgPool:0", + "weights" : { + + }, + "pad_mode" : 1, + "Nx" : 1, + "pad_value" : 0, + "Ny" : 1, + "n_parallel" : 1 + }, + { + "nB" : 1001, + "top" : "final_retrain_ops\/Wx_plus_b\/add:0", + "has_biases" : 1, + "weights" : { + + }, + "nC" : 7, + "blob_weights" : 115, + "type" : "inner_product", + "has_relu" : 0, + "bottom" : "MobilenetV1\/Logits\/Conv2d_1c_1x1\/BiasAdd:0", + "blob_biases" : 113, + "has_tanh" : 0, + "name" : "final_retrain_ops\/Wx_plus_b\/MatMul", + "has_prelu" : 0 + }, + { + "C" : 2, + "weights" : { + + }, + "top" : "final_result__0", + "type" : "softmax", + "name" : "final_result:0", + "bottom" : "final_retrain_ops\/Wx_plus_b\/add:0" + } + ] +} \ No newline at end of file diff --git a/Playground/PlaygroundStub/PlaygroundStub/EmojiSketches.mlmodelc/model.espresso.shape b/Playground/PlaygroundStub/PlaygroundStub/EmojiSketches.mlmodelc/model.espresso.shape new file mode 100644 index 0000000..65c1500 --- /dev/null +++ b/Playground/PlaygroundStub/PlaygroundStub/EmojiSketches.mlmodelc/model.espresso.shape @@ -0,0 +1,682 @@ +{ + "layer_shapes" : { + "relu_MobilenetV1\/MobilenetV1\/Conv2d_10_depthwise\/Relu6:0_neg" : { + "k" : 512, + "w" : 14, + "n" : 1, + "h" : 14 + }, + "relu_MobilenetV1\/MobilenetV1\/Conv2d_2_pointwise\/Relu6:0_neg" : { + "k" : 128, + "w" : 56, + "n" : 1, + "h" : 56 + }, + "relu_MobilenetV1\/MobilenetV1\/Conv2d_5_pointwise\/Relu6:0_clip" : { + "k" : 256, + "w" : 28, + "n" : 1, + "h" : 28 + }, + "MobilenetV1\/MobilenetV1\/Conv2d_5_depthwise\/Relu6:0" : { + "k" : 256, + "w" : 28, + "n" : 1, + "h" : 28 + }, + "relu_MobilenetV1\/MobilenetV1\/Conv2d_8_pointwise\/Relu6:0_clip" : { + "k" : 512, + "w" : 14, + "n" : 1, + "h" : 14 + }, + "MobilenetV1\/MobilenetV1\/Conv2d_9_pointwise\/Relu6:0" : { + "k" : 512, + "w" : 14, + "n" : 1, + "h" : 14 + }, + "relu_MobilenetV1\/MobilenetV1\/Conv2d_2_depthwise\/Relu6:0_clip" : { + "k" : 64, + "w" : 56, + "n" : 1, + "h" : 56 + }, + "relu_MobilenetV1\/MobilenetV1\/Conv2d_5_depthwise\/Relu6:0" : { + "k" : 256, + "w" : 28, + "n" : 1, + "h" : 28 + }, + "MobilenetV1\/MobilenetV1\/Conv2d_10_depthwise\/Relu6:0" : { + "k" : 512, + "w" : 14, + "n" : 1, + "h" : 14 + }, + "MobilenetV1\/MobilenetV1\/Conv2d_3_pointwise\/Relu6:0" : { + "k" : 128, + "w" : 56, + "n" : 1, + "h" : 56 + }, + "relu_MobilenetV1\/MobilenetV1\/Conv2d_5_depthwise\/Relu6:0_clip" : { + "k" : 256, + "w" : 28, + "n" : 1, + "h" : 28 + }, + "relu_MobilenetV1\/MobilenetV1\/Conv2d_11_pointwise\/Relu6:0_clip" : { + "k" : 512, + "w" : 14, + "n" : 1, + "h" : 14 + }, + "relu_MobilenetV1\/MobilenetV1\/Conv2d_2_pointwise\/Relu6:0" : { + "k" : 128, + "w" : 56, + "n" : 1, + "h" : 56 + }, + "relu_MobilenetV1\/MobilenetV1\/Conv2d_4_depthwise\/Relu6:0_neg" : { + "k" : 128, + "w" : 28, + "n" : 1, + "h" : 28 + }, + "MobilenetV1\/MobilenetV1\/Conv2d_6_pointwise\/Relu6:0" : { + "k" : 512, + "w" : 14, + "n" : 1, + "h" : 14 + }, + "relu_MobilenetV1\/MobilenetV1\/Conv2d_11_pointwise\/Relu6:0_neg" : { + "k" : 512, + "w" : 14, + "n" : 1, + "h" : 14 + }, + "input__0" : { + "k" : 3, + "w" : 224, + "n" : 1, + "h" : 224 + }, + "final_retrain_ops\/Wx_plus_b\/add:0" : { + "k" : 7, + "w" : 1, + "n" : 1, + "h" : 1 + }, + "relu_MobilenetV1\/MobilenetV1\/Conv2d_1_depthwise\/Relu6:0" : { + "k" : 32, + "w" : 112, + "n" : 1, + "h" : 112 + }, + "MobilenetV1\/MobilenetV1\/Conv2d_11_depthwise\/Relu6:0" : { + "k" : 512, + "w" : 14, + "n" : 1, + "h" : 14 + }, + "relu_MobilenetV1\/MobilenetV1\/Conv2d_7_pointwise\/Relu6:0_neg" : { + "k" : 512, + "w" : 14, + "n" : 1, + "h" : 14 + }, + "relu_MobilenetV1\/MobilenetV1\/Conv2d_11_pointwise\/Relu6:0" : { + "k" : 512, + "w" : 14, + "n" : 1, + "h" : 14 + }, + "relu_MobilenetV1\/MobilenetV1\/Conv2d_13_pointwise\/Relu6:0_neg" : { + "k" : 1024, + "w" : 7, + "n" : 1, + "h" : 7 + }, + "relu_MobilenetV1\/MobilenetV1\/Conv2d_4_pointwise\/Relu6:0_neg" : { + "k" : 256, + "w" : 28, + "n" : 1, + "h" : 28 + }, + "relu_MobilenetV1\/MobilenetV1\/Conv2d_3_pointwise\/Relu6:0_clip" : { + "k" : 128, + "w" : 56, + "n" : 1, + "h" : 56 + }, + "relu_MobilenetV1\/MobilenetV1\/Conv2d_11_depthwise\/Relu6:0_clip" : { + "k" : 512, + "w" : 14, + "n" : 1, + "h" : 14 + }, + "relu_MobilenetV1\/MobilenetV1\/Conv2d_0\/Relu6:0_clip" : { + "k" : 32, + "w" : 112, + "n" : 1, + "h" : 112 + }, + "relu_MobilenetV1\/MobilenetV1\/Conv2d_8_depthwise\/Relu6:0_clip" : { + "k" : 512, + "w" : 14, + "n" : 1, + "h" : 14 + }, + "relu_MobilenetV1\/MobilenetV1\/Conv2d_7_pointwise\/Relu6:0" : { + "k" : 512, + "w" : 14, + "n" : 1, + "h" : 14 + }, + "MobilenetV1\/MobilenetV1\/Conv2d_0\/Relu6:0" : { + "k" : 32, + "w" : 112, + "n" : 1, + "h" : 112 + }, + "MobilenetV1\/MobilenetV1\/Conv2d_7_depthwise\/Relu6:0" : { + "k" : 512, + "w" : 14, + "n" : 1, + "h" : 14 + }, + "relu_MobilenetV1\/MobilenetV1\/Conv2d_10_depthwise\/Relu6:0" : { + "k" : 512, + "w" : 14, + "n" : 1, + "h" : 14 + }, + "relu_MobilenetV1\/MobilenetV1\/Conv2d_9_depthwise\/Relu6:0_neg" : { + "k" : 512, + "w" : 14, + "n" : 1, + "h" : 14 + }, + "relu_MobilenetV1\/MobilenetV1\/Conv2d_1_depthwise\/Relu6:0_neg" : { + "k" : 32, + "w" : 112, + "n" : 1, + "h" : 112 + }, + "MobilenetV1\/MobilenetV1\/Conv2d_1_depthwise\/Relu6:0" : { + "k" : 32, + "w" : 112, + "n" : 1, + "h" : 112 + }, + "MobilenetV1\/MobilenetV1\/Conv2d_12_depthwise\/Relu6:0" : { + "k" : 512, + "w" : 7, + "n" : 1, + "h" : 7 + }, + "relu_MobilenetV1\/MobilenetV1\/Conv2d_6_depthwise\/Relu6:0_neg" : { + "k" : 256, + "w" : 14, + "n" : 1, + "h" : 14 + }, + "relu_MobilenetV1\/MobilenetV1\/Conv2d_10_pointwise\/Relu6:0_clip" : { + "k" : 512, + "w" : 14, + "n" : 1, + "h" : 14 + }, + "relu_MobilenetV1\/MobilenetV1\/Conv2d_6_depthwise\/Relu6:0" : { + "k" : 256, + "w" : 14, + "n" : 1, + "h" : 14 + }, + "MobilenetV1\/MobilenetV1\/Conv2d_10_pointwise\/Relu6:0" : { + "k" : 512, + "w" : 14, + "n" : 1, + "h" : 14 + }, + "final_result__0" : { + "k" : 7, + "w" : 1, + "n" : 1, + "h" : 1 + }, + "MobilenetV1\/MobilenetV1\/Conv2d_4_depthwise\/Relu6:0" : { + "k" : 128, + "w" : 28, + "n" : 1, + "h" : 28 + }, + "relu_MobilenetV1\/MobilenetV1\/Conv2d_1_pointwise\/Relu6:0_neg" : { + "k" : 64, + "w" : 112, + "n" : 1, + "h" : 112 + }, + "relu_MobilenetV1\/MobilenetV1\/Conv2d_9_pointwise\/Relu6:0_neg" : { + "k" : 512, + "w" : 14, + "n" : 1, + "h" : 14 + }, + "relu_MobilenetV1\/MobilenetV1\/Conv2d_6_pointwise\/Relu6:0_clip" : { + "k" : 512, + "w" : 14, + "n" : 1, + "h" : 14 + }, + "MobilenetV1\/MobilenetV1\/Conv2d_8_pointwise\/Relu6:0" : { + "k" : 512, + "w" : 14, + "n" : 1, + "h" : 14 + }, + "relu_MobilenetV1\/MobilenetV1\/Conv2d_3_pointwise\/Relu6:0" : { + "k" : 128, + "w" : 56, + "n" : 1, + "h" : 56 + }, + "MobilenetV1\/MobilenetV1\/Conv2d_2_pointwise\/Relu6:0" : { + "k" : 128, + "w" : 56, + "n" : 1, + "h" : 56 + }, + "MobilenetV1\/MobilenetV1\/Conv2d_13_depthwise\/Relu6:0" : { + "k" : 1024, + "w" : 7, + "n" : 1, + "h" : 7 + }, + "relu_MobilenetV1\/MobilenetV1\/Conv2d_3_depthwise\/Relu6:0_clip" : { + "k" : 128, + "w" : 56, + "n" : 1, + "h" : 56 + }, + "relu_MobilenetV1\/MobilenetV1\/Conv2d_13_pointwise\/Relu6:0_clip" : { + "k" : 1024, + "w" : 7, + "n" : 1, + "h" : 7 + }, + "relu_MobilenetV1\/MobilenetV1\/Conv2d_6_depthwise\/Relu6:0_clip" : { + "k" : 256, + "w" : 14, + "n" : 1, + "h" : 14 + }, + "relu_MobilenetV1\/MobilenetV1\/Conv2d_2_depthwise\/Relu6:0" : { + "k" : 64, + "w" : 56, + "n" : 1, + "h" : 56 + }, + "MobilenetV1\/MobilenetV1\/Conv2d_11_pointwise\/Relu6:0" : { + "k" : 512, + "w" : 14, + "n" : 1, + "h" : 14 + }, + "relu_MobilenetV1\/MobilenetV1\/Conv2d_7_depthwise\/Relu6:0" : { + "k" : 512, + "w" : 14, + "n" : 1, + "h" : 14 + }, + "MobilenetV1\/MobilenetV1\/Conv2d_5_pointwise\/Relu6:0" : { + "k" : 256, + "w" : 28, + "n" : 1, + "h" : 28 + }, + "relu_MobilenetV1\/MobilenetV1\/Conv2d_13_depthwise\/Relu6:0" : { + "k" : 1024, + "w" : 7, + "n" : 1, + "h" : 7 + }, + "relu_MobilenetV1\/MobilenetV1\/Conv2d_3_depthwise\/Relu6:0_neg" : { + "k" : 128, + "w" : 56, + "n" : 1, + "h" : 56 + }, + "relu_MobilenetV1\/MobilenetV1\/Conv2d_10_pointwise\/Relu6:0" : { + "k" : 512, + "w" : 14, + "n" : 1, + "h" : 14 + }, + "relu_MobilenetV1\/MobilenetV1\/Conv2d_11_depthwise\/Relu6:0_neg" : { + "k" : 512, + "w" : 14, + "n" : 1, + "h" : 14 + }, + "MobilenetV1\/MobilenetV1\/Conv2d_13_pointwise\/Relu6:0" : { + "k" : 1024, + "w" : 7, + "n" : 1, + "h" : 7 + }, + "relu_MobilenetV1\/MobilenetV1\/Conv2d_8_pointwise\/Relu6:0" : { + "k" : 512, + "w" : 14, + "n" : 1, + "h" : 14 + }, + "relu_MobilenetV1\/MobilenetV1\/Conv2d_6_pointwise\/Relu6:0_neg" : { + "k" : 512, + "w" : 14, + "n" : 1, + "h" : 14 + }, + "relu_MobilenetV1\/MobilenetV1\/Conv2d_9_pointwise\/Relu6:0_clip" : { + "k" : 512, + "w" : 14, + "n" : 1, + "h" : 14 + }, + "relu_MobilenetV1\/MobilenetV1\/Conv2d_1_pointwise\/Relu6:0_clip" : { + "k" : 64, + "w" : 112, + "n" : 1, + "h" : 112 + }, + "MobilenetV1\/MobilenetV1\/Conv2d_12_pointwise\/Relu6:0" : { + "k" : 1024, + "w" : 7, + "n" : 1, + "h" : 7 + }, + "relu_MobilenetV1\/MobilenetV1\/Conv2d_13_depthwise\/Relu6:0_neg" : { + "k" : 1024, + "w" : 7, + "n" : 1, + "h" : 7 + }, + "relu_MobilenetV1\/MobilenetV1\/Conv2d_3_depthwise\/Relu6:0" : { + "k" : 128, + "w" : 56, + "n" : 1, + "h" : 56 + }, + "relu_MobilenetV1\/MobilenetV1\/Conv2d_9_depthwise\/Relu6:0_clip" : { + "k" : 512, + "w" : 14, + "n" : 1, + "h" : 14 + }, + "relu_MobilenetV1\/MobilenetV1\/Conv2d_1_depthwise\/Relu6:0_clip" : { + "k" : 32, + "w" : 112, + "n" : 1, + "h" : 112 + }, + "MobilenetV1\/MobilenetV1\/Conv2d_6_depthwise\/Relu6:0" : { + "k" : 256, + "w" : 14, + "n" : 1, + "h" : 14 + }, + "relu_MobilenetV1\/MobilenetV1\/Conv2d_10_depthwise\/Relu6:0_clip" : { + "k" : 512, + "w" : 14, + "n" : 1, + "h" : 14 + }, + "relu_MobilenetV1\/MobilenetV1\/Conv2d_4_pointwise\/Relu6:0" : { + "k" : 256, + "w" : 28, + "n" : 1, + "h" : 28 + }, + "relu_MobilenetV1\/MobilenetV1\/Conv2d_8_depthwise\/Relu6:0_neg" : { + "k" : 512, + "w" : 14, + "n" : 1, + "h" : 14 + }, + "relu_MobilenetV1\/MobilenetV1\/Conv2d_9_pointwise\/Relu6:0" : { + "k" : 512, + "w" : 14, + "n" : 1, + "h" : 14 + }, + "MobilenetV1\/MobilenetV1\/Conv2d_9_depthwise\/Relu6:0" : { + "k" : 512, + "w" : 14, + "n" : 1, + "h" : 14 + }, + "MobilenetV1\/Logits\/AvgPool_1a\/AvgPool:0" : { + "k" : 1024, + "w" : 1, + "n" : 1, + "h" : 1 + }, + "relu_MobilenetV1\/MobilenetV1\/Conv2d_10_pointwise\/Relu6:0_neg" : { + "k" : 512, + "w" : 14, + "n" : 1, + "h" : 14 + }, + "MobilenetV1\/MobilenetV1\/Conv2d_3_depthwise\/Relu6:0" : { + "k" : 128, + "w" : 56, + "n" : 1, + "h" : 56 + }, + "relu_MobilenetV1\/MobilenetV1\/Conv2d_3_pointwise\/Relu6:0_neg" : { + "k" : 128, + "w" : 56, + "n" : 1, + "h" : 56 + }, + "relu_MobilenetV1\/MobilenetV1\/Conv2d_4_pointwise\/Relu6:0_clip" : { + "k" : 256, + "w" : 28, + "n" : 1, + "h" : 28 + }, + "relu_MobilenetV1\/MobilenetV1\/Conv2d_8_depthwise\/Relu6:0" : { + "k" : 512, + "w" : 14, + "n" : 1, + "h" : 14 + }, + "MobilenetV1\/MobilenetV1\/Conv2d_1_pointwise\/Relu6:0" : { + "k" : 64, + "w" : 112, + "n" : 1, + "h" : 112 + }, + "relu_MobilenetV1\/MobilenetV1\/Conv2d_13_depthwise\/Relu6:0_clip" : { + "k" : 1024, + "w" : 7, + "n" : 1, + "h" : 7 + }, + "relu_MobilenetV1\/MobilenetV1\/Conv2d_7_pointwise\/Relu6:0_clip" : { + "k" : 512, + "w" : 14, + "n" : 1, + "h" : 14 + }, + "relu_MobilenetV1\/MobilenetV1\/Conv2d_12_depthwise\/Relu6:0" : { + "k" : 512, + "w" : 7, + "n" : 1, + "h" : 7 + }, + "relu_MobilenetV1\/MobilenetV1\/Conv2d_12_pointwise\/Relu6:0_neg" : { + "k" : 1024, + "w" : 7, + "n" : 1, + "h" : 7 + }, + "relu_MobilenetV1\/MobilenetV1\/Conv2d_5_pointwise\/Relu6:0" : { + "k" : 256, + "w" : 28, + "n" : 1, + "h" : 28 + }, + "relu_MobilenetV1\/MobilenetV1\/Conv2d_4_depthwise\/Relu6:0_clip" : { + "k" : 128, + "w" : 28, + "n" : 1, + "h" : 28 + }, + "relu_MobilenetV1\/MobilenetV1\/Conv2d_13_pointwise\/Relu6:0" : { + "k" : 1024, + "w" : 7, + "n" : 1, + "h" : 7 + }, + "MobilenetV1\/MobilenetV1\/Conv2d_4_pointwise\/Relu6:0" : { + "k" : 256, + "w" : 28, + "n" : 1, + "h" : 28 + }, + "relu_MobilenetV1\/MobilenetV1\/Conv2d_5_depthwise\/Relu6:0_neg" : { + "k" : 256, + "w" : 28, + "n" : 1, + "h" : 28 + }, + "relu_MobilenetV1\/MobilenetV1\/Conv2d_12_pointwise\/Relu6:0_clip" : { + "k" : 1024, + "w" : 7, + "n" : 1, + "h" : 7 + }, + "relu_MobilenetV1\/MobilenetV1\/Conv2d_4_depthwise\/Relu6:0" : { + "k" : 128, + "w" : 28, + "n" : 1, + "h" : 28 + }, + "relu_MobilenetV1\/MobilenetV1\/Conv2d_2_depthwise\/Relu6:0_neg" : { + "k" : 64, + "w" : 56, + "n" : 1, + "h" : 56 + }, + "MobilenetV1\/MobilenetV1\/Conv2d_7_pointwise\/Relu6:0" : { + "k" : 512, + "w" : 14, + "n" : 1, + "h" : 14 + }, + "relu_MobilenetV1\/MobilenetV1\/Conv2d_8_pointwise\/Relu6:0_neg" : { + "k" : 512, + "w" : 14, + "n" : 1, + "h" : 14 + }, + "relu_MobilenetV1\/MobilenetV1\/Conv2d_5_pointwise\/Relu6:0_neg" : { + "k" : 256, + "w" : 28, + "n" : 1, + "h" : 28 + }, + "relu_MobilenetV1\/MobilenetV1\/Conv2d_2_pointwise\/Relu6:0_clip" : { + "k" : 128, + "w" : 56, + "n" : 1, + "h" : 56 + }, + "relu_MobilenetV1\/MobilenetV1\/Conv2d_0\/Relu6:0" : { + "k" : 32, + "w" : 112, + "n" : 1, + "h" : 112 + }, + "MobilenetV1\/Logits\/Conv2d_1c_1x1\/BiasAdd:0" : { + "k" : 1001, + "w" : 1, + "n" : 1, + "h" : 1 + }, + "relu_MobilenetV1\/MobilenetV1\/Conv2d_7_depthwise\/Relu6:0_clip" : { + "k" : 512, + "w" : 14, + "n" : 1, + "h" : 14 + }, + "relu_MobilenetV1\/MobilenetV1\/Conv2d_12_pointwise\/Relu6:0" : { + "k" : 1024, + "w" : 7, + "n" : 1, + "h" : 7 + }, + "relu_MobilenetV1\/MobilenetV1\/Conv2d_12_depthwise\/Relu6:0_neg" : { + "k" : 512, + "w" : 7, + "n" : 1, + "h" : 7 + }, + "relu_MobilenetV1\/MobilenetV1\/Conv2d_9_depthwise\/Relu6:0" : { + "k" : 512, + "w" : 14, + "n" : 1, + "h" : 14 + }, + "MobilenetV1\/MobilenetV1\/Conv2d_8_depthwise\/Relu6:0" : { + "k" : 512, + "w" : 14, + "n" : 1, + "h" : 14 + }, + "relu_MobilenetV1\/MobilenetV1\/Conv2d_7_depthwise\/Relu6:0_neg" : { + "k" : 512, + "w" : 14, + "n" : 1, + "h" : 14 + }, + "relu_MobilenetV1\/MobilenetV1\/Conv2d_12_depthwise\/Relu6:0_clip" : { + "k" : 512, + "w" : 7, + "n" : 1, + "h" : 7 + }, + "MobilenetV1\/MobilenetV1\/Conv2d_2_depthwise\/Relu6:0" : { + "k" : 64, + "w" : 56, + "n" : 1, + "h" : 56 + }, + "relu_MobilenetV1\/MobilenetV1\/Conv2d_1_pointwise\/Relu6:0" : { + "k" : 64, + "w" : 112, + "n" : 1, + "h" : 112 + }, + "relu_MobilenetV1\/MobilenetV1\/Conv2d_6_pointwise\/Relu6:0" : { + "k" : 512, + "w" : 14, + "n" : 1, + "h" : 14 + }, + "relu_MobilenetV1\/MobilenetV1\/Conv2d_11_depthwise\/Relu6:0" : { + "k" : 512, + "w" : 14, + "n" : 1, + "h" : 14 + }, + "relu_MobilenetV1\/MobilenetV1\/Conv2d_0\/Relu6:0_neg" : { + "k" : 32, + "w" : 112, + "n" : 1, + "h" : 112 + } + } +} \ No newline at end of file diff --git a/Playground/PlaygroundStub/PlaygroundStub/EmojiSketches.mlmodelc/model.espresso.weights b/Playground/PlaygroundStub/PlaygroundStub/EmojiSketches.mlmodelc/model.espresso.weights new file mode 100644 index 0000000..5321b47 Binary files /dev/null and b/Playground/PlaygroundStub/PlaygroundStub/EmojiSketches.mlmodelc/model.espresso.weights differ diff --git a/Playground/PlaygroundStub/PlaygroundStub/EmojiSketches.mlmodelc/model/coremldata.bin b/Playground/PlaygroundStub/PlaygroundStub/EmojiSketches.mlmodelc/model/coremldata.bin new file mode 100644 index 0000000..e7e00c0 Binary files /dev/null and b/Playground/PlaygroundStub/PlaygroundStub/EmojiSketches.mlmodelc/model/coremldata.bin differ diff --git a/Playground/PlaygroundStub/PlaygroundStub/Info.plist b/Playground/PlaygroundStub/PlaygroundStub/Info.plist new file mode 100644 index 0000000..d1788a3 --- /dev/null +++ b/Playground/PlaygroundStub/PlaygroundStub/Info.plist @@ -0,0 +1,47 @@ + + + + + CFBundleDevelopmentRegion + $(DEVELOPMENT_LANGUAGE) + CFBundleDisplayName + MLMOJI + CFBundleExecutable + $(EXECUTABLE_NAME) + CFBundleIdentifier + $(PRODUCT_BUNDLE_IDENTIFIER) + CFBundleInfoDictionaryVersion + 6.0 + CFBundleName + $(PRODUCT_NAME) + CFBundlePackageType + APPL + CFBundleShortVersionString + 1.0 + CFBundleVersion + 1 + LSRequiresIPhoneOS + + UILaunchStoryboardName + LaunchScreen + UIMainStoryboardFile + Main + UIRequiredDeviceCapabilities + + armv7 + + UISupportedInterfaceOrientations + + UIInterfaceOrientationPortrait + UIInterfaceOrientationLandscapeLeft + UIInterfaceOrientationLandscapeRight + + UISupportedInterfaceOrientations~ipad + + UIInterfaceOrientationPortrait + UIInterfaceOrientationPortraitUpsideDown + UIInterfaceOrientationLandscapeLeft + UIInterfaceOrientationLandscapeRight + + + diff --git a/Playground/PlaygroundStub/PlaygroundStub/PredictionStubViewController.swift b/Playground/PlaygroundStub/PlaygroundStub/PredictionStubViewController.swift new file mode 100644 index 0000000..de1e625 --- /dev/null +++ b/Playground/PlaygroundStub/PlaygroundStub/PredictionStubViewController.swift @@ -0,0 +1,37 @@ +// +// MLMOJI +// +// This file is part of Alexis Aubry's WWDC18 scolarship submission open source project. +// Copyright © 2018 Alexis Aubry. Available under the terms of the MIT License. +// + +import UIKit + +class PredictionStubViewController: UIViewController { + + var predictionVC: PredictionViewController! + + override func viewDidLoad() { + super.viewDidLoad() + view.backgroundColor = .black + + predictionVC = PredictionViewController() + predictionVC.containerLayoutGuide = view.safeAreaLayoutGuide + + addChildViewController(predictionVC) + view.addSubview(predictionVC.view) + predictionVC.configureConstraints() + + view.backgroundColor = UIColor(patternImage: #imageLiteral(resourceName: "ArtificalBackground")) + predictionVC.view.pinEdges(to: self.view) + + predictionVC.didMove(toParentViewController: self) + + navigationItem.rightBarButtonItem = UIBarButtonItem(title: "Start Predictions", style: .plain, target: self, action: #selector(startButtonTapped)) + } + + @objc func startButtonTapped() { + self.predictionVC.startPredictions() + } + +} diff --git a/README.md b/README.md new file mode 100644 index 0000000..18b55be --- /dev/null +++ b/README.md @@ -0,0 +1,31 @@ +# MLMOJI + +Emoji are becoming more central in the way we express ourselves digitally, whether we want to convey an emotion or put some fun into a conversation. This growth’s impact is visible on mobile devices. The available emoji keep increasing and it often requires a lot of swiping to find the right character. + +My playground’s goal is to research a way to make this process more fun and intuitive. Using touch, Core Graphics, and deep learning, I implemented a keyboard that recognizes hand-drawn emoji and converts them to text. + +You can download the playground book, the data set and the Core ML model from the [releases tab](https://github.com/alexaubry/mlmoji/releases). + +For a live demo, you can watch this video: + +[![MLMOJI Demo](.github/VideoThumbnail.png)](https://www.youtube.com/watch?v=Z7jdLrorctQ) + +## 🔧 Building MLMOJI + +The first step was to create the drawings themselves. I made a view that builds up points as the user’s finger moves on the screen and renders the stroke incrementally. When the user lifts their finger, a `UIGraphicsImageRenderer` flattens the strokes together into a static image, improving rendering performance. To achieve smoother lines, I used touch coalescing, which allows detection of more touch points. + +The second core component of the playground is a classifier that recognizes a drawn emoji. Building it involved three tasks: gathering training data as images, training the model, and improving its accuracy. + +The training data is used by the model to learn the features of each emoji class. This training data came from an app I built that uses the above drawing component to speed up the process of generating drawings. + +With the training images now available, I looked into training a Core ML classifier. For this, a convolutional neural network (CNN) was appropriate because it learns efficiently for image recognition tasks. + +Training a CNN from scratch can take several weeks because of the complexity of the operations applied to the input. Therefore, I used the “transfer learning” technique to train my classifier. This approach enables you to retrain a general, pre-trained model to detect new features. + +Using the TensorFlow ML framework, a Docker container, and a Python script, I was able to train a small, fast, mobile-friendly neural network implementation (MobileNet) with each emoji’s features. I imported the resulting `mlmodel` into my playground. + +The first version of the classifier was too specialized and not very reliable because my original data set was not large enough (only 50 drawings per class). I used data augmentation techniques (such as scaling, distortion, and flipping) to generate more training images from the manual drawings. Then I repeated the training process to reach a more acceptable accuracy. + +Finally, using the Playground Books format, I created an interactive playground that explains the techniques used and demonstrates a proof of concept. Using features like the glossary and live view proxies, the book provides an accessible and enjoyable learning experience. + +The final result comes with limitations. Because of the assignment’s time and size constraints, I was only able to train data for 7 emoji and to reach a somewhat fluctuating level of accuracy. However, building this playground taught me a lot about deep learning techniques for mobile devices and encouraged me to pursue further research in this field. diff --git a/Scholarship.xcworkspace/contents.xcworkspacedata b/Scholarship.xcworkspace/contents.xcworkspacedata new file mode 100644 index 0000000..c541485 --- /dev/null +++ b/Scholarship.xcworkspace/contents.xcworkspacedata @@ -0,0 +1,16 @@ + + + + + + + + + + + diff --git a/Scholarship.xcworkspace/xcshareddata/IDEWorkspaceChecks.plist b/Scholarship.xcworkspace/xcshareddata/IDEWorkspaceChecks.plist new file mode 100644 index 0000000..18d9810 --- /dev/null +++ b/Scholarship.xcworkspace/xcshareddata/IDEWorkspaceChecks.plist @@ -0,0 +1,8 @@ + + + + + IDEDidComputeMac32BitWarning + + +