Concise-iPython-Notebooks-for-Deep-learning-on-Images-and-text

This github repository is a collection of iPython notebooks for solving different problems in deep learning using keras API and tensorflow backend. The notebooks are written in as easy to read manner as possible. Everyone is welcome to openly contribute towards this repository and add/ recommend to add more useful and interesting notebooks.

The notebooks having examples of image processing deals with problems like:

1. Image Segmentation to segment (find the boundary of) certain object in an image, was performed using U-Net architecture of the auto encoder model.

2. Object Detection was done using Single Shot MultiBox Detector (SSD) model.

3. Image Classification was done using convolutional network.

4. The task of finding duplicate images among the given set of images is done here.

The notebooks meant for processing/ understanding texts deals with problems like:

1. Basic entity extraction from text using Named Entity Recognition and tagging the text using POS taggers.

2. Topic modelling using LDA and converting pdf documents to readable text format in python.

3. Classification of text queries into positive or negative comments. GloVe and FastText embedding were used and multiple architectures including bidirectional GRU,LSTM, Attention networks and combinations of these were experimented with.

4. Relation extraction from a sentence, between 2 entities. A model which was aware of the entity words when finding the relation was made. Self-attention and GRU was used for feature extraction

5. Intent classifier was made to classify incoming queries into one of the intents. This can be used to understand the type of query a user is making in a chat-bot application. Intent can be to book a cab, or to find a restaurant, etc.

A demo program for showing use of machine learning to produce results for different input values is done. This work is done for purpose of demo, which allows the user to change the features of the model using an interactive web app and the output is produced accordingly and shown in the web app.

Motivation

I wanted to have this repository for everyone with good understanding of theory in Deep Learning, to have easy to read reference for some common tasks in Deep Learning.

Packages used

Common Packages – Tensorflow, Tensorboard, keras, sklearn, numpy, pandas, mlflow

Text Based Packages – NLTK, genism, pdfminer, keras_self_attention, keras_multi_head

Image Based Packages – openCV, matplotlib

Code style

Standard. Python.

Index

Text

1. Text Classifier – Have shown examples of different models to classify IMDB dataset . Text classifiers are very useful in tasks like passing queries to relevant department, in understanding customer review like in case of this dataset.

   1.1. Text_classifier – Performs text classification by using different architecture/ layers like GRU, LSTM, Sequence-self-attention, multi-head-attention, global max pooling and global average pooling. Different combinations of above layers can be used by passing arguments to a function to train the different models. GloVe and FastText embeddings have been experimented with.

   1.2. Text_Classifer_2 – Here GloVe and FastText embeddings have been used as different features and there features are concatenated just before the dense layers. The final f1 score (90.5) for this notebook is highest among all the methods.

2. Relation Extraction – Data from SemEval_task8 is used to show an example of finding relations between two entities in a sentence. A keras model is built with concatenation of RNN features, self-attention features and max pooled features of the entities in the sentence.

3. Intent Classifier – Intent classifier is performed on a dataset containing 7 different intents. This is an example for how deep learning models can be successfully used to understand the intent of a text query. Customer intent can be determined and then a prewritten text can be generated to answer the user query. A simple yet effective chat-bot can be built this way, depending on the different intents possible and data available for each of those intients.

4. Miscellaneous –

   4.1. POS tagger – POS tagging is to find the part-of-speech tag of the words in the sentence. The tags of the words can be passed as information to neural networks.

       4.1.1. NLTK POS tagger – Using NLTK package to perform the POS tagging.

       4.1.2. Stanford POS tagger – Using pre-trained model of Stanford for POS tagging.

   4.2. NER tagger – NER tagging or Named Entity Recognition is to find some common entities in text like name, place, etc. or more subject dependent entity like years of experience, skills, etc. can all be entities while parsing a resume. NER are generally used to find some important words in the document and one can train their own document specific NER tagger.

       4.2.1. Stanford NER tagger – Pre-trained NER provided by the Stanford libraries for entities – Person, organization, location.

       4.2.2. Self-trained keras model – An example of training your own NER model. (To be done)

   4.3. PDF to Doc – a very useful tool to read the pdf documents in python. PDFminer package is used here.

   4.4. RegEx – Some powerful and commonly used Regular Expressions.

   4.5. Embeddings – A document going through different embeddings including sentence embedding.

   4.6. Topic Modelling – Topic modelling in text processing is to cluster document into topics according to the word frequencies, or basically, sentences in the document. Each topic are dominated by certain words which can also be extracted. Here topic modelling is done using LDA from sklearn and genism packages.

Image

1. Image Segmentation – Image segmentaion or Pixel wise segmentation is a task in which each pixel in the image are classified into 2 or more classes. All the notebooks here have an auto encoder model with U-Net architecture to find the lung lesions from the CT images.

   1.1. lungs_conv_unet - An autoencoder model with U-Net architecture is used.

   1.2. lungs_incp_unet - Here convolution layers are replaced with inception blocks.

   1.3. lungs_incp_unet_snapshot - Model exactly same as the lungs_incp_unet model with the addition of cosine annealed Learning rate.

   1.4. lungs_incp_unet_snapshot_mlflow - Contains files for lungs_incp_unet_snapshot model deployed using mlflow workflow.

2. Single Shot MultiBox Detector (SSD) model – An example implementation of the SSD model is shown for objecct detection in pascal VOC dataset.

3. Image Classification – Image classification is a task of classifying images into two or more classes. A simple yet powerful neural network models are built to classify the images.

   3.1 skin_cancer_classification_1 - A deep neural network model, with convolution layers for feature extraction.

   3.2. skin_cancer_classification_2 - Model is same as the skin_cancer_classification_1, only with the addition of cosine annealed learning rate during the training.

4. Finding Duplicates - Notebook to find duplicate images by comparing image vectors of the image using Locality Sensitivity Hashing for efficient search.

Reproducibility

To run the code in local machine, requried packasges will need to be installed and the dataset must be downloaded from the links provided.

If someone chose to run the programs online, google colab provides free GPU access. Also this link can be useful for easily using kaggle datasets in googlecolab environment.