Documentation listing supported layers for Keras, Caffe and TensorFlow #409
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,176 @@ | ||
| # Layers supported in Caffe, Keras, and Tensorflow | ||
| Below is are table showing which layers are supported by Caffe, Keras, and Tensorflow: | ||
|
|
||
| ### Core Layers | ||
| | Layer | Caffe | Keras | Tensorflow | | ||
| | :-----------------------: | :-----------: | :----------: | :---------: | | ||
| | Activation | √ | √ | √ | | ||
| | ActivityRegularization | √ | √ | √ | | ||
| | Dense | × | √ | √ | | ||
| | Dropout | √ | √ | √ | | ||
| | Flatten | √ | √ | √ | | ||
| | Lambda | × | √ | √ | | ||
| | Masking | √ | √ | √ | | ||
| | Permute | × | √ | √ | | ||
| | Repeat Vector | × | √ | √ | | ||
| | Reshape | √ | √ | √ | | ||
| | Spatial Dropout 1D | × | √ | √ | | ||
| | Spatial Dropout 2D | × | √ | √ | | ||
| | Spatial Dropout 3D | × | √ | √ | | ||
|
|
||
| #### Convolutional Layers | ||
| | Layer | Caffe | Keras | Tensorflow | | ||
| | :-----------------------: | :-----------: | :----------: | :---------: | | ||
| | Conv1D | √ | √ | √ | | ||
| | Conv2D | √ | √ | √ | | ||
| | DepthwiseConv2D | × | × | √ | | ||
| | SeperableConv1D | × | √ | √ | | ||
| | SeperableConv2D | × | √ | √ | | ||
| | Conv2DTranspose | √ | √ | √ | | ||
| | Conv3D | √ | √ | √ | | ||
| | Conv3DTranspose | √ | √ | √ | | ||
| | Cropping1D | √ | √ | √ | | ||
| | Cropping2D | √ | √ | √ | | ||
| | Cropping3D | √ | √ | √ | | ||
| | Upsampling 1D | √ | √ | √ | | ||
| | Upsampling 2D | √ | √ | √ | | ||
| | Upsampling 3D | √ | √ | √ | | ||
| | ZeroPadding 1D | × | √ | √ | | ||
| | ZeroPadding 2D | × | √ | √ | | ||
| | ZeroPadding 3D | × | √ | √ | | ||
| | Im2Col | √ | × | × | | ||
| | Spatial Pyramid Pooling | √ | × | × | | ||
| * Upsampling in Caffe can be done by using methods shown here: https://gist.github.com/tnarihi/54744612d35776f53278 | ||
|
|
||
| ### Pooling Layers | ||
| | Layer | Caffe | Keras | Tensorflow | | ||
| | :-----------------------: | :-----------: | :----------: | :---------: | | ||
| | MaxPooling1D | √ | √ | √ | | ||
| | MaxPooling2D | √ | √ | √ | | ||
| | MaxPooling3D | √ | √ | √ | | ||
| | AveragePooling1D | √ | √ | √ | | ||
| | AveragePooling2D | √ | √ | √ | | ||
| | AveragePooling3D | √ | √ | √ | | ||
| | GlobalMaxPooling1D | × | √ | √ | | ||
| | GlobalAveragePooling1D | × | √ | √ | | ||
| | GlobalMaxPooling2D | × | √ | √ | | ||
| | GlobalAveragePooling2D | × | √ | √ | | ||
| | GlobalMaxPooling3D | × | √ | √ | | ||
| | GlobalAveragePooling3D | × | √ | √ | | ||
| | Stochastic Pooling | √ | × | × | | ||
|
|
||
| ### Locally-connected Layers | ||
| | Layer | Caffe | Keras | Tensorflow | | ||
| | :-----------------------: | :-----------: | :----------: | :---------: | | ||
| | LocallyConnected1D | × | √ | √ | | ||
| | LocallyConnected2D | × | √ | √ | | ||
|
|
||
| ### Recurrent Layers | ||
| | Layer | Caffe | Keras | Tensorflow | | ||
| | :-----------------------: | :-----------: | :----------: | :---------: | | ||
| | RNN | √ | √ | √ | | ||
| | SimpleRNN | × | √ | √ | | ||
| | GRU | × | √ | √ | | ||
| | LSTM | √ | √ | √ | | ||
| | ConvLSTM2D | × | √ | √ | | ||
| | SimpleRNNCell | × | √ | √ | | ||
| | GRUCell | × | √ | √ | | ||
| | LSTMCell | × | √ | √ | | ||
| | CuDDNGRU | × | √ | √ | | ||
| | CuDDNLSTM | × | √ | √ | | ||
| | StackedRNNCell | × | × | √ | | ||
|
|
||
| ### Embedding Layers | ||
| | Layer | Caffe | Keras | Tensorflow | | ||
| | :-----------------------: | :-----------: | :----------: | :---------: | | ||
| | Embedding | √ | √ | √ | | ||
|
|
||
| ### Merge Layers | ||
| | Layer | Caffe | Keras | Tensorflow | | ||
| | :-----------------------: | :-----------: | :----------: | :---------: | | ||
| | Add | × | √ | √ | | ||
| | Subtract | × | √ | √ | | ||
| | Multiply | × | √ | √ | | ||
| | Average | × | √ | √ | | ||
| | Minium | × | × | √ | | ||
| | Maximum | × | √ | √ | | ||
| | Concatenate | √ | √ | √ | | ||
| | Dot | × | √ | √ | | ||
|
|
||
| ### Activations Layers | ||
| | Layer | Caffe | Keras | Tensorflow | | ||
| | :-----------------------: | :-----------: | :----------: | :---------: | | ||
| | ReLu | √ | √ | √ | | ||
| | LeakyReLu | √ | √ | √ | | ||
| | PReLU | √ | √ | √ | | ||
| | ELU | √ | √ | √ | | ||
| | ThresholdedReLU | √ | √ | √ | | ||
| | Softmax | √ | √ | √ | | ||
| | Argmax | √ | × | × | | ||
| | Sigmoid | √ | √ | √ | | ||
| | TanH | √ | √ | √ | | ||
| | Absolute Value | √ | × | × | | ||
| | Power | √ | √ | × | | ||
| | Exp | √ | √ | × | | ||
| | Linear | × | √ | √ | | ||
| | Log | √ | √ | × | | ||
| | BNLL | √ | × | × | | ||
| | Bias | √ | × | × | | ||
| | Scale | √ | × | × | | ||
|
|
||
| ### Utility Layers | ||
| | Layer | Caffe | Keras | Tensorflow | | ||
| | :-----------------------: | :-----------: | :----------: | :---------: | | ||
| | Argmax | √ | × | × | | ||
| | Slicing | √ | × | × | | ||
| | Eltwise | √ | × | × | | ||
| | Parameter | √ | × | × | | ||
| | Reduction | √ | × | × | | ||
| | Silence | √ | × | × | | ||
|
|
||
| ### Loss Layers | ||
| | Layer | Caffe | Keras | Tensorflow | | ||
| | :-----------------------: | :-----------: | :----------: | :---------: | | ||
| | Multinomial Logistic Loss | √ | × | × | | ||
| | Infogain Loss | √ | × | × | | ||
| | Softmax with Loss | √ | × | √ | | ||
| | Sum-of-Squares/Euclidean | √ | × | × | | ||
| | Hinge / Margin | √ | √ | √ | | ||
| | Sigmoid Cross-Entropy Loss| √ | × | √ | | ||
| | Accuracy / Top-k layer | √ | × | × | | ||
| | Contrastive Loss | √ | × | × | | ||
|
|
||
| ### Normalization Layers | ||
| | Layer | Caffe | Keras | Tensorflow | | ||
| | :-----------------------: | :-----------: | :----------: | :---------: | | ||
| | BatchNormalization | √ | √ | √ | | ||
| | LRN | √ | × | × | | ||
| | MVN | √ | × | × | | ||
|
|
||
| ### Noise Layers | ||
| | Layer | Caffe | Keras | Tensorflow | | ||
| | :-----------------------: | :-----------: | :----------: | :---------: | | ||
| | GaussianNoise | × | √ | √ | | ||
| | GaussianDropout | √ | √ | √ | | ||
| | AlphaDropout | √ | √ | √ | | ||
|
|
||
| ### Layer Wrappers | ||
| | Layer | Caffe | Keras | Tensorflow | | ||
| | :-----------------------: | :-----------: | :----------: | :---------: | | ||
| | TimeDistributed | × | √ | √ | | ||
| | Bidirectional | × | √ | √ | | ||
|
|
||
| ### Custom Layers | ||
| | Caffe | Keras | Tensorflow | | ||
| | :-----------: | :----------: | :---------: | | ||
| | √ | √ | Use Keras API for custom layers | | ||
|
There was a problem hiding this comment. You can add LRN as an example in custom layers supported. |
||
|
|
||
| ## Additional Notes: | ||
|
There was a problem hiding this comment. You can add a markdown link on here. |
||
| * Keras does not support the LRN layer used in Alexnet & many other models. To use the LRN layer refer to here: https://github.com/Cloud-CV/Fabrik/blob/master/tutorials/keras_custom_layer_usage.md. | ||
| * Documentation for writing your own Keras layers is found here: https://keras.io/layers/writing-your-own-keras-layers/ | ||
|
There was a problem hiding this comment. You can add a markdown link on here. |
||
| * Caffe 3D layers can be used in Caffe 2.0 | ||
|
There was a problem hiding this comment. We haven't tested any features for caffe 2.0 yet, have you tested this if not we can drop this line. |
||
|
|
||
| ## Documentation for Caffe, Keras, and Tensorflow layers | ||
| * Documentation for all Keras Layers is found here: https://keras.io/layers/about-keras-layers/ | ||
|
There was a problem hiding this comment. You can add a markdown link on |
||
| * Documentation for all Caffe Layers is found here: http://caffe.berkeleyvision.org/tutorial/layers.html | ||
|
|
||
| * Documentation for all Tensorflow Layers is found here: https://www.tensorflow.org/api_docs/python/tf/layers | ||
|
|
||
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Grammatical mistake.