libxsmm and other frameworks

[cvasfi]

our sparse convolution routine will be migrated to libxsmm library so that it can be also used by other frameworks like TensorFlow

I suppose libxsmm is already used in the project now. Does it mean that it can be used with tensorflow? If so, could you provide a short example/guideline on how to use/test it?

[jspark1105]

You're right that SkimCaffe is already using libxsmm but only for sparse-matrix-times-dense-matrix (SpMDM) multiplication for fully-connected layers. Unfortunately, sparse convolution routine (in include/caffe/utils/sconv.hpp) has not been integrated into libxsmm. It has been on my TODO list so I'll try to do that as soon as I have time. It shouldn't be hard and will be mostly a matter of matching with the sparse matrix format used by SpMDM in libxsmm. Once integrated, we should be able to add a TF op that uses the libxsmm sparse convolution and SpMDM routines to speedup conv and fc layers.

[hfp]

Since you asked if any of LIBXSMM's sparse operations can be used in TF, I want to point you to:

TFROOT/tensorflow/python/kernel_tests/sparse_matmul_op_test.py

This is the test case covering SpMDM, but it is a regular Python script orchestrating TF's frontend. This should provide a reasonable piece of sample code as well. If you want to see the implementation, you may look at https://github.com/tensorflow/tensorflow/blob/master/tensorflow/core/kernels/sparse_matmul_op.cc. As a side note, TF also integrates LIBXSMM's convolution kernels (non-sparse).

[cvasfi]

@jspark1105 Thank you for the reply that's great, i'll be following the news.

@hfp Thanks, that looks quite interesting as well.