[Layer] Improve forwarding logic of ConcatLayer @open sesame 08/08 18:22 #2702
Conversation
This PR updates current ConcatLayer forwarding for faster computation. **Changes proposed in this PR:** - Utilize the Tensor::concat() operation to perform forwarding and replace manual mapping and copying. **Self-evaluation:** 1. Build test: [X]Passed [ ]Failed [ ]Skipped 2. Run test: [X]Passed [ ]Failed [ ]Skipped Signed-off-by: Donghyeon Jeong <[email protected]>
djeong20
requested review from
myungjoo,
jijoongmoon,
again4you,
jaeyun-jung,
leemgs,
wooksong,
helloahn,
kparichay,
gichan-jang,
anyj0527,
zhoonit,
lhs8928,
songgot,
jihochu,
DonghakPark,
SeoHyungjun,
baek2sm,
skykongkong8,
EunjuYang and
a team
as code owners
|
📝 TAOS-CI Version: 1.5.20200925. Thank you for submitting PR #2702. Please a submit 1commit/1PR (one commit per one PR) policy to get comments quickly from reviewers. Your PR must pass all verificiation processes of cibot before starting a review process from reviewers. If you are new member to join this project, please read manuals in documentation folder and wiki page. In order to monitor a progress status of your PR in more detail, visit http://ci.nnstreamer.ai/. |
| if (out_dim[axis] != in_dim[axis]) { | ||
| /// @todo Currently a new output tensor is created. This can be optimized. | ||
| Tensor result = Tensor::cat(input_tensors, axis); | ||
| output.copy(result); |
There was a problem hiding this comment.
copy() is unnecessary here. this will be replaced with in-place ops in a later PR.
djeong20
changed the title
| input.reshape(irh); | ||
| original_input_dims.push_back(input.getDim()); | ||
| input.reshape(input_reshape_helper[idx]); | ||
| input_tensors.push_back(input); |
There was a problem hiding this comment.
Just for sure, when we do this push_back, there is no deep copy happened due to the
nntrainer/nntrainer/tensor/tensor.cpp
Line 102 in 32d901c
and could you check whether we are not using leading_helper_dim at #L80
There was a problem hiding this comment.
it seems leading_helper_dim is used in setBatch()!
Description
This pull request aims to optimize the performance of
forwarding()inConcatLayer.This is accomplished by improving the current concatenation logic by utilizing the tensor operation
concat()instead ofMap().Note that this optimization is only for
forwarding()andcalcDeriv()would be improved in later PR.Result
Before
After
Note
input 0: 1:1:196:256 [ FP16 : NCHW ]
input 1: 1:1:196:1024 [ FP16 : NCHW ]
output: 1:1:196:1280 [ FP16 : NCHW ]