Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[phi] Move sequence_pool to phi - Step 3 :sequence_pool_grad_op #52680

Merged
merged 29 commits into from
Apr 27, 2023

Conversation

gouzil
Copy link
Member

@gouzil gouzil commented Apr 8, 2023

PR types

Others

PR changes

Others

Description

sequence_pool 迁移至 phi

相关issues

@paddle-bot
Copy link

paddle-bot bot commented Apr 8, 2023

你的PR提交成功,感谢你对开源项目的贡献!
请关注后续CI自动化测试结果,详情请参考Paddle-CI手册
Your PR has been submitted. Thanks for your contribution!
Please wait for the result of CI firstly. See Paddle CI Manual for details.

@gouzil gouzil changed the title Move sequence_pool_op to phi [phi] Move sequence_pool_grad_op to phi - - Step 2 Apr 10, 2023
@luotao1 luotao1 removed their assignment Apr 11, 2023
@luotao1
Copy link
Contributor

luotao1 commented Apr 11, 2023

将sequence_pooling_test移动至test/cpp/phi/kernels

这个单独提一个PR吧,两个是不同的任务,不同研发review,感谢 @gouzil

@gouzil gouzil changed the title [phi] Move sequence_pool_grad_op to phi - - Step 2 [phi] Move sequence_pool to phi - Step 3 :sequence_pool_grad_op Apr 11, 2023
@gouzil
Copy link
Member Author

gouzil commented Apr 11, 2023

将sequence_pooling_test移动至test/cpp/phi/kernels

这个单独提一个PR吧,两个是不同的任务,不同研发review,感谢 @gouzil

#52782

gouzil added 3 commits April 13, 2023 23:40
…_pool

# Conflicts:
#	test/cpp/phi/kernels/sequence_pooling_test.cc
…_pool

# Conflicts:
#	paddle/fluid/operators/math/CMakeLists.txt
#	paddle/fluid/operators/sequence_ops/sequence_pool_op.cc
#	paddle/fluid/operators/sequence_ops/sequence_pool_op.h
#	paddle/phi/kernels/impl/sequence_pool_kernel_impl.h
#	paddle/phi/kernels/sequence_pool_kernel.h
#	paddle/phi/ops/compat/sequence_pool_sig.cc
#	test/cpp/phi/kernels/sequence_pooling_test.cc
@paddle-ci-bot
Copy link

paddle-ci-bot bot commented Apr 23, 2023

Sorry to inform you that cee77c3's CIs have passed for more than 7 days. To prevent PR conflicts, you need to re-run all CIs manually.

…l/Paddle into move_sequence_pool

# Conflicts:
#	paddle/fluid/operators/sequence_ops/sequence_pool_op.cc
#	paddle/fluid/operators/sequence_ops/sequence_pool_op.h
#	paddle/phi/kernels/gpu/sequence_pool_grad_kernel.cu
@gouzil
Copy link
Member Author

gouzil commented Apr 24, 2023

需要研发大哥帮忙看看, @From00

test_regularizer为例

https://github.com/gouzil/Paddle/blob/993bc412b85114419b0e3baa30583f109cae437b/python/paddle/fluid/tests/unittests/test_regularizer.py#L157

class TestRegularizer(unittest.TestCase):
    def setUp(self):
        self.word_len = 1500
        self.train_data = [
            [(random.sample(range(1000), 10), [0])] for _ in range(2)  # range(1) 时正常
        ]

是在第二轮, 溢出的

https://github.com/gouzil/Paddle/blob/993bc412b85114419b0e3baa30583f109cae437b/python/paddle/fluid/tests/unittests/test_regularizer.py#L184

exe.run(
                main_prog, feed=feeder.feed(data), fetch_list=param_list
            )

测试机型, 测试时大概还有15g空闲内存:

                    'c.          [email protected]
                 ,xNMM.          -----------------------------
               .OMMMMo           OS: macOS 12.4 21F79 x86_64
               OMMM0,            Host: Hackintosh (SMBIOS: MacBookPro14,3)
     .;loddo:' loolloddol;.      Kernel: 21.5.0
   cKMMMMMMMMMMNWMMMMMMMMMM0:    Uptime: 16 hours, 36 mins
 .KMMMMMMMMMMMMMMMMMMMMMMMWd.    Packages: 178 (brew)
 XMMMMMMMMMMMMMMMMMMMMMMMX.      Shell: zsh 5.8.1
;MMMMMMMMMMMMMMMMMMMMMMMM:       Resolution: 1920x1080
:MMMMMMMMMMMMMMMMMMMMMMMM:       DE: Aqua
.MMMMMMMMMMMMMMMMMMMMMMMMX.      WM: Quartz Compositor
 kMMMMMMMMMMMMMMMMMMMMMMMMWd.    WM Theme: Blue (Light)
 .XMMMMMMMMMMMMMMMMMMMMMMMMMMk   Terminal: WarpTerminal
  .XMMMMMMMMMMMMMMMMMMMMMMMMK.   CPU: Intel i7-7700HQ (8) @ 2.80GHz
    kMMMMMMMMMMMMMMMMMMMMMMd     GPU: Intel HD Graphics 630
     ;KMMMMMMMWXXWMMMMMMMk.      Memory: 17175MiB / 32768MiB
       .cooc,.    .,coo:.

@@ -21,6 +21,16 @@ KernelSignature SequencePoolOpArgumentMapping(
{"Out", "MaxIndex"});
}

KernelSignature SequencePoolGradOpArgumentMapping(
const ArgumentMappingContext& ctx) {
return KernelSignature("sequence_pool_grad",
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

需要根据pooltype属性判断是否映射MaxIndex输入,可以参考下batch_norm_sig.cc

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

还是不行, 而且他是挂在pooltype = SUM的时候,而不是MAX的时候

UpdateCTestConfiguration  from :/Users/gouzi/Documents/git/Paddle/build-mac/DartConfiguration.tcl
UpdateCTestConfiguration  from :/Users/gouzi/Documents/git/Paddle/build-mac/DartConfiguration.tcl
Test project /Users/gouzi/Documents/git/Paddle/build-mac
Constructing a list of tests
Done constructing a list of tests
Updating test list for fixtures
Added 0 tests to meet fixture requirements
Checking test dependency graph...
Checking test dependency graph end
test 773
    Start 773: test_regularizer

773: Test command: /Applications/CMake.app/Contents/bin/cmake "-E" "env" "PYTHONPATH=/Users/gouzi/Documents/git/Paddle/build-mac/python" "/usr/local/Cellar/[email protected]/3.10.10/Frameworks/Python.framework/Versions/3.10/bin/python3.10" "/Users/gouzi/Documents/git/Paddle/tools/test_runner.py" "test_regularizer"
773: Working Directory: /Users/gouzi/Documents/git/Paddle/build-mac/python/paddle/fluid/tests/unittests
773: Test timeout computed to be: 150
773: I0424 22:22:26.906788 137655808 interpretercore.cc:267] New Executor is Running.
773: I0424 22:22:26.986169 137655808 interpreter_util.cc:481] Standalone Executor is Used.
773: SequencePoolGradKernel start========
773: pooltype========: SUM
773: in_w: 8
773: out_pos: 0x7faf70270000
773: sizeof out_pos: 8
773: *out_pos: -0.324429
773: sizeof *out_pos: 4
773: in_pos: 0x7faf9830c000
773: sizeof in_pos: 8
773: *in_pos: 1.58627e-42
773: sizeof *in_pos: 4
773: h:10
773: start for VCOPY=====
773: (*in_pos + r * in_w)): 1.58627e-42
773: r: 0
773: (*in_pos + r * in_w)): 7.67557
773: r: 1
773: (*in_pos + r * in_w)): 15.6756
773: r: 2
773: (*in_pos + r * in_w)): 23.6756
773: r: 3
773: (*in_pos + r * in_w)): 31.6756
773: r: 4
773: (*in_pos + r * in_w)): 39.6756
773: r: 5
773: (*in_pos + r * in_w)): 47.6756
773: r: 6
773: (*in_pos + r * in_w)): 55.6756
773: r: 7
773: (*in_pos + r * in_w)): 63.6756
773: r: 8
773: (*in_pos + r * in_w)): 71.6756
773: r: 9
773: end for VCOPY=====
773: SequencePoolGradKernel end===========
773: W0424 22:22:26.994132 84086784 interpretercore.cc:1024] sequence_pool_grad raises an exception std::bad_alloc, std::bad_alloc
773: test_regularizer failed
773:  ..E.
773: ======================================================================
773: ERROR: test_l2 (test_regularizer.TestRegularizer)
773: ----------------------------------------------------------------------
773: Traceback (most recent call last):
773:   File "/Users/gouzi/Documents/git/Paddle/build-mac/python/paddle/fluid/tests/unittests/test_regularizer.py", line 252, in test_l2
773:     framework_l2 = self.check_l2decay_regularizer(place, model)
773:   File "/Users/gouzi/Documents/git/Paddle/build-mac/python/paddle/fluid/tests/unittests/test_regularizer.py", line 214, in check_l2decay_regularizer
773:     param_sum = self.run_program(place, [data, label])
773:   File "/Users/gouzi/Documents/git/Paddle/build-mac/python/paddle/fluid/tests/unittests/test_regularizer.py", line 184, in run_program
773:     out = exe.run(
773:   File "/Users/gouzi/Documents/git/Paddle/build-mac/python/paddle/fluid/executor.py", line 1426, in run
773:     res = self._run_impl(
773:   File "/Users/gouzi/Documents/git/Paddle/build-mac/python/paddle/fluid/executor.py", line 1658, in _run_impl
773:     ret = new_exe.run(
773:   File "/Users/gouzi/Documents/git/Paddle/build-mac/python/paddle/fluid/executor.py", line 654, in run
773:     tensors = self._new_exe.run(
773: RuntimeError: std::exception
773:
773: ----------------------------------------------------------------------
773: Ran 4 tests in 0.240s
773:
773: FAILED (errors=1)
773:
1/2 Test #773: test_regularizer .................***Failed    2.19 sec
test 774
    Start 774: test_regularizer_api

774: Test command: /Applications/CMake.app/Contents/bin/cmake "-E" "env" "PYTHONPATH=/Users/gouzi/Documents/git/Paddle/build-mac/python" "/usr/local/Cellar/[email protected]/3.10.10/Frameworks/Python.framework/Versions/3.10/bin/python3.10" "/Users/gouzi/Documents/git/Paddle/tools/test_runner.py" "test_regularizer_api"
774: Working Directory: /Users/gouzi/Documents/git/Paddle/build-mac/python/paddle/fluid/tests/unittests
774: Test timeout computed to be: 150
774: I0424 22:22:29.102022 401069568 interpretercore.cc:267] New Executor is Running.
774: I0424 22:22:29.159552 401069568 interpreter_util.cc:481] Standalone Executor is Used.
774: SequencePoolGradKernel start========
774: pooltype========: SUM
774: in_w: 8
774: out_pos: 0x7f8542415000
774: sizeof out_pos: 8
774: *out_pos: -0.260606
774: sizeof *out_pos: 4
774: in_pos: 0x7f8520b25000
774: sizeof in_pos: 8
774: *in_pos: 2.76063e+20
774: sizeof *in_pos: 4
774: h:10
773: start for VCOPY=====
774: (*in_pos + r * in_w)): 2.76063e+20
774: r: 0
774: (*in_pos + r * in_w)): 7.73939
774: r: 1
774: (*in_pos + r * in_w)): 15.7394
774: r: 2
774: (*in_pos + r * in_w)): 23.7394
774: r: 3
774: (*in_pos + r * in_w)): 31.7394
774: r: 4
774: (*in_pos + r * in_w)): 39.7394
774: r: 5
774: (*in_pos + r * in_w)): 47.7394
774: r: 6
774: (*in_pos + r * in_w)): 55.7394
774: r: 7
774: (*in_pos + r * in_w)): 63.7394
774: r: 8
774: (*in_pos + r * in_w)): 71.7394
774: r: 9
773: end for VCOPY=====
774: SequencePoolGradKernel end===========
774: W0424 22:22:29.166538 179060736 interpretercore.cc:1024] sequence_pool_grad raises an exception std::bad_alloc, std::bad_alloc
774: test_regularizer_api failed
774:  E.
774: ======================================================================
774: ERROR: test_l2 (test_regularizer_api.TestRegularizer)
774: ----------------------------------------------------------------------
774: Traceback (most recent call last):
774:   File "/Users/gouzi/Documents/git/Paddle/build-mac/python/paddle/fluid/tests/unittests/test_regularizer_api.py", line 163, in test_l2
774:     framework_l2 = self.check_l2decay_regularizer(place, model)
774:   File "/Users/gouzi/Documents/git/Paddle/build-mac/python/paddle/fluid/tests/unittests/test_regularizer_api.py", line 124, in check_l2decay_regularizer
774:     param_sum = self.run_program(place, [data, label])
774:   File "/Users/gouzi/Documents/git/Paddle/build-mac/python/paddle/fluid/tests/unittests/test_regularizer_api.py", line 93, in run_program
774:     out = exe.run(
774:   File "/Users/gouzi/Documents/git/Paddle/build-mac/python/paddle/fluid/executor.py", line 1426, in run
774:     res = self._run_impl(
774:   File "/Users/gouzi/Documents/git/Paddle/build-mac/python/paddle/fluid/executor.py", line 1658, in _run_impl
774:     ret = new_exe.run(
774:   File "/Users/gouzi/Documents/git/Paddle/build-mac/python/paddle/fluid/executor.py", line 654, in run
774:     tensors = self._new_exe.run(
774: RuntimeError: std::exception
774:
774: ----------------------------------------------------------------------
774: Ran 2 tests in 0.184s
774:
774: FAILED (errors=1)
774:
2/2 Test #774: test_regularizer_api .............***Failed    2.17 sec

0% tests passed, 2 tests failed out of 2

Total Test time (real) =   4.64 sec

The following tests FAILED:
	773 - test_regularizer (Failed)
	774 - test_regularizer_api (Failed)
Errors while running CTest
Output from these tests are in: /Users/gouzi/Documents/git/Paddle/build-mac/Testing/Temporary/LastTest.log
Use "--rerun-failed --output-on-failure" to re-run the failed cases verbosely.

有个问题是我在PR-CI-Mac-Python3上看到这个test_sequence_pool这个是挂的,但是我本地是不会挂的

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

sig参数映射需要与OPMaker对齐,仅在pooltypeMAX时映射MaxIndex输入,包括SUM在内的其它情况均没有MaxIndex输入。你现在的sig判断逻辑与OpMaker并不一致,且SUM被拼错成SUN

  void Apply(GradOpPtr<T> op_desc_ptr) const override {
    op_desc_ptr->SetType("sequence_pool_grad");
    op_desc_ptr->SetInput("X", this->Input("X"));
    if (PADDLE_GET_CONST(std::string, this->GetAttr("pooltype")) == "MAX") {
      op_desc_ptr->SetInput("MaxIndex", this->Output("MaxIndex"));
    }
    op_desc_ptr->SetInput(framework::GradVarName("Out"),
                          this->OutputGrad("Out"));
    op_desc_ptr->SetOutput(framework::GradVarName("X"), this->InputGrad("X"));
    op_desc_ptr->SetAttrMap(this->Attrs());
  }

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

为什么SUM也需要3个input

1593: test_sequence_pool failed
1593:  E.E.E.E.E.E.E.E.E.E.E.E.E.E.E.E.E.E.E.E.E.E.......E.E.E.E.E.E.E.E.E.E.E.E.E.E.E.E.
1593: ======================================================================
1593: ERROR: test_check_grad (test_sequence_pool.TestSeqAvgPool)
1593: ----------------------------------------------------------------------
1593: Traceback (most recent call last):
1593:   File "/Users/gouzi/Documents/git/Paddle/build-mac/test/sequence/test_sequence_pool.py", line 105, in test_check_grad
1593:     self.check_grad(["X"], "Out", check_dygraph=False)
1593:   File "/Users/gouzi/Documents/git/Paddle/build-mac/test/sequence/../../python/paddle/fluid/tests/unittests/eager_op_test.py", line 2306, in check_grad
1593:     self.check_grad_with_place(
1593:   File "/Users/gouzi/Documents/git/Paddle/build-mac/test/sequence/../../python/paddle/fluid/tests/unittests/eager_op_test.py", line 2465, in check_grad_with_place
1593:     analytic_grads = self._get_gradient(
1593:   File "/Users/gouzi/Documents/git/Paddle/build-mac/test/sequence/../../python/paddle/fluid/tests/unittests/eager_op_test.py", line 2792, in _get_gradient
1593:     executor.run(
1593:   File "/Users/gouzi/Documents/git/Paddle/build-mac/python/paddle/fluid/executor.py", line 1426, in run
1593:     res = self._run_impl(
1593:   File "/Users/gouzi/Documents/git/Paddle/build-mac/python/paddle/fluid/executor.py", line 1658, in _run_impl
1593:     ret = new_exe.run(
1593:   File "/Users/gouzi/Documents/git/Paddle/build-mac/python/paddle/fluid/executor.py", line 654, in run
1593:     tensors = self._new_exe.run(
1593: ValueError: In user code:
1593:
1593:     File "/Users/gouzi/Documents/git/Paddle/tools/test_runner.py", line 70, in <module>
1593:       main()
1593:     File "/Users/gouzi/Documents/git/Paddle/tools/test_runner.py", line 52, in main
1593:       res = unittest.TextTestRunner(stream=buffer).run(tests)
1593:     File "/usr/local/Cellar/[email protected]/3.10.10/Frameworks/Python.framework/Versions/3.10/lib/python3.10/unittest/runner.py", line 184, in run
1593:       test(result)
1593:     File "/usr/local/Cellar/[email protected]/3.10.10/Frameworks/Python.framework/Versions/3.10/lib/python3.10/unittest/suite.py", line 84, in __call__
1593:       return self.run(*args, **kwds)
1593:     File "/usr/local/Cellar/[email protected]/3.10.10/Frameworks/Python.framework/Versions/3.10/lib/python3.10/unittest/suite.py", line 122, in run
1593:       test(result)
1593:     File "/usr/local/Cellar/[email protected]/3.10.10/Frameworks/Python.framework/Versions/3.10/lib/python3.10/unittest/suite.py", line 84, in __call__
1593:       return self.run(*args, **kwds)
1593:     File "/usr/local/Cellar/[email protected]/3.10.10/Frameworks/Python.framework/Versions/3.10/lib/python3.10/unittest/suite.py", line 122, in run
1593:       test(result)
1593:     File "/usr/local/Cellar/[email protected]/3.10.10/Frameworks/Python.framework/Versions/3.10/lib/python3.10/unittest/case.py", line 650, in __call__
1593:       return self.run(*args, **kwds)
1593:     File "/usr/local/Cellar/[email protected]/3.10.10/Frameworks/Python.framework/Versions/3.10/lib/python3.10/unittest/case.py", line 591, in run
1593:       self._callTestMethod(testMethod)
1593:     File "/usr/local/Cellar/[email protected]/3.10.10/Frameworks/Python.framework/Versions/3.10/lib/python3.10/unittest/case.py", line 549, in _callTestMethod
1593:       method()
1593:     File "/Users/gouzi/Documents/git/Paddle/build-mac/test/sequence/test_sequence_pool.py", line 105, in test_check_grad
1593:       self.check_grad(["X"], "Out", check_dygraph=False)
1593:     File "/Users/gouzi/Documents/git/Paddle/build-mac/test/sequence/../../python/paddle/fluid/tests/unittests/eager_op_test.py", line 2306, in check_grad
1593:       self.check_grad_with_place(
1593:     File "/Users/gouzi/Documents/git/Paddle/build-mac/test/sequence/../../python/paddle/fluid/tests/unittests/eager_op_test.py", line 2465, in check_grad_with_place
1593:       analytic_grads = self._get_gradient(
1593:     File "/Users/gouzi/Documents/git/Paddle/build-mac/test/sequence/../../python/paddle/fluid/tests/unittests/eager_op_test.py", line 2706, in _get_gradient
1593:       self._append_ops(block)
1593:     File "/Users/gouzi/Documents/git/Paddle/build-mac/test/sequence/../../python/paddle/fluid/tests/unittests/eager_op_test.py", line 676, in _append_ops
1593:       op = block.append_op(
1593:     File "/Users/gouzi/Documents/git/Paddle/build-mac/python/paddle/fluid/framework.py", line 4061, in append_op
1593:       op = Operator(
1593:     File "/Users/gouzi/Documents/git/Paddle/build-mac/python/paddle/fluid/framework.py", line 2829, in __init__
1593:       for frame in traceback.extract_stack():
1593:
1593:     InvalidArgumentError: The size of inputs_args names (2) must be equal to the size of kernel input_defs (3).
1593:       [Hint: Expected input_names.size() == input_defs.size(), but received input_names.size():2 != input_defs.size():3.] (at /Users/gouzi/Documents/git/Paddle/paddle/fluid/framework/new_executor/interpreter/data_transfer.cc:681)
1593:       [operator < sequence_pool_grad > error]
1593:

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

这个case下max_index参数是根据不同情况可选的,等价于标记了AsDispensable,所以在kernel声明时应被声明成optional类型,在SUM的情况下这个input就会被跳过。

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done

@gouzil gouzil requested a review from From00 April 26, 2023 03:26
Copy link
Contributor

@From00 From00 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

Copy link
Contributor

@zhwesky2010 zhwesky2010 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM for change unity_build_rule.cmake

@From00 From00 merged commit fe05339 into PaddlePaddle:develop Apr 27, 2023
@gouzil gouzil deleted the move_sequence_pool branch May 15, 2023 14:32
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
contributor External developers
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants