[wip] group op #1202
[wip] group op #1202
Conversation
| }, | ||
| [](at::xpu::XPUStream&, | ||
| c10::intrusive_ptr<ProcessGroupXCCL::WorkXCCL>&) { | ||
| ccl::group_end(); |
There was a problem hiding this comment.
I think groupStart/groupEnd wraps ccl::group_start/ccl::group_end. Then should you call the wrapped API?
There was a problem hiding this comment.
xcclActiveGroupCounter_ affect batchP2P choice. lets use origin api like nccl
src/xccl/ProcessGroupXCCL.cpp
Outdated
| return true; | ||
| } | ||
|
|
||
| void check_xpu_single_tensor( |
There was a problem hiding this comment.
Should you follow the same naming format like checkSingleTensor?
src/xccl/ProcessGroupXCCL.cpp
Outdated
| } | ||
| } | ||
| } | ||
|
|
||
| int64_t check_xpu_tensors_same_device(const std::vector<at::Tensor>& tensors) { |
There was a problem hiding this comment.
Should you follow the same naming format like checkTensorOnSameDevice?
| @@ -62,6 +109,10 @@ ccl::reduction getXcclReduceOp(const ReduceOp& reduceOp, at::Tensor& input) { | |||
| // Map sum to max for bool tensors to avoid overflow issues with sum. | |||
| return ccl::reduction::max; | |||
| } | |||
| // WA due to oneCCL not support AVG | |||
| if (reduceOp == ReduceOp::AVG) { | |||
There was a problem hiding this comment.
The WA does not mean simply replacing avg with sum, but using sum collective and div SYCL kernel to simulate avg. Please update your comment.
There was a problem hiding this comment.
Please also add comments that oneCCL is expected to support avg in basekit 2025.2 release.
src/xccl/ProcessGroupXCCL.cpp
Outdated
| @@ -31,22 +31,69 @@ const std::map<at::ScalarType, ccl::datatype> xcclDatatypes = { | |||
| {at::kFloat8_e5m2fnuz, ccl::datatype::uint8}, | |||
| }; | |||
|
|
|||
| void checkXPUTensor(at::Tensor& tensor) { | |||
| bool check_same_size(const std::vector<at::Tensor>& input_tensors) { | |||
No description provided.