【Hackathon 5th No.47】API转换 103-124 #346

co63oc · 2023-12-10T02:29:32Z

PR Docs

https://github.com/PaddlePaddle/community/blob/master/hackathon/hackathon_5th/%E3%80%90PaddlePaddle%20Hackathon%205th%E3%80%91%E5%BC%80%E6%BA%90%E8%B4%A1%E7%8C%AE%E4%B8%AA%E4%BA%BA%E6%8C%91%E6%88%98%E8%B5%9B%E6%A1%86%E6%9E%B6%E5%BC%80%E5%8F%91%E4%BB%BB%E5%8A%A1%E5%90%88%E9%9B%86.md#no47%E4%B8%BApaddle%E4%BB%A3%E7%A0%81%E8%BD%AC%E6%8D%A2%E5%B7%A5%E5%85%B7%E6%96%B0%E5%A2%9Eapi%E8%BD%AC%E6%8D%A2%E8%A7%84%E5%88%99
API转换 103-124

文档PR PaddlePaddle/docs#6380
torch.histogramdd
torch.nn.functional.pdist PaddlePaddle/docs#6400

torch.nn.Module.type是按torch.nn.Module 匹配，如果参数带有名称dst_type解析出错，不确定怎么修改paconvert

torch.Tensor.index_fill_ Bug PR PaddlePaddle/Paddle#59863 已修改
torch.Tensor.signbit Bug PR: PaddlePaddle/Paddle#60150

PR APIs

103	torch.histogramdd		  增加文档
104	torch.index_fill / torch.Tensor.index_fill / torch.Tensor.index_fill_	增加 torch.index_fill 文档 
105	torch.masked_fill / torch.Tensor.masked_fill / torch.Tensor.masked_fill_	增加 torch.masked_fill 文档
106	torch.masked_select / torch.Tensor.masked_select	增加 torch.Tensor.masked_select 文档
107	torch.hypot / torch.Tensor.hypot / torch.Tensor.hypot_	增加 torch.Tensor.hypot_ 文档，原文档组合替代实现，修改文档
108	torch.special.multigammaln	增加文档
109	torch.combinations	增加文档
110	torch.Tensor.to	paddle.Tensor.to  修改文档
111	torch.nn.Module.type	paddle.nn.Layer.astype   修改文档
112	torch.optim.lr_scheduler.LinearLR	增加文档
113	torch.optim.lr_scheduler.CosineAnnealingWarmRestarts	增加文档
114	torch.utils.data.ConcatDataset	增加文档
115	torch.utils.data.SubsetRandomSampler	已有文档验证无误 	
116	torch.select_scatter / torch.Tensor.select_scatter         增加文档
117	torch.diagonal_scatter / torch.Tensor.diagonal_scatter	已有文档验证无误 
118	torch.signbit / torch.Tensor.signbit	  增加文档
119	torch.nn.functional.pdist	              增加文档
120	torch.nn.AdaptiveLogSoftmaxWithLoss	        任务开发中
121	torch.atleast_1d / atleast_2d / atleast_3d	已有文档验证无误 
122	torch.linalg.householder_product		增加文档

paddle-bot · 2023-12-10T02:29:36Z

Thanks for your contribution!

luotao1 · 2023-12-18T12:00:56Z

pdist 也已经合入了

zhwesky2010

所有单测都必须符合：

单测覆盖范围要求为：涉及到多个API形参的，应包含各种参数用法（ 全部指定关键字、全部不指定关键字、改变关键字顺序、默认参数均不指定 四种情况必须考虑），不能只考虑最简单常见的用法，要求至少列举5种不同的使用case（越多越好）。

因为目前正在做单测规范化的工作，为避免后续被重构，新合入的都需要规范。

zhwesky2010 · 2023-12-19T07:29:42Z

paconvert/api_matcher.py

+        new_kwargs = self.parse_kwargs(kwargs)
+        if new_kwargs is None:
+            new_kwargs = {}
+        if "copy" in new_kwargs:


这个是不是考虑json配置下，直接kwargs_change成空字符串

kwargs_change 使用GenericMatcher，GenericMatcher 中 dtype, device转换不同不适合使用，然后只有参数 copy ，不需要使用 kwargs_change

zhwesky2010 · 2023-12-19T07:31:31Z

paconvert/api_matcher.py

+        if "memory_format" in new_kwargs:
+            new_kwargs.pop("memory_format")
+        if "non_blocking" in new_kwargs:
+            new_kwargs["blocking"] = "not " + new_kwargs.pop("non_blocking").strip("()")


有个 Reverse* 的Matcher可以参考，看是不是统一成一个功能更强的Matcher更好

是 ReverseMomentumMatcher ，输入是数值，类型不同

tests/test_Tensor_diagonal_scatter.py

tests/test_Tensor_index_fill_.py

zhwesky2010 · 2023-12-19T07:40:13Z

tests/test_atleast_1d.py

+    pytorch_code = textwrap.dedent(
+        """
+        import torch
+        result = torch.atleast_1d(torch.tensor(123, dtype=torch.int32))


这个API有输入关键字的用法吗

参数是*tensors，没有关键字用法

https://pytorch.org/docs/stable/_modules/torch/functional.html#atleast_1d 源码

运行设置tensors提示没有关键字

tests/test_index_fill.py

zhwesky2010 · 2023-12-19T07:51:28Z

tests/test_nn_Module_type.py

 def _test_case_2():
    pytorch_code = textwrap.dedent(
        """
        import torch
        x = torch.tensor([1., 2., 3.])
        module1 = torch.nn.Module()
        module1.register_buffer('buffer', x)
-        module1.type_unsupport(dst_type=torch.float32)
+        module1.type(dst_type=torch.float32)


因为这个提前被torch.Tensor.type拦截了，我建议你将这一块功能融到TensorTypeMatcher里，在其中增加对dst_type参数的处理。

torch.nn.Module.type 和 torch.Tensor.type 两者转完后都是*.astype，代码是一致的，因此即使被torch.Tensor.type误判了，也是可以实现的。

zhwesky2010 · 2023-12-19T07:53:53Z

tests/test_optim_lr_scheduler_CosineAnnealingWarmRestarts.py

+
+def test_case_1():
+    pytorch_code = textwrap.dedent(
+        generate_torch_code(


这个应该冲突了

zhwesky2010 · 2023-12-19T07:54:00Z

tests/test_optim_lr_scheduler_LinearLR.py

-        opt = torch.optim.Adam(model.parameters())
-        result = torch.optim.lr_scheduler.LinearLR(opt, start_factor=0.5, total_iters=4)
-        """
+        generate_torch_code("torch.optim.lr_scheduler.LinearLR(sgd, verbose=True)")


tests/test_select_scatter.py

zhwesky2010 · 2023-12-19T07:55:59Z

@co63oc lr_scheduler这个地方的名字应该有改动了

co63oc · 2023-12-19T09:01:57Z

lr_scheduler

已修改

zhwesky2010 · 2023-12-21T10:00:47Z

paconvert/api_mapping.json

+    "paddle_api": "paddle.atleast_2d",
+    "min_input_args": 1,
+    "args_list": [
+      "*"


已修改 *tensors

zhwesky2010 · 2023-12-21T10:00:52Z

paconvert/api_mapping.json

+    "paddle_api": "paddle.atleast_3d",
+    "min_input_args": 1,
+    "args_list": [
+      "*"


zhwesky2010 · 2023-12-21T10:01:26Z

paconvert/api_mapping.json

+    "paddle_api": "paddle.atleast_1d",
+    "min_input_args": 1,
+    "args_list": [
+      "*"


要写成*tensors，这个是可变参数，*表示的是位置参数与关键字参数的分割，是不同的含义

zhwesky2010 · 2023-12-21T10:27:02Z

paconvert/api_matcher.py

+    def get_paddle_nodes(self, args, kwargs):
+        new_args = self.parse_args(args)
+        new_kwargs = self.parse_kwargs(kwargs)
+        if new_kwargs is None:


什么时候会return None，return None时一般是不支持，应该继续return None

API有bug时返回None，已修改return None

zhwesky2010

LGTM

Add tests

31b37e0

paddle-bot bot added the contributor External developers label Dec 10, 2023

co63oc mentioned this pull request Dec 10, 2023

【Hackathon 5th No.47】API转换 103-124 -part PaddlePaddle/docs#6380

Merged

luotao1 mentioned this pull request Dec 11, 2023

【PaddlePaddle Hackathon 5th】开源贡献个人挑战赛 PaddlePaddle/Paddle#57262

Open

luotao1 assigned luotao1 and zhwesky2010 Dec 11, 2023

Fix

8889264

co63oc force-pushed the group1 branch from ac9fd34 to 8889264 Compare December 18, 2023 08:08

Fix

16c73a4

zhwesky2010 reviewed Dec 19, 2023

View reviewed changes

Fix

83a94a9

Merge branch 'master' into group1

f4b789e

co63oc force-pushed the group1 branch 2 times, most recently from 5e03cc4 to e42a5bf Compare December 21, 2023 00:47

Fix

8252a5a

co63oc force-pushed the group1 branch from 37aaf0a to 8252a5a Compare December 21, 2023 01:16

co63oc added 2 commits December 21, 2023 09:36

Fix

f59a4d1

Fix

c1b69fb

PaddlePaddle locked and limited conversation to collaborators Dec 21, 2023

PaddlePaddle unlocked this conversation Dec 21, 2023

zhwesky2010 reviewed Dec 21, 2023

View reviewed changes

Fix

ab3281f

zhwesky2010 approved these changes Dec 21, 2023

View reviewed changes

luotao1 merged commit c1fb632 into PaddlePaddle:master Dec 21, 2023
8 checks passed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

【Hackathon 5th No.47】API转换 103-124 #346

【Hackathon 5th No.47】API转换 103-124 #346

co63oc commented Dec 10, 2023 •

edited

Loading

paddle-bot bot commented Dec 10, 2023

luotao1 commented Dec 18, 2023

zhwesky2010 left a comment

zhwesky2010 Dec 19, 2023

co63oc Dec 19, 2023 •

edited

Loading

zhwesky2010 Dec 19, 2023 •

edited

Loading

co63oc Dec 19, 2023 •

edited

Loading

zhwesky2010 Dec 19, 2023

co63oc Dec 19, 2023 •

edited

Loading

zhwesky2010 Dec 19, 2023

co63oc Dec 19, 2023 •

edited

Loading

zhwesky2010 Dec 19, 2023

co63oc Dec 19, 2023

zhwesky2010 Dec 19, 2023

co63oc Dec 19, 2023 •

edited

Loading

zhwesky2010 commented Dec 19, 2023

co63oc commented Dec 19, 2023 •

edited

Loading

zhwesky2010 Dec 21, 2023

co63oc Dec 21, 2023

zhwesky2010 Dec 21, 2023

co63oc Dec 21, 2023

zhwesky2010 Dec 21, 2023

co63oc Dec 21, 2023

zhwesky2010 Dec 21, 2023

co63oc Dec 21, 2023

zhwesky2010 left a comment

【Hackathon 5th No.47】API转换 103-124 #346

【Hackathon 5th No.47】API转换 103-124 #346

Conversation

co63oc commented Dec 10, 2023 • edited Loading

PR Docs

PR APIs

paddle-bot bot commented Dec 10, 2023

luotao1 commented Dec 18, 2023

zhwesky2010 left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

co63oc Dec 19, 2023 • edited Loading

Choose a reason for hiding this comment

zhwesky2010 Dec 19, 2023 • edited Loading

Choose a reason for hiding this comment

co63oc Dec 19, 2023 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

co63oc Dec 19, 2023 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

co63oc Dec 19, 2023 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

co63oc Dec 19, 2023 • edited Loading

Choose a reason for hiding this comment

zhwesky2010 commented Dec 19, 2023

co63oc commented Dec 19, 2023 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

zhwesky2010 left a comment

Choose a reason for hiding this comment

co63oc commented Dec 10, 2023 •

edited

Loading

co63oc Dec 19, 2023 •

edited

Loading

zhwesky2010 Dec 19, 2023 •

edited

Loading

co63oc Dec 19, 2023 •

edited

Loading

co63oc Dec 19, 2023 •

edited

Loading

co63oc Dec 19, 2023 •

edited

Loading

co63oc Dec 19, 2023 •

edited

Loading

co63oc commented Dec 19, 2023 •

edited

Loading