Why does the H-Deformable-DETR can retain the end-to-end capabilities?

[volcanolee4]

Thanks for your work!
I wonder why the one2many branch doesn't destroy the end2end ability of DETR?
In the training phase,he only thing it looks different is an attention mask, the only difference between the one2one branch and one2many branch is that the proposals in one2one brach are topk300, while the proposals in one2many branch are topk300-1800.