CornerNet : Detecting Objects as Paired Keypoints

1. CornerNet: Detecting Objects as Paired Keypoints HeiLaw1,2 Jia Deng1,2 Princeton University1, *UniversityofMichigan2 *Work done at the University of Michigan

2. Object Detection Person Person

3. Two-Stage Detector [Girshick et al. CVPR’14] [He et al. ECCV’14][He et al. ICCV’17] [Cai & Vasconcelos, CVPR’18][Singh & Davis, CVPR’18] Region of Interest [Renetal.NIPS’15] Person 2nd Network 1st Network Person Region Pooling [Girshick,ICCV’15]

4. One-stage Detector [Redmon & Farhadi, CVPR’17] [Shen et al. ICCV’17][Liu et al. ECCV’16] [Fu et al. arXiv’17][Lin et al. ICCV’17] [Zhang et al. CVPR’18] Person Class Anchors ConvNet Person Class Anchors Background Class Anchors

5. Drawbacks of Anchor Boxes • Need a large number of anchors • A tiny fraction of anchors are positive examples • Slow down training [Lin et al. ICCV’17] • Extra hyperparameters – sizes and aspect ratios At least one anchor sufficiently overlaps with ground-truth

6. CornerNet: Detecting Objects as Paired Keypoints

7. CornerNet: Detecting Objects as Paired Keypoints Whose Bottom-Right? Whose Top-Left? Bottom-Right Corner? Top-Left Corner? Class Class No Yes Person ConvNet Yes Person No Yes No Person No Yes Person

8. CornerNet: Detecting Objects as Paired Keypoints Whose Bottom-Right? Whose Top-Left? Bottom-Right Corner? Top-Left Corner? Class Class No Yes Person Yes Person No Yes No Person No Yes Person

9. CornerNet: Detecting Objects as Paired Keypoints Whose Bottom-Right? Whose Top-Left? Bottom-Right Corner? Top-Left Corner? Class Class No Yes Person Loss: distance Yes Person No Loss: similarity Yes No Person No Yes Person

10. AssociativeEmbedding[Newelletal.NIPS’17]

11. Advantages of Detecting Corners Corner: 2 sides Center: 4 sides Detecting corner is easier than detecting center

12. Advantages of Detecting Corner w possible proposals corners h Represent possible proposals using only corners

13. Supervising Corner Detection Boxes sufficiently overlap with ground-truths Reducing penalties

14. Corner Pooling

15. Top-Left Corner Pooling max max feature maps

16. Top-Left Corner CornerNet Embedding Class Corner? Top-Left Corner Pooling Yes Person Yes Person Hourglass Network [Newell et al. ECCV’16] Yes Person Bottom-Right Corner Pooling Yes Person Bottom-Right Corner

17. [Lin et al. ECCV’14] One of the most challenging object detection benchmarks

18. Experiment: CornerNet versus Others

19. Experiment: Corner Pooling

22. Related Work Main difference: how grouping is done DeNet: [Tychsen-Smith & Petersson, CVPR’17] Two-stage; grouping in stage 2 by classifier Ours: One-stage; grouping by associative embedding Point Linking Network: [Wang et al. arXiv’17] Grouping by predicting pixel locations Ours: Grouping by associative embedding

23. Conclusions • CornerNet: Detecting objects as pairs of top-left and bottom-right corners • Corner pooling to help better localize corners • State-of-the-art performance among single-stage detectors

24. CornerNet: Detecting Objects as Paired Keypoints Poster P-4B-03 on the 2nd floor of the Philharmonie foyer Code Available At: https://github.com/umich-vl/CornerNet