ITADN

SoccerNet-tracking benchmark methodology

#277Openholma91 创建于 2026-02-19
H
holma91commented
Hey! I was looking at the [tracker comparison page](https://trackers.roboflow.com/latest/trackers/comparison/#soccernet-tracking) and I noticed that the reported numbers are quite a bit higher than what's in the published literature, and I couldn't find documentation on the exact evaluation setup. From the original soccernet-tracking paper https://arxiv.org/abs/2204.06918 and from the GTA paper https://arxiv.org/abs/2411.08216, these are the HOTA and MOTA scores for ByteTrack | Source | HOTA | MOTA | | ---------------------------------------------- | ---- | ---- | | SoccerNet paper, w/ GT detections | 71.5 | 94.6 | | SoccerNet paper, w/o GT detections | 47.2 | 31.7 | | GTA paper (2024), w/o GT detections | 67.3 | 84.7 | | Roboflow comparison page | 84.0 | 97.8 | On your comparison page it states that ByteTrack gets 84.0 HOTA and 97.8 MOTA. I presume this is with ground truth detections (but that's not obvious from the docs either)? Even so, these numbers are significantly higher than the GT-detection results from the original paper (71.5 HOTA, 94.6 MOTA), which suggests a different evaluation protocol? Same goes for the SORT numbers. It would be helpful if the comparison page documented the evaluation setup and ideally provided a one-off script to get the exact score, so the numbers can be interpreted relative to the published baselines, because right now these numbers are very different from what can be found in any published papers Love the work you guys are doing for open source btw!
4 条评论