Live Demonstration: Transformer-Based Visual Object Detection and Tracking on MPSoC
Visual object detectors with transformers are very demanding models for hardware constrained devices such as FPGAs or custom solutions. This live demonstration shows a design that integrates our own nano version of the off-the-shelf RT-DETR transformer with a custom tracker. Our design provides a frame rate of more than 5 fps with images of 640×480 pixels for tracking faces on a ZCU-102 MPSoC with a drop in AP50 precision for object detection from 53.6 in floating-point representation to 51.9 with our own nano model quantized to INT8. The tracker keeps the identity of a given face for its whole trajectory while in the field of view of the camera, addressing the lack of detections at faster frame rates.
keywords: Visual object detectors, Tracking, MPSoC, FPGA, Embedded devices, Edge computing
Publication: Congress
1782990009254
July 2, 2026
/research/publications/live-demonstration-transformer-based-visual-object-detection-and-tracking-on-mpsoc
Visual object detectors with transformers are very demanding models for hardware constrained devices such as FPGAs or custom solutions. This live demonstration shows a design that integrates our own nano version of the off-the-shelf RT-DETR transformer with a custom tracker. Our design provides a frame rate of more than 5 fps with images of 640×480 pixels for tracking faces on a ZCU-102 MPSoC with a drop in AP50 precision for object detection from 53.6 in floating-point representation to 51.9 with our own nano model quantized to INT8. The tracker keeps the identity of a given face for its whole trajectory while in the field of view of the camera, addressing the lack of detections at faster frame rates. - M. Romero-Romero, M. Bendaña, P. Gil-P´erez, D. Cores, F. Pardo, V.M. Brea and M. Mucientes - 10.1109/ISCAS66217.2026.11562030 - 979-8-3315-7769-8
publications_en