DePuy Synthes automates orthopedic surgical tray inspection, cutting inspection time 47.3% with computer vision

DePuy Synthes manually inspected all orthopedic joint reconstruction loaner trays before shipment and upon return to verify presence and correct placement of components. Each tray contains hundreds of distinct but highly similar components in designated slots, making the process labor-intensive, slow, and prone to human error.

An end-to-end automated pipeline was developed combining a custom YOLOv7-X model trained on 1,039 unique component classes from 74 trays, with a LoFTR-based layout verification algorithm. Custom non-maximum suppression (NMS) was applied to handle overlapping detections, and USAC-MAGSAC image registration aligned test images to reference trays to confirm correct component placement.

The pipeline achieved a mean average precision (mAP@0.5) of 0.94 ± 0.10 and a false-positive rate of 0.05 ± 0.08 across 12 test scenarios on 139 inspection images. In 7 of 12 scenarios, mAP exceeded 0.99. The enhanced YOLOv7-X architecture outperformed the Faster-RCNN-ResNet101 baseline by 24%, and the solution reduced overall inspection time by 47.3%.

• Training a single YOLO model on 1,000+ fine-grained classes is feasible with customized NMS tailored to expected component type and quantity per tray.
• Combining object detection with transformer-based feature matching (LoFTR) enables robust layout verification even under variable lighting, angle, and camera distance.
• The pipeline is lightweight and scalable, making it straightforward to extend to other surgical tray configurations or industrial assembly verification tasks.