Benchmarking Vision-Based Object Tracking for USVs in Complex Maritime Environments
Muhayy Ud Din, Ahsan B. Bakht, Waseem Akram, Lakmal Seneviratne, and Irfan Hussain
Overview
Vision-based target tracking is crucial for unmanned surface vehicles to perform tasks such as
inspection, monitoring, and surveillance. However, real-time tracking in complex maritime environments is
challenging due to dynamic camera movement, low visibility, and scale variation. Typically, object detection
methods combined with filtering techniques are commonly used for tracking, but they often lack robustness,
particularly in the presence of camera motion and missed detections. Although advanced tracking methods
have been proposed recently, their application in maritime scenarios is limited. To address this gap, this
study proposes a vision-guided object tracking framework for USVs, integrating state-of-the-art tracking
algorithms with low-level control systems to enable precise tracking in dynamic maritime environments.
We benchmarked the performance of seven distinct trackers, developed using advanced deep learning
techniques such as Siames Networks and Transformers, by evaluating them on both simulated and real-world
maritime datasets. In addition, we evaluated the robustness of various control algorithms in conjunction
with these tracking systems. The proposed framework was validated through simulations and real-world sea
experiments, demonstrating its effectiveness in handling dynamic maritime conditions. The results show that
SeqTrack, a Transformer-based tracker, performed best in adverse conditions, such as dust storms. Among
the control algorithms evaluated, the LQR controller demonstrated the most robust and smooth control,
allowing for stable tracking of the USV.
Tracking results
Trackers performance
We evaluated the performance of the trackers under varying conditions, including clear sea environments and dust storms. The results are showcased in the videos below.
Trackers performance on real data.
Tracking Framework
We proposed a vision-guided object tracking framework for USVs, integrating state-of-the-art tracking
algorithms with low-level control systems to enable precise tracking in dynamic maritime environments. The framework is composed of three main modules: the Perception
module, which incorporates sensors for environmental perception and state estimation; the Guidance module, which includes a vision-based tracker and computes
guidance commands based on pixel and distance errors; and the Control module, which integrates surge and yaw control to generate tracking commands that drive
the USV’s thrusters, enabling precise target tracking.
Controller Tunning and Performance
The animation below illustrates how different controllers work to minimize yaw error over time.
The controllers performance in thrust generation for tracking.
FAQs
We selected these six tracker named as, SiamFC, ATOM, DiMP, ToMP, SeqTrack, and TaMOs because they represent a range of state-of-the-art methods, from convolutional neural networks and correlation filters to advanced transformer-based models. Each tracker was chosen for its unique strengths in handling challenges like occlusion, motion blur, and dynamic background changes, which are essential for a significant performance in complex maritime environments.
Each controller has its strengths depending on the complexity of the environment and system dynamics. We chose these controllers to capitalize on the diversity in their approaches: PID for its simplicity and ease of tuning, SMC for its robustness in handling disturbances and uncertainties, and LQR for its optimal control and smooth performance in dynamic and energy-efficient operations.
The benefit of this research lies in its development of a robust vision-based tracking framework tailored for Unmanned Surface Vehicles (USVs) in complex maritime environments. By integrating state-of-the-art tracking algorithms, such as Siamese Networks and Transformers, with low-level control systems, the research addresses challenges like dynamic camera movement, low visibility, and missed detections that are common in real-world sea conditions.