Vision–Language Target Retrieval for Drone-Based Maritime Search and Rescue

Coastal search and rescue (SAR) operations require rapid identification of victims in highly dynamic maritime environments where spatial uncertainty and time sensitivity are critical. Although drone-based surveillance systems have been widely studied for maritime monitoring, conventional object detection approaches primarily focus on detecting the presence of people rather than identifying specific individuals among multiple candidates. This study proposes a vision-language target retrieval framework for drone-based maritime SAR. The proposed system integrates three sequential modules: (1) person detection using a YOLOv12-based model, (2) visual attribute extraction using a vision-language model, and (3) query-based target retrieval combining attribute-based candidate filtering with a large language model (LLM)-based semantic re-ranking. Extracted attributes—including gender, top color, bottom color, shoes, action, and belonging—are stored in a relational database management system (RDBMS) and matched against natural-language queries provided by rescue operators. The framework was evaluated using a dataset of 620 human objects collected from multiple maritime and aerial datasets with 60 natural-language queries simulating real-world SAR scenarios. Experimental results show that the proposed two-stage retrieval framework significantly improves retrieval performance compared with an LLM-only retrieval setting. The system achieved an E2EHit@5 of 0.8333 with an average query-stage latency of 2.32 seconds, whereas the LLM-only setting achieved 0.1333 with a query-stage latency of 25.26 seconds. These results demonstrate that attribute-based candidate filtering effectively reduces the search space while LLM-based semantic re-ranking improves target identification accuracy, providing an efficient decision-support framework for rapid victim identification in maritime SAR operations.