LabelBOT: A Human-in-the-Loop Classification Assistance in the BIIGLE Image Annotation Tool

In marine science and engineering, growing volumes of imaging data are used in exploration and monitoring. The process of manual annotation of this data (i.e. marking and describing objects of interest with labels), to extract quantitative data or to collect many examples for AI applications, is a very time-consuming task and requires a high level of expert experience to achieve annotations of sufficient quality and consistency. To address this challenge, we introduce LabelBOT, a human-in-the-loop classification annotation assistant that supports and accelerates the annotation process by providing relevant, real-time label suggestions. Rather than replacing human annotators, LabelBOT functions as a collaborative tool to reduce repetitive work, benefiting both experts and non-experts. LabelBOT operates on 384-dimensional feature vectors extracted using the DINOv2 self-supervised model and performs classification through cosine similarity-based Approximate-Nearest Neighbors (ANN) using the Hierarchical Navigable Small World (HNSW) index for fast sub-second vector search. The system has been evaluated on two expert-annotated datasets: a marine fish imagery dataset (OBSEA) and an industrial dataset from visual marine inspections (Marine Infrastructure). It achieves 61% Top-1 and 90% Top-3 accuracy on OBSEA, and 71% Top-1 and 92% Top-3 accuracy on the Marine Infrastructure dataset. LabelBOT is fully integrated into the BIIGLE image annotation platform utilizing ONNX-runtime, for fast and efficient browser-based feature vector generation. Its responsive user interface offers immediate classification suggestions accelerating the workflow without disrupting the user’s control.