LongSemAnnotator: A Longformer Framework for Column Type Annotation

Column type annotation (CTA) is a critical task in semantic table interpretation (STI), where meaningful semantic labels are automatically assigned to table columns. This process enriches tabular data with essential semantic context, making it fundamental for efficient data integration, cataloging, governance, and the development of natural language database interfaces. Many existing methods struggle with large tables and integrating context from multiple tables. To address these challenges, we introduce a novel approach to improve the accuracy and scalability of CTA. Our framework employs sentence transformers to generate semantic embeddings for columns, enabling the identification of similar columns across different tables. We then utilize a Longformer-based model, which can handle long input sequences, to incorporate inter-table context into the annotation process. Evaluated on the SOTAB benchmark, our approach achieves competitive overall performance, reaching 82.86 micro-F1 when trained on the full training split and 79.13 micro-F1 with the small training split. We also investigate the model’s robustness on test sets containing missing values, format inconsistencies, and edge cases. Experimental results show that incorporating inter-table context and using the Longformer model significantly improves CTA accuracy compared to the best models in the benchmark. Detailed error analysis identifies semantic overlap between similar label types as the primary remaining challenge, with retrieval quality directly constraining annotation accuracy. However, challenges remain with columns exhibiting high semantic overlap and those requiring intra-table context for disambiguation.