Test-Time Adaptation of a Multi-Class Object Localization and Size Estimation Framework for Smart Agriculture Applications

Cognitive Computation, vol. 17

Abstract

Abstract Smart agriculture brings massive amounts of real-time images generated via modern information and communication technology. Promptly providing accurate estimates of fruit/vegetable information, such as location, quantity, and size, is worth studying. Therefore, we focus on exploring a deep learning-based backbone model for heatmap regression to capture the yield information. This singular and lightweight architecture effectively addresses the unified challenge of object counting, location detection, and size estimation for fruits/vegetables. However, when dealing with real-world applications, the data distribution shift would happen in response to the collection of new data. Moreover, some unseen fruits/vegetables often appear during the training process. All of these give rise to the open set recognition (OSR) problem. In such an OSR environment, a test-time domain adaptation approach based on deep learning is proposed for multi-class object localization and size estimation. This is the first attempt at unsupervised domain adaptation for heatmap regression tasks. Furthermore, to overcome the drawback of lacking a public dataset, a new benchmark dataset (including synthetic and real image data) has been created and collected to train, test, and evaluate our approach. Extensive experimental evaluations prove that our approach can achieve accurate predictions in the OSR setting within a single epoch of test-time optimization without altering the training process.

Authors 4

1. Zixu Liu

   University of Southampton

   Affiliation as printed

   Southampton Business School, University of Southampton, University Road, Southampton, SO17 1BJ, Hampshire, UK

2. Qinhao Wu corresponding Aachen Leiden Academic Centre for Drug Research

   Leiden University

   Affiliation as printed

   Leiden Academic Centre for Drug Research, Leiden University, Rapenburg 70, Leiden, 2311 EZ, Netherlands

3. Yuan Chai corresponding

   Guangdong University of Technology

   Affiliation as printed

   School of Computer Science and Technology, Guangdong University of Technology, Panyu District, Guangzhou, 510006, Guangdong, China

4. Huan Yu

   University of Southampton

   Affiliation as printed

   Southampton Business School, University of Southampton, University Road, Southampton, SO17 1BJ, Hampshire, UK