Image Synthesis and Modified BlendMask Instance Segmentation for Automated Nanoparticle Phenotyping

IEEE Transactions on Medical Imaging, vol. 42, pp. 3665–3677

Abstract

Automated nanoparticle phenotyping is a critical aspect of high-throughput drug research, which requires analyzing nanoparticle size, shape, and surface topography from microscopy images. To automate this process, we present an instance segmentation pipeline that partitions individual nanoparticles on microscopy images. Our pipeline makes two key contributions. Firstly, we synthesize diverse and approximately realistic nanoparticle images to improve robust learning. Secondly, we improve the BlendMask model to segment tiny, overlapping, or sparse particle images. Specifically, we propose a parameterized approach for generating novel pairs of single particles and their masks, encouraging greater diversity in the training data. To synthesize more realistic particle images, we explore three particle placement rules and an image selection criterion. The improved one-stage instance segmentation network extracts distinctive features of nanoparticles and their context at both local and global levels, which addresses the data challenges associated with tiny, overlapping, or sparse nanoparticles. Extensive experiments demonstrate the effectiveness of our pipeline for automating nanoparticle partitioning and phenotyping in drug research using microscopy images.

Authors 7

1. Xiaoqin Tang

   Southwest University

   Affiliation as printed

   School of Computer and Information Science, Southwest University, Chongqing, China

2. Lingpeng Lv

   Southwest University

   Affiliation as printed

   School of Computer and Information Science, Southwest University, Chongqing, China

3. Shima Javanmardi Aachen Leiden Institution of Advance Computer Science

   Leiden University

   Affiliation as printed

   Leiden Institution of Advance Computer Science, Leiden University, Leiden, The Netherlands

4. Yunfeng Wang

   Southwest University

   Affiliation as printed

   School of Computer and Information Science, Southwest University, Chongqing, China

5. Jingchuan Fan

   Chongqing Medical University

   Affiliation as printed

   Institution of Life Sciences, Chongqing Medical University, Chongqing, China

6. Fons J. Verbeek Aachen Leiden Institution of Advance Computer Science

   Leiden University

   Affiliation as printed

   Leiden Institution of Advance Computer Science, Leiden University, Leiden, The Netherlands

7. Guoqiang Xiao

   Southwest University

   Affiliation as printed

   School of Computer and Information Science, Southwest University, Chongqing, China