Standardized Evaluation of Current Ultrasound Bone Segmentation Algorithms on Multiple Datasets

EPiC series in health sciences, vol. 5, pp. 148–141

Abstract

Ultrasound (US) bone segmentation is an important component of US-guided or- thopaedic procedures. While there are many published segmentation techniques, there is no direct way to compare their performance. We present a solution to this, by curating a multi-institutional set of US images and corresponding segmentations, and systematically evaluating six previously-published bone segmentation algorithms using consistent metric definitions. We find that learning-based segmentation methods outperform traditional al- gorithms that rely on hand-crafted image features, as measured by their Dice scores, RMS distance errors and segmentation success rates. However, there is no single best performing algorithm across the datasets, emphasizing the need for carefully evaluating techniques on large, heterogenous datasets. The datasets and evaluation framework described can be used to accelerate development of new segmentation algorithms.

Authors 15

1. Benjamin Hohlmann Aachen

   RWTH Aachen University

   Affiliation as printed

   RWTH Aachen University , Aachen , Germany

   RWTH Aachen University, Aachen, Germany

2. Peter Brößner Aachen

   RWTH Aachen University

   Affiliation as printed

   RWTH Aachen University, Aachen, Germany

3. Ilker Hacihaliloglu

   University of British Columbia

   Affiliation as printed

   University of British Columbia , Vancouver , BC , Canada

   University of British Columbia, Vancouver, BC, Canada

4. Keiran Barr

   Queen's University

   Affiliation as printed

   Queen's University , Kingston , ON , Canada

   Queen's University, Kingston, ON, Canada

5. Tamás Ungi

   Queen's University

   Affiliation as printed

   Queen's University , Kingston , ON , Canada

   Queen's University, Kingston, ON, Canada

6. Oliver Zettinig
   Affiliation as printed

   ImFusion GmbH , Munich , Germany

   ImFusion GmbH, Munich, Germany

7. Raphaël Prevost
   Affiliation as printed

   ImFusion GmbH , Munich , Germany

   ImFusion GmbH, Munich, Germany

8. Guillaume Dardenne

   Centre Hospitalier Régional Universitaire de Brest

   Affiliation as printed

   University Hospital of Brest , Brest , France

   University Hospital of Brest, Brest, France

9. Zian Fanti

   Universidad Nacional Autónoma de México

   Affiliation as printed

   Universidad Nacional Autónoma de México , CDMX , Mexico

   Universidad Nacional Autónoma de México, CDMX, Mexico

10. Wolfgang Wein
    Affiliation as printed

    ImFusion GmbH , Munich , Germany

    ImFusion GmbH, Munich, Germany

11. Éric Stindel

    Centre Hospitalier Régional Universitaire de Brest

    Affiliation as printed

    University Hospital of Brest , Brest , France

    University Hospital of Brest, Brest, France

12. Fernando Arámbula Cosı́o

    Universidad Nacional Autónoma de México

    Affiliation as printed

    Universidad Nacional Autónoma de México , CDMX , Mexico

    Universidad Nacional Autónoma de México, CDMX, Mexico

13. Gábor Fichtinger

    Queen's University

    Affiliation as printed

    Queen's University , Kingston , ON , Canada

    Queen's University, Kingston, ON, Canada

14. Klaus Radermacher Aachen

    RWTH Aachen University

    Affiliation as printed

    RWTH Aachen University , Aachen , Germany

    RWTH Aachen University, Aachen, Germany

15. Antony J. Hodgson

    University of British Columbia

    Affiliation as printed

    University of British Columbia , Vancouver , BC , Canada

    University of British Columbia, Vancouver, BC, Canada