Image segmentation in electron microscopy

Image segmentation of ultrastructures in large EM volumes is a challenging task.

These challenges mainly concern discontinuities in cellular membranes, the large size of EM volumes, and the large number of neuronal processes that are tightly packed together, which restricts applying segmentation techniques with a high computation complexity. Therefore, automated segmentation of EM volumes of the brain tissue requires specifically developed algorithms that address these challenges. In this section, we review several automated segmentation algorithms that use the earlier introduced methods in the context of EM segmentation.

Lee et al. (Lee et al., 2019) and our automatic segmentation of axons (ACSON;

study I) (Abdollahzadeh et al., 2019) used the SRG algorithm to segment myelinated axons in high-resolution EM volumes of white matter. In (Lee et al., 2019), seeds were located manually for each myelinated axon, while in ACSON, we used the regional maxima of the Euclidean distance transform of myelin to define the seed locations.

The Chan and Vese active contours have been widely used for segmenting ultrastructures in EM volumes. In (Perez et al., 2014), the authors trained an organelle-specific pixel classifier and used the binarized probability maps of organelles to initialize the evolution of active contours. Jorstad and Fua (Jorstad and Fua, 2015) used active surfaces to refine mitochondria segmentation in EM volumes, given the initial boundary prediction from a machine learning-based segmentation algorithm. Tasel et al. (Tasel et al., 2016) used a parabolic arc model to extract membrane structures for anisotropic image stacks and then employed the curve energy based on an active contour to obtain the roughly outlined candidate mitochondrial regions. In the DeepACSON pipeline (study II;

(Abdollahzadeh et al., 2021)), we used deep learning to achieve a semantic Figure 4. U-net architecture with 32×32 pixels in the lowest resolution. Each blue box corresponds to a multi-channel feature maps. The number of channels is written on top of the box and the size of feature maps at the lower left edge of the box. White boxes represent copied feature maps. The arrows denote different operations. This figure is taken from Jackson Huang’s Github under the MIT license.

segmentation of cell nuclei and applied active contours for their instance segmentation.

State-of-the-art automated segmentation techniques currently use deep CNNs to trace ultrastructures in EM volumes (Zeng, Wu and Ji, 2017; Haberl et al., 2018;

Januszewski et al., 2018; Falk et al., 2019; Funke et al., 2019; Meirovitch et al., 2019;

Miocchi et al., 2021). CNNs are typically used for semantic segmentation, whereas other more traditional image processing techniques are used for instance

segmentation. Zeng et al. in DeepEM3D (Zeng, Wu and Ji, 2017) and Haberl et al. in the cloud version of DeepEM3D (Haberl et al., 2018) used a modified version of Inception-V4 (Szegedy et al., 2015) by reducing the number of inception-residual blocks to account for the overfitting while the semantic segmentation. DeepEM3D applied 3D watershed transform for the instance segmentation of neuronal processes. Funke et al. (Funke et al., 2019) trained 3D U-NET to predict affinities using an extension of the MALIS loss function (Turaga et al., 2009), minimizing topological errors of hypothetical thresholding and connected component analysis. The predicted affinities obtained an over-segmentation that is then merged into the final segmentation using a percentile-based agglomeration algorithm. In DeepACSON (study II), we applied a light-weighted FCN for the semantic segmentation of ultrastructures and developed a cylindrical shape decomposition (CSD; study III) (Abdollahzadeh, Sierra and Tohka, 2021) algorithm for the instance segmentation of myelinated axons, incorporating the tubularity of myelinated axons as a global objective in the segmentation process. Januszewski et al. (Januszewski et al., 2018) suggested flood-filling networks (FFNs), a single-object tracking technique, and Meirovitch et al. (Meirovitch et al., 2019) introduced cross-classification clustering, a multi-object tracking technique, merging the semantic- and instance segmentation in recurrent neural networks. The recurrent networks in these techniques maintain the prediction of the shape of objects and learn to reconstruct neuronal processes with more plausible shapes. Roels et al. (Roels et al., 2019) used a reconstruction decoder for domain adaptation to use the existing annotated EM datasets and reduce the dependency on a large training set for EM segmentation. Falk et al. (Falk et al., 2019) presented an ImageJ plugin of U-Net and proposed inserting an artificial one-pixel wide background ridge between touching instances in the training set to address under-segmentation errors. This method applies a connected component analysis for the instance segmentation of neuronal processes in EM volumes.

3 AIM OF THE STUDY

In this thesis, we developed automated segmentation pipelines to trace

ultrastructures in large SBEM volumes of white matter in sham-operated rats and rats after a traumatic brain injury (TBI) to make it possible to analyze the changes in the morphology and spatial distribution of ultrastructures in 3D.

The specific aims of this study are as follows:

1. Segmentation and morphology analysis of ultrastructures in high-resolution SBEM volumes of white matter acquired in small fields of view. In study I, we developed the ACSON pipeline to annotate white matter into myelin, myelinated axons, unmyelinated axons, mitochondria, and cell

bodies/processes. ACSON is an SRG-based algorithm whose seeds were determined automatically. Using ACSON, we measured the true

morphology of myelinated axons in 3D and quantified pathomorphological changes of ultrastructures in TBI.

2. Segmentation and morphology analysis of ultrastructures in low-resolution SBEM volumes of white matter acquired in large fields of view. In studies II and III, we developed an automated segmentation and quantification pipeline called DeepACSON, which performed FCN-based semantic segmentation and shape decomposition-based instance segmentation. In order to train DeepACSON, we used the ACSON segmentations of high-resolution EM datasets from study I. The novel instance segmentation of the DeepACSON pipeline called cylindrical shape decomposition and was published separately in study III. DeepACSON segmented white matter SBEM volumes into myelin, myelinated axons, mitochondria, and cell nuclei.

Using DeepACSON, we measured the 3D morphology and spatial

distribution of ultrastructures and quantified morphological changes after TBI.

4 MATERIALS AND METHODS