Browsing by Author "Wiehman, Stiaan"

Now showing 1 - 2 of 2
  • Thumbnail Image
    Item
    Investigating fully convolutional networks for bio-image segmentation
    (Stellenbosch : Stellenbosch University, 2018-03) Wiehman, Stiaan; Kroon, R. S. (Steve); De Villiers, H. A. C.; Stellenbosch University. Faculty of Science. Dept. of Mathematical Sciences (Computer Science)
    ENGLISH ABSTRACT : Bio-image analysis is a useful tool for life science researchers with a wide variety of potential applications. A specific area of interest is applying semantic segmentation methods to bio-images, which is challenging due to the typically small data sets in this application area. Neural networks have shown great promise in both general image segmentation problems, as well as bio-image segmentation problems. A recently developed class of neural networks, Fully Convolutional Networks (FCNs), have shown state-of-the-art performance on various semantic segmentation tasks. This thesis provides a thorough investigation into FCN architectures and their use in the semantic segmentation of two bio-image data sets. FCNs have been shown to provide improved performance over regular convolutional neural networks (CNNs). This work starts by comparing these two classes of networks by applying a CNN and three FCNs on the Broad Institute’s Caenorhabditis elegans data set. We showed that the three FCNs performed better on the task of semantic segmentation and provide key insights into the difference in their performance. Recent FCNs can be characterized by two main design aspects: the number of pooling steps in the architecture, and the presence or absence of skip connections. In existing literature, these hyperparameters are typically used without a detailed analysis of their effects. We build on this work by investigating these design aspects and determine their contribution towards the overall performance of the network. Using the recently presented U-net architecture and the accompanying nerve cell membrane data set, this investigation revealed that: (1) increasing the depth of the network by adding additional pooling steps could improve performance up to a (hypothesized) domain-specific saturation point (assuming the inclusion of the necessary skip connections), and (2) each skip connection in the architecture appears to make a different contribution towards the behavior of the network, with some skip connections being more important than others. These findings could provide a better understanding on how to construct new FCN architectures for future applications. We complete this investigation by exploring the possibility of performing end-to-end unsupervised learning as a pre-training technique, and test the resulting models on both fully labeled bio-image data and artificially created partially labeled bio-image data. We proposed a novel augmentation to FCN architectures which allows them to undergo end-to-end unsupervised pretraining. We showed that our unsupervised pre-training approach provides a significant reduction in the variance of the performance of the models. We then applied the supervised version and the pre-trained version of the U-net model on various amounts of partially labeled data, and found that the FCNs are capable of reaching competitive performance with as little as 0.2% of the original pixel labels. The results generated in this thesis provide the foundation for further research into a more sophisticated unsupervised pre-training approach. Such an approach might reduce the need for fully annotated bio-image data, consequently reducing the time and financial resources required to perform the annotations.
  • Thumbnail Image
    Item
    Unsupervised pre-training for fully convolutional neural networks
    (Institute of Electrical and Electronics Engineers, 2016) Wiehman, Stiaan; Kroon, Steve; De Villiers, Hendrik
    Unsupervised pre-training of neural networks has been shown to act as a regularization technique, improving performance and reducing model variance. Recently, fully con-volutional networks (FCNs) have shown state-of-the-art results on various semantic segmentation tasks. Unfortunately, there is no efficient approach available for FCNs to benefit from unsupervised pre-training. Given the unique property of FCNs to output segmentation maps, we explore a novel variation of unsupervised pre-training specifically designed for FCNs. We extend an existing FCN, called U-net, to facilitate end-to-end unsupervised pre-training and apply it on the ISBI 2012 EM segmentation challenge data set. We performed a battery of significance tests for both equality of means and equality of variance, and show that our results are consistent with previous work on unsupervised pre-training obtained from much smaller networks. We conclude that end-to-end unsupervised pre-training for FCNs adds robustness to random initialization, thus reducing model variance.