Department of Computer Science

Permanent URI for this community

https://scholar.sun.ac.za/handle/10019.1/96336

Browse

Browsing Department of Computer Science by browse.metadata.advisor "De Villiers, H. A. C."

Now showing 1 - 3 of 3
  • Thumbnail Image
    Item
    Creating 3D models using reconstruction techniques
    (Stellenbosch : Stellenbosch University, 2018-12) Martin, Javonne Jason; Kroon, R. S. (Steve); De Villiers, H. A. C.; Stellenbosch University. Faculty of Science. Dept. of Mathematical Sciences. Division Computer Science.
    ENGLISH ABSTRACT :Virtual reality models of real world environments have a number of compelling applications, such as preserving the architecture and designs of older buildings. This process can be achieved by using 3D artists to reconstruct the environment, however this is a long and expensive process. Thus, this thesis investigates various techniques and approaches used in 3D reconstruction of environments using a single RGB-D camera and aims to reconstruct the 3D environment to generate a 3D model. This would allow non-technical users to reconstruct environments and use these models in business and simulations, such as selling real-estate, modifying pre-existing structures for renovation and planning. With the recent improvements in virtual reality technology such as the Oculus Rift and HTC Vive, a user can be immersed into virtual reality environments created from real world structures. A system based on Kinect Fusion is implemented to reconstruct an environment and track the motion of the camera within the environment. The system is designed as a series of selfcontained subsystems that allows for each of the subsystems to be modified, expanded upon or easily replaced by alternative methods. The system is made available as an open source C++ project using Nvidia’s CUDA framework to aid reproducibility and provides a platform for future research. The system makes use of the Kinect sensor to capture information about the environment. A coarse-to-fine least squares approach is used to estimate the motion of the camera. In addition, the system employs a frame-to-model approach that uses a view of the estimated reconstruction of the model as the reference frame and the incoming scene data as the target. This minimises the drift with respect to the true trajectory of the camera. The model is built using a volumetric approach, with volumetric information implicitly stored as a truncated signed distance function. The system filters out noise in the raw sensor data by using a bilateral filter. A point cloud is extracted from the volume using an orthogonal ray caster which enables an improved hole-filling approach. This allows the system to extract both the explicit and implicit structure from the volume. The 3D reconstruction is followed by mesh generation based on the point cloud. This is achieved by using an approach related to Delaunay triangulation, the ball-pivot algorithm. The resulting system processes frames at 30Hz, enabling real-time point cloud generation, while the mesh generation occurs offline. This system is initially tested using Blender to generate synthetic data, followed by a series of real world tests. The synthetic data is used to test the presented system’s motion tracking against the ground truth. While the presented system suffers from the effects of drift over long frame sequences, it is shown to be capable of tracking the motion of the camera. This thesis finds that the ball pivot algorithm can generate the edges and faces for synthetic point clouds, however it performs poorly when using the noisy synthetic and real world data sets. Based on the results obtained it is recommended that the obtained point cloud be preprocessed to remove noise before it is provided to the mesh generation algorithm and an alternative mesh generation technique should be employed that is more robust to noise.
  • Thumbnail Image
    Item
    Investigating fully convolutional networks for bio-image segmentation
    (Stellenbosch : Stellenbosch University, 2018-03) Wiehman, Stiaan; Kroon, R. S. (Steve); De Villiers, H. A. C.; Stellenbosch University. Faculty of Science. Dept. of Mathematical Sciences (Computer Science)
    ENGLISH ABSTRACT : Bio-image analysis is a useful tool for life science researchers with a wide variety of potential applications. A specific area of interest is applying semantic segmentation methods to bio-images, which is challenging due to the typically small data sets in this application area. Neural networks have shown great promise in both general image segmentation problems, as well as bio-image segmentation problems. A recently developed class of neural networks, Fully Convolutional Networks (FCNs), have shown state-of-the-art performance on various semantic segmentation tasks. This thesis provides a thorough investigation into FCN architectures and their use in the semantic segmentation of two bio-image data sets. FCNs have been shown to provide improved performance over regular convolutional neural networks (CNNs). This work starts by comparing these two classes of networks by applying a CNN and three FCNs on the Broad Institute’s Caenorhabditis elegans data set. We showed that the three FCNs performed better on the task of semantic segmentation and provide key insights into the difference in their performance. Recent FCNs can be characterized by two main design aspects: the number of pooling steps in the architecture, and the presence or absence of skip connections. In existing literature, these hyperparameters are typically used without a detailed analysis of their effects. We build on this work by investigating these design aspects and determine their contribution towards the overall performance of the network. Using the recently presented U-net architecture and the accompanying nerve cell membrane data set, this investigation revealed that: (1) increasing the depth of the network by adding additional pooling steps could improve performance up to a (hypothesized) domain-specific saturation point (assuming the inclusion of the necessary skip connections), and (2) each skip connection in the architecture appears to make a different contribution towards the behavior of the network, with some skip connections being more important than others. These findings could provide a better understanding on how to construct new FCN architectures for future applications. We complete this investigation by exploring the possibility of performing end-to-end unsupervised learning as a pre-training technique, and test the resulting models on both fully labeled bio-image data and artificially created partially labeled bio-image data. We proposed a novel augmentation to FCN architectures which allows them to undergo end-to-end unsupervised pretraining. We showed that our unsupervised pre-training approach provides a significant reduction in the variance of the performance of the models. We then applied the supervised version and the pre-trained version of the U-net model on various amounts of partially labeled data, and found that the FCNs are capable of reaching competitive performance with as little as 0.2% of the original pixel labels. The results generated in this thesis provide the foundation for further research into a more sophisticated unsupervised pre-training approach. Such an approach might reduce the need for fully annotated bio-image data, consequently reducing the time and financial resources required to perform the annotations.
  • Thumbnail Image
    Item
    Texture synthesis with neural networks
    (Stellenbosch : Stellenbosch University, 2018-12) Schreiber, Shaun; Geldenhuys, J.; De Villiers, H. A. C.; Stellenbosch University. Faculty of Science. Dept. of Mathematical Sciences. Division Computer Science.
    ENGLISH ABSTRACT : Creating detailed texture maps for virtual environments is often a timeconsuming process. Procedural texture generation enables the creation of more rich and detailed virtual environments with minimal input needed from an artist. However, finding a flexible generative model of real world textures remains an open problem. There are currently two key limiting factors. The first key limitation is a lack of available knowledge on the capability of the various neural network based techniques and how the components associated with each technique affects the quality of synthesized textures. The second key limitation in modern generative models is the inability to apply localized constraints in situations where there are complex interactions between two regions within a texture. To address these limitations, three areas of interest (training set, network architecture, and texture representation) involving the synthesis process are identified specifically for neural network-based techniques and their effects on the synthesized textures are investigated. Included in this investigation is a comparative study focusing on subjective quality and quantitative error measurement between the currently available techniques. Second, a novel convolutional neural network-based texture model is proposed, consisting of four summary statistics (content or feature maps, Gramian matrices, transformed Gramian matrices, and total variation), as well as spectrum constraints. The Fourier transform and windowed Fourier transform are investigated in applying spectrum constraints, and it is found that the windowed Fourier transform improved the quality and consistency of the generated textures. During the component investigation, it was identified that the VGG-19 network still produces comparable results when compared to more modern network architectures. Additionally, it was also demonstrated that direct methods are capable of producing results equal to the iterative approach if stochastic textures are synthesized, but produces unsatisfactory results with irregular and regular textures. Finally, the efficacy of the proposed technique is demonstrated by comparing the generated output with that of related techniques.