Seite - 143 - in Proceedings of the OAGM&ARW Joint Workshop - Vision, Automation and Robotics

Fig. 2. Sample images and segmentation masks from the real training data (top) compared to synthetic data created by the GAN trained on the full training set (bottom) mini-batch size was set to 16. The network was trained until the average of the validation error over the last 10 epochs started to increase. The input data was scaled to be in the range of [−1,1]. Since the generated GAN images and segmentation masks are in the value range of [−1,1], the resulting segmentation image needs to be post-processed to arrive at a binary segmentation mask, which can then be used as an input for the U-Net. To achieve this post-processing, a threshold, largest component and hole filling filter were applied to the generated GAN segmentation masks before they were fed into the U-Net. The threshold was set at the pixel value of 150, and the hole-filling algorithm used is based on geodesic morphology as described in Chapter 6 of [13]. We tested the segmentation performance when using only real training data, a mix of real and synthetic data, as well as only synthetic data. For the synthetic data, we generated a batch of 120 images and segmentation masks from the fully trained GAN. For evaluating the segmentation results, we used the Dice coefficient and Hausdorff distance metrics. Training the U-Net took approximately 3 hours per experiment on the same machine as described above. TABLE I QUANTITATIVE RESULTS OF SEGMENTATION USING THE FULL TRAINING SET U-Net training data Evaluation metrics # Real # Synthetic Dice (mean) Dice (stddev) Hausdorff (mean) Hausdorff (stddev) 145 0 0.9608 0.0101 6.1229 5.0183 145 120 0.9537 0.0121 6.3147 4.8708 0 120 0.9172 0.0283 9.3564 6.0651 C. Results For the full dataset, Fig. 2 illustrates generated images and segmentation masks from the fully trained GAN, compared to real images and segmentation masks. The quantitative evaluation results for the full dataset can be seen in Table I. For the reduced dataset, the quantitative evaluation results are shown in Table II. V. DISCUSSION AND CONCLUSION Smalldatasetspose large issues fordeep learningmethods, leading to overfitting and lack of generalization. We propose an adaptation of Generative Adversarial Networks, where the generator network is trained to generate artificial images in addition to their corresponding segmentation masks. While the qualitative results shown look very promising, they also heavily depend on the amount of training the GAN receives. Fig. 2 shows that using a fully trained GAN to create segmentation data in addition to image data still leads to high quality images. The segmentation also matches the generated images very well, suggesting that both the generator and discriminator are forced to learn the structure of the segmentation as well. However, it can also be seen that small noise artefacts appear in the region of the left lung of the image. These artefacts do not appear if the GAN TABLE II QUANTITATIVE RESULTS OF SEGMENTATION USING THE REDUCED TRAINING SET U-Net training data Evaluation metrics # Real # Synthetic Dice (mean) Dice (stddev) Hausdorff (mean) Hausdorff (stddev) 30 0 0.9464 0.0158 7.6384 6.0395 30 120 0.9394 0.0133 7.2885 5.1007 0 120 0.9312 0.0199 7.6091 5.5654 143