Multi-Modal Retinal Image Registration Via Deep Neural Networks PDF Download
Are you looking for read ebook online? Search for your book and save it on your Kindle device, PC, phones or tablets. Download Multi-Modal Retinal Image Registration Via Deep Neural Networks PDF full book. Access full book title Multi-Modal Retinal Image Registration Via Deep Neural Networks by Junkang Zhang. Download full books in PDF and EPUB format.
Author: Junkang Zhang
Publisher:
ISBN:
Category :
Languages : en
Pages : 0
Get Book Here
Book Description
Multi-modal retinal images provide complementary anatomical information at various resolutions, color wavelengths, and fields of view. Aligning multi-modal images will establish a comprehensive view of the retina and benefit the screening and diagnosis of eye diseases. However, the inconsistent anatomical patterns across modalities create outliers in feature matching, and the lack of retinal boundaries may also fool the intensity-based alignment metrics, both of which will influence the alignment qualities. Besides, the varying distortion levels across Ultra-Widefield (UWF) and Narrow-Angle (NA) images, due to different camera parameters, will cause large alignment errors in global transformation. In addressing the issue of inconsistent patterns, we use retinal vasculature as a common signal for alignment. First, we build a two-step coarse-to-fine registration pipeline fully based on deep neural networks. The coarse alignment step estimates a global transformation via vessel segmentation, feature detection and description, and outlier rejection. While the fine alignment step corrects the remaining misalignment through deformable registration. In addition, we propose an unsupervised learning scheme based on style transfer to jointly train the networks for vessel segmentation and deformable registration. Finally, we also introduce Monogenical Phase signal as an alternative guidance in training the deformable registration network. Then, to deal with the issue of various distortion levels across UWF and NA modalities, we propose a distortion correction function to create images with similar distortion levels. Based on the assumptions of spherical eyeball shape and fixed UWF camera pose, the function reprojects the UWF pixels by an estimated correction camera with similar parameters as the NA camera. Besides, we incorporate the function into the coarse alignment networks which will simultaneously optimize the correction camera pose and refine the global alignment results. Moreover, to further reduce misalignment from the UWF-to-NA global registration, we estimate a 3D dense scene for the UWF pixels to represent a more flexible eyeball shape. Both the scene and the NA camera parameters are iteratively optimized to reduce the alignment error between the 3D-to-2D reprojected images and the original ones, which is also concatenated with the coarse alignment networks with distortion correction function.
Author: Junkang Zhang
Publisher:
ISBN:
Category :
Languages : en
Pages : 0
Get Book Here
Book Description
Multi-modal retinal images provide complementary anatomical information at various resolutions, color wavelengths, and fields of view. Aligning multi-modal images will establish a comprehensive view of the retina and benefit the screening and diagnosis of eye diseases. However, the inconsistent anatomical patterns across modalities create outliers in feature matching, and the lack of retinal boundaries may also fool the intensity-based alignment metrics, both of which will influence the alignment qualities. Besides, the varying distortion levels across Ultra-Widefield (UWF) and Narrow-Angle (NA) images, due to different camera parameters, will cause large alignment errors in global transformation. In addressing the issue of inconsistent patterns, we use retinal vasculature as a common signal for alignment. First, we build a two-step coarse-to-fine registration pipeline fully based on deep neural networks. The coarse alignment step estimates a global transformation via vessel segmentation, feature detection and description, and outlier rejection. While the fine alignment step corrects the remaining misalignment through deformable registration. In addition, we propose an unsupervised learning scheme based on style transfer to jointly train the networks for vessel segmentation and deformable registration. Finally, we also introduce Monogenical Phase signal as an alternative guidance in training the deformable registration network. Then, to deal with the issue of various distortion levels across UWF and NA modalities, we propose a distortion correction function to create images with similar distortion levels. Based on the assumptions of spherical eyeball shape and fixed UWF camera pose, the function reprojects the UWF pixels by an estimated correction camera with similar parameters as the NA camera. Besides, we incorporate the function into the coarse alignment networks which will simultaneously optimize the correction camera pose and refine the global alignment results. Moreover, to further reduce misalignment from the UWF-to-NA global registration, we estimate a 3D dense scene for the UWF pixels to represent a more flexible eyeball shape. Both the scene and the NA camera parameters are iteratively optimized to reduce the alignment error between the 3D-to-2D reprojected images and the original ones, which is also concatenated with the coarse alignment networks with distortion correction function.
Author: Yiqian Wang
Publisher:
ISBN:
Category :
Languages : en
Pages : 0
Get Book Here
Book Description
The retina, the light sensitive tissue lining the interior of the eye, is the only part of the central nervous system (CNS) that can be imaged at micron resolution in vivo. Retinal diseases including age-related macular degeneration, diabetes retinopathy, and vascular occlusions are important causes of vision loss and have systemic implications for millions of patients. Retinal imaging is of great significance to diagnosing and monitoring both retinal diseases and systematic diseases that manifest in the retina. A variety of imaging devices have been developed, including color fundus (CF) photography, infrared reflectance (IR), fundus autofluorescence (FAF), dye-based angiography, optical coherence tomography (OCT), and OCT angiography (OCT-A). Each imaging modality is particularly useful for observing certain aspects of the retina, and can be utilized for visualization of specific diseases. In this dissertation, we propose deep learning based methods for retinal image processing, including multimodal retinal image registration, OCT motion correction, and OCT retinal layer segmentation. We present our established work on a deep learning framework for multimodal retinal image registration, a comprehensive study of the correlation between subjective and objective evaluation metrics for multimodal retinal image registration, convolutional neural networks for correction of axial and coronal motion artifacts in 3D OCT volumes, and joint motion correction and 3D OCT layer segmentation network. The dissertation not only proposes novel approaches in image processing, enhances the observation of retinal diseases, but will also provide insights on observing systematic diseases through the retina, including diabetes, cardiovascular disease, and preclinical Alzheimer's Disease. The proposed deep learning based retinal image processing approaches would build a connection between ophthalmology and image processing literature, and the findings may provide a good insight for researchers who investigate retinal image registration, retinal image segmentation and retinal disease detection.
Author: Li Ding (Electrical and computer engineering researcher)
Publisher:
ISBN:
Category :
Languages : en
Pages : 0
Get Book Here
Book Description
"Geometry and semantics are two sub-fields in computer vision that have been researched extensively as two separate problems over decades. However, semantics and geometry in computer vision are not mutually exclusive and techniques developed for one could complement the other. Unfortunately, the interplay of these two fields has received limited attention. In this thesis, we design symbiotic geometric and semantic computer vision methods in the specific context of retinal image analysis, where we consider the semantic problem of retinal vessel detection and the geometric problem of retinal image registration. First, we propose a novel pipeline for vessel detection in fluorescein angiography (FA) using deep neural networks (DNNs) that reduces the effort required for labeling ground truth data by combining cross-modality transfer and human-in-the-loop learning. The cross-modality transfer exploits concurrently captured color fundus (CF) and fundus FA. Binary vessels maps detected from CF images with a pre-trained network are geometrically registered with FA images via robust parametric chamfer alignment. Using the transferred vessels as initial ground truth labels, the human-in-the-loop approach progressively improves the ground truth labeling by iterating between deep-learning and labeling. Experiments show that the proposed pipeline significantly reduces the annotation effort and outperforms prior FA vessel detection methods by a significant margin. Next, we describe an annotation-efficient deep learning framework for vessel detection in UWF fundus photography (FP) that does not require de novo labeled UWF FP vessel maps. Our approach uses concurrently captured UWF FA and iterates between a multi-modal registration step and a weakly-supervised learning step. In the registration step, UWF FA vessel maps detected with a pre-trained DNN are registered with the UWF FP via parametric chamfer alignment. The warped vessel maps are used as the tentative training data but inevitably contain incorrect labels due to the differences between the two modalities and the errors in the registration. In the learning step, a robust learning method is proposed to train DNNs with noisy labels. The registration and the vessel detection benefit from each other and are progressively improved. Results on two datasets show that the proposed approach provides accurate vessel detection, without requiring manually labeled UWF FP training data. Finally, we present a hybrid framework for registering retinal images in the presence of extreme geometric distortions that are commonly encountered in UWF FA. Our approach consists of a feature-based global registration and a vessel-based local refinement. For the global registration, we introduce a modified RANSAC algorithm that jointly identifies corresponding keypoints and estimates a polynomial geometric transformation consistent with the identified correspondences between reference and target images. Our RANSAC modification particularly improves feature matching and the registration in peripheral regions that are most severely impacted by the geometric distortions. The local refinement is formulated as a parametric chamfer alignment for vessel maps obtained using DNNs. Because the complete vessel maps contribute to the chamfer alignment, this approach not only improves registration accuracy but also aligns with clinical practice, where vessels are typically a key focus of examinations. Experiments conducted on two datasets show that the proposed framework significantly outperforms the existing retinal image registration methods"--Pages xv-xvii.
Author: Joseph V. Hajnal
Publisher: CRC Press
ISBN: 1420042475
Category : Medical
Languages : en
Pages : 394
Get Book Here
Book Description
Image registration is the process of systematically placing separate images in a common frame of reference so that the information they contain can be optimally integrated or compared. This is becoming the central tool for image analysis, understanding, and visualization in both medical and scientific applications. Medical Image Registration provid
Author: Courtney K. Guo
Publisher:
ISBN:
Category :
Languages : en
Pages : 40
Get Book Here
Book Description
In this thesis, we tackle learning-based multi-modal image registration. Multi-modal registration, in which two images of dierent modalities need to be aligned to each other, is a difficult yet essential task for medical imaging analysis. Classical methods have been developed for single-modal and multi-modal registration, but are slow because they solve an optimization problem for each pair of images. Recently, deep learning methods for registration have been proposed, and have been shown to shorten registration time by learning a global function to perform registration, which can then be applied quickly on a pair of test images. These methods perform well for single-modal registration but have not yet been extended to the harder task of multi-modal registration. We bridge this gap by implementing classical multi-modal metrics in a differentiable and efficient manner to enable deep image registration for multi-modal data. We nd that our method for multi-modal registration performs significantly better than baselines, in terms of both accuracy and runtime.
Author: Wei Gong
Publisher: Frontiers Media SA
ISBN: 283252107X
Category : Science
Languages : en
Pages : 104
Get Book Here
Book Description
Author: Jacqueline Le Moigne
Publisher: Cambridge University Press
ISBN: 1139494376
Category : Technology & Engineering
Languages : en
Pages : 515
Get Book Here
Book Description
This book provides a summary of current research in the application of image registration to satellite imagery. Presenting algorithms for creating mosaics and tracking changes on the planet's surface over time, it is an indispensable resource for researchers and advanced students in Earth and space science, and image processing.
Author: Linwei Wang
Publisher: Springer Nature
ISBN: 3031164466
Category : Computers
Languages : en
Pages : 842
Get Book Here
Book Description
The eight-volume set LNCS 13431, 13432, 13433, 13434, 13435, 13436, 13437, and 13438 constitutes the refereed proceedings of the 25th International Conference on Medical Image Computing and Computer-Assisted Intervention, MICCAI 2022, which was held in Singapore in September 2022. The 574 revised full papers presented were carefully reviewed and selected from 1831 submissions in a double-blind review process. The papers are organized in the following topical sections: Part I: Brain development and atlases; DWI and tractography; functional brain networks; neuroimaging; heart and lung imaging; dermatology; Part II: Computational (integrative) pathology; computational anatomy and physiology; ophthalmology; fetal imaging; Part III: Breast imaging; colonoscopy; computer aided diagnosis; Part IV: Microscopic image analysis; positron emission tomography; ultrasound imaging; video data analysis; image segmentation I; Part V: Image segmentation II; integration of imaging with non-imaging biomarkers; Part VI: Image registration; image reconstruction; Part VII: Image-Guided interventions and surgery; outcome and disease prediction; surgical data science; surgical planning and simulation; machine learning – domain adaptation and generalization; Part VIII: Machine learning – weakly-supervised learning; machine learning – model interpretation; machine learning – uncertainty; machine learning theory and methodologies.
Author: Elizabeth McKenzie
Publisher:
ISBN:
Category :
Languages : en
Pages : 117
Get Book Here
Book Description
Deformable image registration (DIR) is an important component of a patient's radiation therapy treatment. During the planning stage it combines complementary information from different imaging modalities and time points. During treatment, it aligns the patient to a reproducible position for accurate dose delivery. As the treatment progresses, it can inform clinicians of important changes in anatomy which trigger plan adjustment. And finally, after the treatment is complete, registering images at subsequent time points can help to monitor the patient's health. The body's natural non-rigid motion makes DIR a complex challenge. Recently neural networks have shown impressive improvements in image processing and have been leveraged for DIR tasks. This thesis is a compilation of neural network-based approaches addressing lingering issues in medical DIR, namely 1) multi-modality registration, 2) registration with different scan extents, and 3) modeling large motion in registration. For the first task we employed a cycle consistent generative adversarial network to translate images in the MRI domain to the CT domain, such that the moving and target images were in a common domain. DIR could then proceed as a synthetically bridged mono-modality registration. The second task used advances in network-based inpainting to artificially extend images beyond their scan extent. The third task leveraged axial self-attention networks' ability to learn long range interactions to predict the deformation in the presence of large motion. For all these studies we used images from the head and neck, which exhibit all of these challenges, although these results can be generalized to other parts of the anatomy.The results of our experiments yielded networks that showed significant improvements in multi-modal DIR relative to traditional methods. We also produced network which can successfully predict missing tissue and demonstrated a DIR workflow that is independent of scan length. Finally, we trained a network whose accuracy is a balance between large and small motion prediction, and which opens the door to non-convolution-based DIR. By leveraging the power of artificial intelligence, we demonstrate a new paradigm in deformable image registration. Neural networks learn new patterns and connections in imaging data which go beyond the hand-crafted features of traditional image processing. This thesis shows how each step of registration, from the image pre-processing to the registration itself, can benefit from this exciting and cutting-edge approach.
Author: Wenxing Fu
Publisher: Springer Nature
ISBN: 981990479X
Category : Technology & Engineering
Languages : en
Pages : 3985
Get Book Here
Book Description
This book includes original, peer-reviewed research papers from the ICAUS 2022, which offers a unique and interesting platform for scientists, engineers and practitioners throughout the world to present and share their most recent research and innovative ideas. The aim of the ICAUS 2022 is to stimulate researchers active in the areas pertinent to intelligent unmanned systems. The topics covered include but are not limited to Unmanned Aerial/Ground/Surface/Underwater Systems, Robotic, Autonomous Control/Navigation and Positioning/ Architecture, Energy and Task Planning and Effectiveness Evaluation Technologies, Artificial Intelligence Algorithm/Bionic Technology and Its Application in Unmanned Systems. The papers showcased here share the latest findings on Unmanned Systems, Robotics, Automation, Intelligent Systems, Control Systems, Integrated Networks, Modeling and Simulation. It makes the book a valuable asset for researchers, engineers, and university students alike.
