Deep learning based medical imaging analysis has demonstrated remarkable capabilities in enhancing clinical diagnostics, facilitating better decision-making through improved visualization and analysis. Despite these achievements, the complexity of various imaging modalities presents significant challenges, highlighting the need for robust application across different systems. In this thesis, I address these challenges by integrating advanced deep learning models for specific imaging modalities including MRI, diffusion MRI and CT scans. First, I introduced the Anatomy Context-Encoding Network (ACEnet), a novel 2.5D deep learning model that leverages 3D spatial and anatomical contexts in a 2D framework. This model was designed to enhance the segmentation of brain structures from MRI scans. ACEnet’s performance on benchmark datasets demonstrated superior accuracy and speed, significantly outperforming existing state-of-the-art methods and illustrating its effectiveness for real-time clinical applications. Second, I developed DeepADC-Net, a CNN-Transformer based model for producing accurate Apparent Diffusion Coefficient (ADC) maps from accelerated diffusion-weighted MRI data. By efficiently reconstructing accelerated ADC maps to match fully-sampled ADC maps, DeepADC-Net significantly reduced scan time from 25 minutes to just over six minutes while maintaining high-quality output. This reduction is crucial for improving patient comfort and enhancing the diagnostic process for tumor detection. Lastly, I proposed CycleFormer, a novel cross-modality domain adaptation method that enhances urinary tract segmentation. Utilizing CT urograms, CycleFormer effectively improves the segmentation performance on non-contrast CT scans, specifically for ureters, and has shown notable improvement over alternative state-of-the-art 2D CNN models, providing more accurate and reliable diagnostic tools for clinical practice. In conclusion, the deep learning models developed in this thesis significantly advance the field of medical image analysis, providing robust, efficient and more accurate diagnostic tools that enhance the capability of medical imaging technologies in clinical diagnosis. These contributions demonstrate the potential of deep learning approaches across various medical imaging modalities, highlighting their effectiveness in clinical diagnosis and healthcare settings.