Cyclic conditional diffusion models for CT-to-MR synthetic image segmentation with misaligned image pairs

Abstract

Abdominal CT is a commonly used imaging modality for diagnosing various diseases and conditions. Despite its potential clinical utility, depicting low-contrast structures with sufficient contrast by using CT alone remains challenging. To overcome this limitation, supplementary magnetic resonance imaging (MRI) examinations are often performed to visualize specific structures with low contrast against surrounding tissues. However, supplementary imaging tests require additional scan time and cost. In this paper, we train a synthetic image segmentation model with misaligned clinical CT-MR image pairs to predict soft structures in the source CT images. Our synthetic image segmentation model consists of CT-to-MR image translation and subsequent MR-like image segmentation. For CT-to-MR image translation, a conditional diffusion model is trained to recover noisy MR images by using CT images as the conditional input. Conditioned on MR images, a separate conditional diffusion model is trained to recover noisy CT images. To couple the two conditional diffusion models during the training process, each model retakes the output of the other model as the conditional input. We also propose an adaptive learning strategy for adversarial image-to-image translation with misaligned CT-MR image pairs. In end-to-end learning of synthetic image segmentation, cyclically translated MR images, as well as real MR images, are used to train the subsequent segmentation network. Through a series of clinical experiments with aligned, misaligned, and unpaired CT-MR datasets, we show that the proposed method outperforms other competing methods in both CT-to-MR image translation and synthetic image segmentation. In particular, the faithful MR-like images from the proposed framework are capable of bridging the gap between the original CT images and the predicted segmentation masks.