DLBIRHOUI

Deep Learning Based Image Reconstruction for Hybrid Optoacoustic and Ultrasound Imaging

Started
August 1, 2020
Status
Completed
Share this project

Abstract

Over the last decade, the Razansky lab was instrumental in the development of multi-spectral optoacoustic tomography (MSOT), transforming this novel bio-imaging technology from the initial demonstration of technical feasibility, through establishment of image reconstruction methodologies all the way toward its clinical translation. The method rapidly finds its place as a potent clinical imaging tool due to its high sensitivity and molecular specificity as well as non-invasive, real-time and high-resolution volumetric imaging capabilities deep in living biological tissues. Despite great promise demonstrated in the pilot clinical studies, human imaging with MSOT is afflicted by a limited tomographic access to the region of interest while significant constraints are further imposed on the light deposition in deep tissues. This project aims to develop new artificial intelligence capabilities for improving image quality and diagnostic capacity of MSOT images acquired by sub-optimal scanner configurations resulting from, for example, application-related constraints or low cost design considerations. In particular, we will devise machine learning approaches to enable efficient and robust multimodal combination of MSOT with pulse-echo ultrasonography by training neural networks on high-resolution and high-quality training datasets generated by dedicated optimally designed scanner configurations. The trained models will be used to restore quality of artifactual images produced by various sub-optimal scanner configurations with limited tomographic view or sparsely acquired data in typical clinical imaging scenarios. Those advancements will help reducing inter-clinician variability and enable a more efficient, rapid, and objective analysis of large amounts of image data, thus relaxing requirements for specialized training and facilitating the wider adoption of MSOT apparatus in primary care and other non-hospital settings.

People

Collaborators

SDSC Team:
Anna Susmelj
Firat Ozdemir

PI | Partners:

Multi-Scale Functional and Molecular Imaging:

  • Prof. Daniel Razansky
  • Berkan Lafci

More info

description

Motivation

As an emerging non-invasive imaging modality optoacoustic (OA) imaging has a lot of potential in clinical and pre-clinical applications (Figure 1). Unleashing this potential relies on overcoming challenges from limited view tomographic reconstruction to various forms of noise. The project tackles these challenges with several data-driven approaches.

Proposed Approach / Solution

A first of its kind, large scale, diverse, user-friendliness oriented, in vivo and simulated optoacoustic data (Experimental and Synthetic Clinical Optoacoustic Data: OADAT) has been publicly released along with a multitude of demos to use OADAT for data science applications (Figures 2 and 3). Additional proposed solutions include neural network-based methodologies for producing high fidelity limited view and sparse view OA image reconstruction as well as exploiting raw OA signals to improve reconstruction of limited view OA images.

Impact

OA imaging requires significant know-how and hardware to acquire data, causing the research to be limited to a few groups. The released dataset, OADAT, not only breaks the entry barrier for the broader community to develop solutions for the OA imaging field, but also serves as a benchmark for a multitude of OA downstream tasks, which the community can build upon and compare their methodologies.

Figure 1: Optoacoustic imaging complements anatomical information from ultrasound to allow for non-invasive imaging of oxygen state, plaques, and lipid accumulation in carotid artery. For example, using an optoacoustic transducer, one can visualize both vascular structures and their inner compositions on the neck. Image courtesy of Merčep et al. 2018.
Figure 2: A representation of the composition of Experimental and Synthetic Clinical Optoacoustic Data (OADAT).
Figure 3: Visualization of different OA signal acquisition schemes used in in vivo samples of OADAT.

Gallery

Annexe

Publications

  • Lafci, B., Merčep, E., Morscher, S., Deán-Ben, X. L., & Razansky, D. (2020). Deep learning for automatic segmentation of hybrid optoacoustic ultrasound (OPUS) images. IEEE transactions on ultrasonics, ferroelectrics, and frequency control, 68(3), 688-696. doi: 10.1109/TUFFC.2020.3022324.
  • Lafci, B., Merčep, E., Herraiz, J. L., Dean-Ben, X. L., & Razansky, D. (2020). Noninvasive multiparametric characterization of mammary tumors with transmission-reflection optoacoustic ultrasound. Neoplasia, 22(12), 770-777. doi: 10.1016/j.neo.2020.10.008
  • Davoudi, N., Lafci, B., Özbek, A., Deán-Ben, X. L., & Razansky, D. (2021). Deep learning of image-and time-domain data enhances the visibility of structures in optoacoustic tomography. Optics letters, 46(13), 3029-3032. doi: 10.1364/OL.424571
  • Lafci, B., Merčep, E., Herraiz, J. L., Deán-Ben, X. L., & Razansky, D. (2021, March). Transmission-reflection optoacoustic ultrasound (TROPUS) imaging of mammary tumors. In Photons Plus Ultrasound: Imaging and Sensing 2021 (Vol. 11642, pp. 192-197). SPIE. doi: 10.1117/12.2577907
  • Hu, Y., Lafci, B., Luzgin, A., Wang, H., Klohs, J., Dean-Ben, X. L., ... & Ren, W. (2022). Deep learning facilitates fully automated brain image registration of optoacoustic tomography and magnetic resonance imaging. Biomedical Optics Express, 13(9), 4817-4833. doi: https://doi.org/10.1364/BOE.458182
  • Susmelj, A. K., Lafci, B., Ozdemir, F., Davoudi, N., Dean-Ben, X. L., Perez-Cruz, F., & Razansky, D. (2022, December). Signal domain learning approach for optoacoustic image reconstruction from limited view data. In International Conference on Medical Imaging with Deep Learning (pp. 1173-1191). PMLR.
  • Lafci, B., Robin, J., Dean-Ben, X. L., & Razansky, D. (2022). Expediting image acquisition in reflection ultrasound computed tomography. IEEE Transactions on Ultrasonics, Ferroelectrics, and Frequency Control, 69(10), 2837-2848. doi: 10.1109/TUFFC.2022.3172713.
  • Ozdemir, F., Lafci, B., Deán-Ben, X. L., Razansky, D., & Perez-Cruz, F. (2023). OADAT: experimental and synthetic clinical optoacoustic data for standardized image processing. Transactions on Machine Learning Research: 2835-8856
  • OADAT: Experimental and Synthetic Clinical Optoacoustic Data for...
  • Lafci, B., Hadjihambi, A., Determann, M., Konstantinou, C., Freijo, C., Herraiz, J. L., ... & Razansky, D. (2023). Multimodal assessment of non-alcoholic fatty liver disease with transmission-reflection optoacoustic ultrasound. Theranostics, 13(12), 4217. doi:10.7150/thno.78548.
  • Multimodal assessment of non-alcoholic fatty liver disease with transmission-reflection optoacoustic ultrasound

Bibliography

  1. Steinberg, I., Huland, D. M., Vermesh, O., Frostig, H. E., Tummers, W. S., & Gambhir, S. S. (2019). Photoacoustic clinical imaging. Photoacoustics, 14, 77-98. doi: 10.1016/j.pacs.2019.05.001
  2. Su, J. L., Wang, B., Wilson, K. E., Bayer, C. L., Chen, Y. S., Kim, S., ... & Emelianov, S. Y. (2010). Advances in clinical and biomedical applications of photoacoustic imaging. Expert opinion on medical diagnostics, 4(6), 497-510. doi: 10.1517/ 17530059.2010.529127
  3. Lafci, B., Merčep, E., Herraiz, J. L., Dean-Ben, X. L., & Razansky, D. (2020). Noninvasive multiparametric characterization of mammary tumors with transmission-reflection optoacoustic ultrasound. Neoplasia, 22(12), 770-777. doi: https://doi.org/10.1016/j.neo.2020.10.008
  4. Merčep, E., Burton, N. C., Claussen, J., & Razansky, D. (2015). Whole-body live mouse imaging by hybrid reflection-mode ultrasound and optoacoustic tomography. Optics letters, 40(20), 4643-4646. doi: 10.1364/OL.40.004643.
  5. Merčep, E., Deán-Ben, X. L., & Razansky, D. (2018). Imaging of blood flow and oxygen state with a multi-segment optoacoustic ultrasound array. Photoacoustics, 10, 48-53. doi: https://doi.org/10.1016/j.pacs.2018.04.002

Publications

More projects

ML-L3DNDT

Completed
Robust and scalable Machine Learning algorithms for Laue 3-Dimensional Neutron Diffraction Tomography
Big Science Data

BioDetect

Completed
Deep Learning for Biodiversity Detection and Classification
Energy, Climate & Environment

IRMA

In Progress
Interpretable and Robust Machine Learning for Mobility Analysis
No items found.

FLBI

In Progress
Feature Learning for Bayesian Inference
No items found.

News

Latest news

Smartair | An active learning algorithm for real-time acquisition and regression of flow field data
May 1, 2024

Smartair | An active learning algorithm for real-time acquisition and regression of flow field data

Smartair | An active learning algorithm for real-time acquisition and regression of flow field data

We’ve developed a smart solution for wind tunnel testing that learns as it works, providing accurate results faster. It provides an accurate mean flow field and turbulence field reconstruction while shortening the sampling time.
The Promise of AI in Pharmaceutical Manufacturing
April 22, 2024

The Promise of AI in Pharmaceutical Manufacturing

The Promise of AI in Pharmaceutical Manufacturing

Innovation in pharmaceutical manufacturing raises key questions: How will AI change our operations? What does this mean for the skills of our workforce? How will it reshape our collaborative efforts? And crucially, how can we fully leverage these changes?
Efficient and scalable graph generation through iterative local expansion
March 20, 2024

Efficient and scalable graph generation through iterative local expansion

Efficient and scalable graph generation through iterative local expansion

Have you ever considered the complexity of generating large-scale, intricate graphs akin to those that represent the vast relational structures of our world? Our research introduces a pioneering approach to graph generation that tackles the scalability and complexity of creating such expansive, real-world graphs.

Contact us

Let’s talk Data Science

Do you need our services or expertise?
Contact us for your next Data Science project!