
4D-Brains
Extracting activity from large 4D whole-brain image datasets

Abstract
Whole-brain recordings hold promise to revolutionize neuroscience. In the last decade, innovations in fast 3D microscopy, protein engineering, genetics, and microfluidics have allowed brain researchers to read out calcium activity at high temporal resolution from a large number of neurons in the brains of Caenorhabditis elegans, Danionella translucida, Hydra, and zebrafish simultaneously. This technology is considered to be a game changer for neuroscience as it leaves far fewer variables hidden than when only a small fraction of neuronal activities could be recorded. Many fundamental and challenging questions of neuroscience can now be pursued:
- What global brain activity determines an organism's responses to stimuli?
- How are decisions computed by networks of neurons?
- What is the idle activity of an unstimulated brain?
The field suffers from a critical bottleneck. Neuronal activities are recorded as local intensity changes in 4D microscopy images. Extracting this information for a moving animal is very labor-intensive and requires expertise. The promise of whole-brain recordings cannot be fully realized unless the image analysis problem is solved.
There are several challenges:
- 3D images are generally difficult to annotate manually.
- The worm moves, rotates, bends, and compresses fast.
- To avoid blurring, the exposure time and the image quality are limited.
- The resolution in the z-direction is low.
Presentation
People
Scientists


Benjamín Béjar received a PhD in Electrical Engineering from Universidad Politécnica de Madrid in 2012. He served as a postdoctoral fellow at École Polytechnique Fédérale de Lausanne until 2017, and then he moved to Johns Hopkins University where he held a Research Faculty position until Dec. 2019. His research interests lie at the intersection of signal processing and machine learning methods, and he has worked on topics such as sparse signal recovery, time-series analysis, and computer vision methods with special emphasis on biomedical applications. Since 2021, Benjamin leads the SDSC office at the Paul Scherrer Institute in Villigen.


Corinne joined the SDSC as a senior data scientist in December 2020. She graduated with a PhD in Statistics from the University of Washington in 2020. Her doctoral research focused on representation learning for partitioning problems. Prior to her PhD, she obtained bachelor's degrees in Math, Statistics, and Economics, along with a master's degree in Economics, from Penn State University. Her research interests include deep learning and kernel-based methods, with applications in fields ranging from computer vision to oceanography.


Guillaume Obozinski graduated with a PhD in Statistics from UC Berkeley in 2009. He did his postdoc and held until 2012 a researcher position in the Willow and Sierra teams at INRIA and Ecole Normale Supérieure in Paris. He was then Research Faculty at Ecole des Ponts ParisTech until 2018. Guillaume has broad interests in statistics and machine learning and worked over time on sparse modeling, optimization for large scale learning, graphical models, relational learning and semantic embeddings, with applications in various domains from computational biology to computer vision.


Isinsu joined the SDSC in May 2022. She obtained a Bachelor’s degree in Computer Science from Middle East Technical University, Turkey, and later received a Master’s degree in Computer Science from EPFL. She conducted her doctoral studies in the Computer Vision Lab at EPFL with a focus on human body pose estimation and motion prediction using deep learning-based methods. Her research interests include self- and semi-supervised learning for image-based analysis of the human body and motion, with applications ranging from athletic training to healthcare.
Laboratory of the Physics of Biological Systems:
- Prof. Sahand Rahi
- Dr. Elif Gençtürk
- Alice Gross
- Mahsa Barzegarkeshteli
- Matthieu Schmidt
description
Goals:
The goals of the collaboration consist of identifying specific neurons across 4D images (segmentation & tracking), mapping every pixel in 4D images onto a 3D reference (registration) and speeding up the former tasks for real-time feedback to the animal.
Solution:
The SDSC will help to design robust, efficient algorithm for tracking a specific set of neurons in videos of freely moving worms, and will propose machine learning techniques to align the worm images within each video and extract the activities of the neurons.
Impact:
Efficient image analysis techniques would reduce the burden of manual annotation and unleash the growth of the field. Faster image analysis would mean that:– more and a more diverse range of experiments can be performed,– more animals can be analyzed ‘per paper’, making results more statistically rigorous,– more scientists could perform such experiments,– “high-throughput neuroscience” with freely moving animals will become possible,– new questions will become accessible, for example, individual differences between animals cannot be studied in a statistically rigorous way with the few worms that are usually analyzed ‘per paper’.
Gallery

Annexe
Additionnal resources
Bibliography
- S. Chaudhary, S. A. Lee, Y. Li, D. S. Patel, and H. Lu. Graphical-model framework for automated annotation of cell identities in dense cellular images. In eLife 10:e60321, Feb. 2021.
- S. Kato, H. S. Kaplan, T. Schrödel, S. Skora, T. H. Lindsay, E. Yemini, S. Lockery, and M. Zimmer. Global brain dynamics embed the motor command sequence of Caenorhabditis elegans. In Cell, 163 (3):656–669, Oct. 2015.
- J. P. Nguyen, A. N. Linder, G. S. Plummer, J. W. Shaevitz, and A. M. Leifer. Automatically tracking neurons in a moving and deforming brain. In PLOS Computational Biology, 13(5):1–19, 05 2017.
- X. Yu, M. S. Creamer, F. Randi, A. K. Sharma, S. W. Linderman, and A. M. Leifer. Fast deep neural correspondence for tracking and identifying neurons in C. elegans using semi-synthetic training. In eLife, 10:e66410, Jul. 2021.
Publications
Related Pages
More projects
CLIMIS4AVAL
News
Latest news


Climate-smart agriculture in sub-Saharan Africa: optimizing nitrogen fertilization with data science
Climate-smart agriculture in sub-Saharan Africa: optimizing nitrogen fertilization with data science


Street2Vec | Self-supervised learning unveils change in urban housing from street-level images
Street2Vec | Self-supervised learning unveils change in urban housing from street-level images


DLBIRHOUI | Deep Learning Based Image Reconstruction for Hybrid Optoacoustic and Ultrasound Imaging
DLBIRHOUI | Deep Learning Based Image Reconstruction for Hybrid Optoacoustic and Ultrasound Imaging
Contact us
Let’s talk Data Science
Do you need our services or expertise?
Contact us for your next Data Science project!