Extracting activity from large 4D whole-brain image datasets

July 1, 2021
Share this project


Whole-brain recordings hold promise to revolutionize neuroscience. In the last decade, innovations in fast 3D microscopy, protein engineering, genetics, and microfluidics have allowed brain researchers to read out calcium activity at high temporal resolution from a large number of neurons in the brains of Caenorhabditis elegans, Danionella translucida, Hydra, and zebrafish simultaneously. This technology is considered to be a game changer for neuroscience as it leaves far fewer variables hidden than when only a small fraction of neuronal activities could be recorded. Many fundamental and challenging questions of neuroscience can now be pursued:

  • What global brain activity determines an organism's responses to stimuli?
  • How are decisions computed by networks of neurons?
  • What is the idle activity of an unstimulated brain?

The field suffers from a critical bottleneck. Neuronal activities are recorded as local intensity changes in 4D microscopy images. Extracting this information for a moving animal is very labor-intensive and requires expertise. The promise of whole-brain recordings cannot be fully realized unless the image analysis problem is solved.

There are several challenges:

  1. 3D images are generally difficult to annotate manually.
  2. The worm moves, rotates, bends, and compresses fast.
  3. To avoid blurring, the exposure time and the image quality are limited.
  4. The resolution in the z-direction is low.



SDSC Team:
Corinne Jones
Isinsu Katircioglu
Benjamin Béjar Haro
Guillaume Obozinski

PI | Partners:

EPFL, Laboratory of the Physics of Biological Systems:

  • Prof. Sahand Rahi
  • Dr. Elif Gençtürk
  • Alice Gross
  • Mahsa Barzegarkeshteli
  • Matthieu Schmidt

More info



The goals of the collaboration consist of identifying specific neurons across 4D images (segmentation & tracking), mapping every pixel in 4D images onto a 3D reference (registration) and speeding up the former tasks for real-time feedback to the animal.


The SDSC will help to design robust, efficient algorithm for tracking a specific set of neurons in videos of freely moving worms, and will propose machine learning techniques to align the worm images within each video and extract the activities of the neurons.


Efficient image analysis techniques would reduce the burden of manual annotation and unleash the growth of the field. Faster image analysis would mean that:– more and a more diverse range of experiments can be performed,– more animals can be analyzed ‘per paper’, making results more statistically rigorous,– more scientists could perform such experiments,– “high-throughput neuroscience” with freely moving animals will become possible,– new questions will become accessible, for example, individual differences between animals cannot be studied in a statistically rigorous way with the few worms that are usually analyzed ‘per paper’.


Figure 1: We aim to identify pieces of neurons or whole neurons in 3D images and track them in time. This can be done by mapping 3D images from different time points onto the same reference 3D image.



  • C. F. Park, M. B. Keshteli, K. Korchagina, A. Delrocq, V. Susoy, C. L. Jones, A. D. T. Samuel, S. J. Rahi. Automated neuron tracking inside moving and deforming C. elegans using deep learning and targeted augmentation. In Nature Methods, 21, 142–149 (2024).
  • C. Jones, M. Barzegar-Keshteli, A. Gross, G. Obozinski, S. J. Rahi. A Graph Matching Approach to Tracking Neurons in Freely-Moving C. elegans. In bioRxiv 2023.11.30.569341 (2023).

Additional resources


  1. S. Chaudhary, S. A. Lee, Y. Li, D. S. Patel, and H. Lu. Graphical-model framework for automated annotation of cell identities in dense cellular images. In eLife 10:e60321, Feb. 2021.
  2. S. Kato, H. S. Kaplan, T. Schrödel, S. Skora, T. H. Lindsay, E. Yemini, S. Lockery, and M. Zimmer. Global brain dynamics embed the motor command sequence of Caenorhabditis elegans. In Cell, 163 (3):656–669, Oct. 2015.
  3. J. P. Nguyen, A. N. Linder, G. S. Plummer, J. W. Shaevitz, and A. M. Leifer. Automatically tracking neurons in a moving and deforming brain. In PLOS Computational Biology, 13(5):1–19, 05 2017.
  4. X. Yu, M. S. Creamer, F. Randi, A. K. Sharma, S. W. Linderman, and A. M. Leifer. Fast deep neural correspondence for tracking and identifying neurons in C. elegans using semi-synthetic training. In eLife, 10:e66410, Jul. 2021.


Related Pages

More projects


In Progress
Detecting drought impacts on forests in earth observation data
Energy, Climate & Environment


In Progress
Acceleration of Free-boundary Grad-Shafranov codes using advanced numerical methods
Energy, Climate & Environment


In Progress
Sustainable pathways towards net zero Switzerland
Energy, Climate & Environment


In Progress
MaCHIne-Learning-assisted Ptychographic nanotomography
Big Science Data


Latest news

Smartair | An active learning algorithm for real-time acquisition and regression of flow field data
May 1, 2024

Smartair | An active learning algorithm for real-time acquisition and regression of flow field data

Smartair | An active learning algorithm for real-time acquisition and regression of flow field data

We’ve developed a smart solution for wind tunnel testing that learns as it works, providing accurate results faster. It provides an accurate mean flow field and turbulence field reconstruction while shortening the sampling time.
The Promise of AI in Pharmaceutical Manufacturing
April 22, 2024

The Promise of AI in Pharmaceutical Manufacturing

The Promise of AI in Pharmaceutical Manufacturing

Innovation in pharmaceutical manufacturing raises key questions: How will AI change our operations? What does this mean for the skills of our workforce? How will it reshape our collaborative efforts? And crucially, how can we fully leverage these changes?
Efficient and scalable graph generation through iterative local expansion
March 20, 2024

Efficient and scalable graph generation through iterative local expansion

Efficient and scalable graph generation through iterative local expansion

Have you ever considered the complexity of generating large-scale, intricate graphs akin to those that represent the vast relational structures of our world? Our research introduces a pioneering approach to graph generation that tackles the scalability and complexity of creating such expansive, real-world graphs.

Contact us

Let’s talk Data Science

Do you need our services or expertise?
Contact us for your next Data Science project!