ALBUM

Motivation

‍Understanding the detailed wiring of the brain requires imaging and analyzing neural tissue at nanometer-scale resolution over very large volumes. Volume electron microscopy (vEM) has become a key technology for this purpose, as it can reveal cellular and subcellular structures with the resolution needed for connectomics and the study of healthy and diseased brain tissue. However, current vEM workflows remain difficult to scale. Conventional serial-section methods rely on cutting thousands of ultrathin sections, typically 30–40 nm thick, and collecting them reliably. This process is slow, technically demanding, and highly sensitive to artifacts such as section loss, cracks, folds, distortions, and insufficient axial resolution. These artifacts directly affect the quality of reconstructed 3D volumes and are among the main reasons why automated neuron segmentation still produces errors such as over-merging and over-segmentation.

The motivation of this project is to develop a more scalable computational imaging and analysis pipeline for large-scale brain tissue reconstruction. By using semi-thin sections combined with iterative broad ion-beam milling and cumulative transmission imaging, we aim to reduce the dependence on fragile ultrathin sectioning while still recovering high-resolution 3D information. This creates an important computational challenge: the acquired images are cumulative projections through the remaining sample, so recovering the underlying 3D volume requires solving an ill-conditioned inverse problem. At the same time, the reconstructed volumes must be suitable for downstream automated neuron segmentation, which is essential for biological interpretation.

Therefore, this project is motivated by the need to jointly address reconstruction and segmentation rather than treating them as independent problems. Improved reconstruction can provide cleaner 3D volumes, while robust deep-learning-based segmentation can transform these volumes into biologically meaningful neural structures. By combining inverse-problem-based reconstruction, optimized acquisition design, and biologically informed deep-learning segmentation, the project aims to enable more accurate, efficient, and scalable analysis of large vEM datasets for connectomics and neuroscience.

Proposed Approach / Solution

‍The proposed solution is a complete computational pipeline that addresses both 3D reconstruction and 3D neuron segmentation for cumulative volume electron microscopy. The first part of the approach focuses on reconstructing high-resolution 3D volumes from cumulative transmission measurements acquired during iterative layer removal. In this imaging setup, semi-thin sections are gradually milled using a broad ion beam, and after each milling step, a projection image is acquired through the remaining sample using multibeam scanning transmission electron microscopy. Since each image contains cumulative information from the remaining layers, recovering the underlying 3D structure is formulated as an inverse problem. We model the image formation process using a cumulative-sum forward operator and develop optimization-based reconstruction strategies to recover the individual layers from the projection sequence. To address the ill-conditioning of the inverse problem and detector noise, we use linearization and regularized optimization methods. In addition, we investigate acquisition design by identifying maximally informative layer-removal patterns, so that better reconstructions can be obtained under the same experimental budget.

The second part of the approach focuses on automated 3D neuron segmentation from the reconstructed EM volumes. We establish two strong deep-learning baselines: CAD, which uses cross-dimension affinity distillation to transfer 3D structural knowledge to an efficient 2D student network, and AGQ, which uses affinity-guided queries to directly predict neuron segmentations more efficiently than traditional watershed-based pipelines. Building on these baselines, we propose a biologically informed refinement strategy that improves AGQ segmentation outputs using a biological graph representation. This graph-based refinement incorporates structural consistency and biological plausibility into the final segmentation, helping to reduce over-merging and over-segmentation errors.

Overall, the proposed approach combines inverse-problem-based 3D reconstruction, information-theoretic acquisition optimization, and deep-learning-based neuron segmentation. This integrated solution is designed to produce high-quality 3D EM volumes and more accurate neuron segmentations for scalable connectomics analysis.

Impact

‍The expected impact of this project is to make large-scale volume electron microscopy more accurate, efficient, and scalable for brain connectomics. By reducing the dependence on fragile ultrathin serial sectioning, the proposed imaging and reconstruction pipeline can lower artifact-related failures and improve the quality of 3D EM volumes. At the same time, improved deep-learning-based neuron segmentation can reduce over-merging and over-segmentation errors, enabling more reliable analysis of neural circuits. Together, these advances can support petabyte-scale reconstruction and segmentation of brain tissue, helping researchers better study ultrastructural changes in healthy and diseased brains.

‍