Abstract
Tomography has made a revolutionary impact on the physical, biological and medical sciences. The mathematical foundation of tomography is to reconstruct a three-dimensional (3D) object from a set of two-dimensional (2D) projections. As the number of projections that can be measured from a sample is usually limited by the tolerable radiation dose and/or the geometric constraint on the tilt range, a main challenge in tomography is to achieve the best possible 3D reconstruction from a limited number of projections with noise. Over the years, a number of tomographic reconstruction methods have been developed including direct inversion, real-space, and Fourier-based iterative algorithms. Here, we report the development of a real-space iterative reconstruction (RESIRE) algorithm for accurate tomographic reconstruction. RESIRE iterates between the update of a reconstructed 3D object and the measured projections using a forward and back projection step. The forward projection step is implemented by the Fourier slice theorem or the Radon transform, and the back projection step by a linear transformation. Our numerical and experimental results demonstrate that RESIRE performs more accurate 3D reconstructions than other existing tomographic algorithms, when there are a limited number of projections with noise. Furthermore, RESIRE can be used to reconstruct the 3D structure of extended objects as demonstrated by the determination of the 3D atomic structure of an amorphous Ta thin film. We expect that RESIRE can be widely employed in the tomography applications in different fields. Finally, to make the method accessible to the general user community, the MATLAB source code of RESIRE and all the simulated and experimental data are available at https://zenodo.org/record/7273314.
Similar content being viewed by others
Introduction
Tomography has had a radical impact on diverse fields ranging from the 3D determination of the individual atoms in matter to the diagnosis of disease in medicine1,2,3,4. In the physical sciences, atomic electron tomography (AET) has been developed to resolve the atomic structure of crystal defects and amorphous materials in unprecedented 3D detail1,3,5,6,7,8,9,10. In the biological sciences, cryo-electro nmicroscopy has become a powerful tool for 3D structural determination of macromolecules with identical or similar conformations at near-atomic resolution2,11,12,13,14. For pleomorphic biological structures, cryo-electron tomography is a method of choice with a resolution in the range of 2–5 nm15,16. Tomography has also been combined with coherent diffractive imaging methods17,18,19 to perform quantitative 3D imaging of thick samples with a resolution of tens of nanometers20,21,22,23,24,25,26,27,28,29. In medicine, computed tomography has been routinely used as a diagnostic imaging procedure4. Although the applications of tomography are wide and diverse, a central problem associated with its mathematical and experimental implementation is similar, that is, how to accurately reconstruct a 3D structure from noisy and incomplete projection data30. When there are a large number of projections with low noise and no missing data, a direct inversion technique, named filtered back-projection (FBP)2,4,30, is accurate and fast. However, for most applications, there are usually missing data and high noise due to the radiation damage to the samples and/or the geometric constraint on the tilt range. To deal with these issues, real-space iterative algorithms have been developed such as algebraic reconstruction technique (ART)30,31, simultaneous algebraic reconstruction technique (SART)32 and simultaneous iterative reconstruction technique (SIRT)33,34. These algorithms minimize the difference between measured and calculated projections using least-square optimization, which can be implemented in parallel computing with a fast running time. Although ART, SIRT and SART usually outperform FBP, the forward and back projection steps in the real-space iterative algorithms are based on mathematical approximations that are not accurate enough to produce high-quality reconstructions. Fourier-based iterative algorithms have been developed to overcome this accuracy limitation, such as equal slope tomography (EST), generalized Fourier iterative reconstruction (GENFIRE) and others8,35,36,37,38,39, which rely on the Fourier transform and the inversion as the accurate forward and back projection steps, respectively. Numerical simulation and experimental results have demonstrated that EST and GENFIRE perform better than FBP, ART, SIRT and SART when there are only a limited number of projections8,35,36,40,41,42,43. EST and GENFIRE have also played an essential role in the development of AET for the 3D determination of crystal defects at the single-atom level1,3,5,6,7,8,9,10,44. Albeit powerful, Fourier-based iterative algorithms require a large memory to assemble a 3D Fourier array in each reconstruction and are also computationally expensive due to the difficulty in implementing parallel computing9,35,36. Here, we report the development of RESIRE for accurate tomographic reconstruction. Compared with real-space iterative algorithms, RESIRE uses the Fourier slice theorem or the Radon transform as the forward projection and a linear transformation as the back projection, both of which are accurate. Compared with EST and GENFIRE, RESIRE is a real-space iterative algorithm and can be implemented in parallel computing with a fast running time. Our numerical simulation and experimental results indicate that RESIRE outperforms all existing algorithms for accurate tomographic reconstruction. Furthermore, RESIRE has been used by AET to determine the 3D atomic structure of amorphous materials for the first time45,46.
Methods
Mathematical foundation of RESIRE
The tomographic reconstruction can be formulated as a least-square optimization problem which minimizes the following sum of squared errors (SSE)
where \(\{b_{\theta }\}_{\theta }\) and \(\{\theta \}_{\theta }\) are the projections and their corresponding tilt angles measured from a 3D object O. We denote that \(\Pi _{\theta }\) is a linear projection operator with respect to the tilt angle \(\theta\). For simplicity, we represent Eq. (1) with a single tilt axis, but RESIRE works as well for multiple tilt axis case with three Euler angles. The error metric \(\varepsilon\) can be decomposed into a sum of \(\varepsilon _{\theta }\) which is the error metric for each projection \(b_{\theta }\).
The gradient of the error metric \(\varepsilon\) was computed with respect to the object O using the calculus of variation method47. The following derivation is a discrete version of the continuous problem. For a voxel with a 3D coordinate \(\{u,v,w\}\), the gradient is calculated by:
where \(R_{\theta }\) is the rotation matrix. The first line of Eq. (3) is obtained by the vanilla chain rule and the second line is based on an assumption that each voxel of the object is independent from each other, that is, \(\partial O \{x,y,z \} / \partial O \{u,v,w \} = 1\) if \(\{x,y,z\} = \{u,v,w\}\) and 0 otherwise. As the transpose of a rotation matrix is its inverse, we derive the following transformation
where \(R_{i,j}\) is the (i, j)th element of the rotation matrix \(R_{\theta }\), and all the voxels \(O\{u,v,w\}\) are on a Cartesian grid (integer numbers) but the coordinates \(\{x,y\}\) are not.
Using Eqs. (3)–(4), RESIRE iterates between an updated object and the measured projections with each iteration consisting of a forward and back projection step. In the forward projection step, the projections, \(\Pi _{\theta }(O)\), are calculated from the object O of the current iteration using one of the two approaches: the Fourier slice theorem (FST) or the Radon transform. The FST approach first pads zeros to the object and then computes its oversampled Fourier transform, where padding zeros in real space is equivalent to oversampling in reciprocal space and can improve the accuracy of the approach48. 2D projections can then be calculated by taking the inverse Fourier transform of the corresponding 2D slices through the origin of the oversampled 3D Fourier transform. Alternatively, the projections, \(\Pi _{\theta }(O)\), can be computed from object O via the Radon transform49,50. Each voxel of the object is divided into equally sub-voxels, each of which is independently projected along specific tilt directions to calculate 2D projections with sub-pixel precision. The finer the sub-pixel is, the better the precision is, but at the expense of more computational power. In the “back projection” step, the measured projections are subtracted from the forward projections to obtain the differences \(\Pi _{\theta }(O) - b_{\theta }\). The gradient of the error metric \(\varepsilon _{\theta } (O)\) is computed by applying \(\Pi ^T\) on this difference \(\Pi _{\theta }(O) - b_{\theta }\). Equation (4) shows how to “back-project” a 2D image to 3D domain. The “back-projection” is in fact a linear transformation which includes a rotation and a translation, that is, each (u, v) slice of the gradient is a linear transformation of the difference. In addition, the (u, v) slices where \(w\ne 0\) corresponds to the translation of the (u, v) slice where \(w=0\). The amount of the shift is the zero order term \([R_{3,1} \;\; R_{3,2}]^T w\) in Eq. (4).
Remarks
-
1.
Regarding performance, the FST requires the oversampling ratio OR to be greater than 3; however, OR larger than 4 makes little or no further improvement. To balance the trade-off between performance and efficiency, we choose \(OR=3\) in all our experiments.
-
2.
The “forward projection” using the FST and the Radon transform are slightly different. While the FST produces smoothed images, the Radon transform constructs sharper figures. Based on specific applications, users can choose either the FST or the Radon transform to perform the forward projection step.
-
3.
Although the FST has been commonly used for 3D volume assembly in single particle cryo-EM38,39 via a software package named RELION, the limitation is the memory usage and the interpolation in the Fourier domain. RELION uses a statistical approach and works well in the case of numerous measurements. But when the measurements are limited, such as AET, which only has around 50–120 projections in a range of \([-70,70]\) degrees, tomography reconstruction becomes an ill-posed problem. Real-space constraints and regularizers can help to stabilize the reconstruction. Thus, one can further improve the Radon transform-based RESIRE by iteratively applying sparse and Tikhonov regularizes similar to the optimization techniques, such as Alternating Direction Methods of Multiplier51 (ADMM), Douglas Rachford52 (DR), Primal-Dual Hybrid Gradient53 (PDHG), and Gradient Projection Method54 (GPM).
RESIRE convergence
To prove the convergence of RESIRE, we need to find the Lipchitz constant L of the gradient that satisfies the inequality
The Lipchitz property by Eq. (5) will guarantee that RESIRE converges with the step size 1/L. When applying the inequality to RESIRE, we first assume y is the single tilt axis and reduce the analysis to the 2D case where the reconstruction is a 2D array of size \(N_z \times N_x\). For simplicity, let us assume that it is a square array with \(N_x = N_z\), and the object has a compact support with a circle shape of diameter \(N_z\) and the center coinciding with the origin of the reconstruction. We then vectorize x by stacking its columns into a vector, i.e. \(x \in {\mathbb {R}}^{N_z N_x \times 1}\). Next we decompose the projection operator \(\Pi _{\theta } \in {\mathbb {R}}^{N_x \times N_z N_x}\) into a product of two operators \(S \in {\mathbb {R}}^{N_x \times N_z N_x}\) and \(P_{\theta } \in {\mathbb {R}}^{N_zN_x \times N_z N_x}\), i.e. \(\Pi _{\theta } = S P_{\theta }\) where \(P_{\theta }\) is a rotation operator for angle \(\theta\), and S is projection operator along the z axis:
where \(\textbf{1} \in {\mathbb {R}}^{N_z \times 1}\) is a vector of all ones. As all the elements of \(P_{\theta }\) are in range [0, 1] and each row of \(P_{\theta }\) is summed to 1, the rotation operator \(P_{\theta }\) is non-expansive, that is, \(\Vert P_{\theta }^T \Vert = \Vert P_{\theta }\Vert \le 1\) where \(\Vert .\Vert\) is the \(l_2\) induced matrix norm. Furthermore, the rotation of an object must be an orthogonal operator, i.e. \(P_{\theta }^T P_{\theta } = I\). Since we approximate the rotation by linear interpolation, we have \(P_{\theta }^T P_{\theta } \preceq I\), implying that \(I - P_{\theta }^T P_{\theta }\) is semi-positive definite. Based on the above analysis, we derive the following inequalities
where all the inequalities are obtained by the triangle inequality. \(S^T S \in {\mathbb {R}}^{N_zN_x \times N_z N_x}\) is a block-diagonal matrix, containing \(N_x\) identical blocks \(\mathbf{1 \, 1}^T \in {\mathbb {R}}^{N_z \times N_z}\) which is a rank-one matrix with all elements equal to one. Since \(\mathbf{1 \, 1}^T\) has exactly one non-zero eigenvalue \(\lambda = N_z\), it results in \(\Vert S^TS\Vert = N_z\). Since \(P_{\theta }\) is non-expansive and \(\Vert S^TS\Vert =N_z\), we approximate the Lipschitz constant
This inequality is an important result of the step size analysis, showing that \(\nabla \varepsilon _{\theta }\) is \(N_z\)-Lipshitz. Since there are n projections contributing to the gradient, the Lipschitz constant increases by a factor of n, that is, the accumulated Lipchitz constant becomes \(L = nN_z\). The gradient descent step size is determined by \(t/(nN_z)\) where \(t\in (0,1]\) is the normalized step size. Although our analysis assumes \(N_x = N_z\), in practice we usually have \(N_z \le N_x\). Since Eq. (7) shows the upper bound of the Lipchitz constant, the same step size t is still applicable when \(N_z\) is not too small relative to \(N_x\). Furthermore, we can choose larger t when the object is sparse. Our experimental results show that RESIRE converges well with \(t=2\). This t value has been chosen for all our experimental results reported in this paper. Finally, we derive the iterative equation of RESIRE:
where the superscript k represents the kth iteration. Having shown that \(\nabla \epsilon\) is Lipchitz continuous with constant \(L = n N_z\), we can conclude that RESIRE with a fixed step size \(s = t/(n N_z) \le 1/L\) after k iterations will yield a solution \(O^k\) which satisfies:
where \(\epsilon (O^*)\) and \(O^*\) are the optimal error value and solution, respectively. In addition, Eq. (9) shows that the sequences \(\{O^k\}_k\) produced by the iteration in the Eq. (8) will converge to the optimal solution \(O^*\) in the first order where the rate of convergence is \(O(\frac{1}{k})\). The proof can be found elsewhere55.
Although the above analysis focuses on the single axis rotation case, RESIRE is generalized to the multiple axis rotation reconstructions. For the general case, we use the three Euler angles \((\phi ,\theta ,\psi )\) to describe the orientation of a 3D object, corresponding to the rotations around the z, y, and x-axes, respectively. The composed rotation matrix is \(R_{\{\phi ,\theta ,\psi \}} = Z_{\phi } Y_{\theta } X_{\psi }\) where \(Z_{\phi }\), \(Y_{\theta }\) and \(X_{\psi }\) are the rotation matrices around the z, y, and x-axes, respectively. By replacing \(R_{\theta }\) with \(R_{\phi ,\theta ,\psi }\) in Eq. (3), RESIRE can be used for the general rotation case.
RESIRE pseudocode
The pseudocode of the RESIRE algorithm is described below, which can be implemented with parallel computing with a fast running time.
Figure 1 shows the schematic layout of the RESIRE algorithm. To reconstruct a 3D object from given experimental projections, RESIRE minimizes the L2-norm metric using gradient descent. Each iteration of the algorithm includes four steps:
-
1.
RESIRE first computes the forward projections \(\Pi _{\theta _i}(O^k)\) of an arbitrary initial object using FST or the Radon transform, where an empty object is often used.
-
2.
The second step computes the differences \(R_i (O^k) = \Pi _{\theta _i}(O^k) - b_{\theta _i}\) between the calculated and experimental projections
-
3.
The third step back-projects the difference to 3D real space, yielding the gradient of the object: \(\nabla \varepsilon _{\theta _i}(O^k) = \Pi _{\theta ^i}^T \; R_i(O^k)\)
-
4.
Lastly, the initial object is updated using the gradient: \(O^{k+1} = O^k - \frac{t}{nN} \sum _{i=1}^n \nabla \varepsilon _{\theta _i}(O^k)\). Optional real-space constraints, such as positivity and support, if applicable, are enforced in RESIRE for better convergence and accuracy.
Repeat these four steps until the algorithm converges or the L2-norm error does not change. Typically, RESIRE reconstructs high-quality object with sufficiently optimized error metrics after several hundreds of iterations.
Angular refinement
In many tomography experiments, the measured tilt angles are not always accurate due to instrument misalignment, motor slipping, beam-induced motion, vibration, thermal effects, and/or software error. RESIRE implements the following angular refinement procedure to improve the tilt angle accuracy. Using the current tilt angles, an initial 3D object is reconstructed by RESIRE. For each jth measured projection, a series of 2D projections are calculated from the 3D object by varying the three Euler angles \(\phi \in [ \phi _j - \delta _{\phi }, \phi _j + \delta _{\phi } ]\), \(\theta \in [ \theta _j - \delta _{\theta }, \theta _j + \delta _{\theta } ]\), and \(\psi \in [ \psi _j - \delta _{\psi }, \psi _j + \delta _{\psi } ]\) are the current tilt angles. Since our angular refinement uses brutal force, we suggest the searching range is \(\pm 3^{\circ }\) within the initial angles for the sake of efficiency, i.e. \(\delta _{\phi }=\delta _{\theta }=\delta _{\psi }=3^{\circ }\). Each calculated 2D projection is then compared with the corresponding measured projection via an error metric. The three Euler angles with the smallest R-factor are recorded as the new angles for the jth measured projection. After repeating this step for all the measured projections, a new 3D object is reconstructed using the new angles. The refinement procedure is repeated until no further improvement can be made. This angular refinement procedure can improve the tilt angle accuracy and the quality of the 3D reconstruction as demonstrated with numerical simulation and experimental data36.
When working with other imaging sciences, such as single-particle, one can consider other refinement processes, such as the EMC algorithm56,57. There is a potential to combine RESIRE with such spatial refinements.
Results
RESIRE reconstruction of a simulated biological vesicle
A \(64\times 64 \times 64\) voxel model of a biological vesicle (Fig. 2a–c) is used in this test to demonstrate the performance of RESIRE relative to the other tomographic reconstruction methods. In the simulation, the tilt axis is the y-axis and the other two Euler angles \(\phi\) and \(\psi\) are set to zero. When there is enough measurement and no missing wedge, the performance of all Tomography algorithms are all great; hence we omit this test. Instead, we focus on the missing wedge case and large angular increment. The missing wedge problem is believed to be more severe than the coarser tilt angles and causes trouble to Tomographic reconstructions. A tilt series of 41 projections are calculated from the 3D model with a tilt range of \(\pm 70^{\circ }\) and an angular increment of \(3.5^{\circ }\). Poisson noise is added to each projection to make the simulation data more realistic. Reconstructions are performed using FBP, SIRT GENFIRE and RESIRE. To monitor the convergence, an R-factor \(R_F\) is used as an error metric to quantify the difference between the calculated and measured projections.
All the iterative algorithms RESIRE, GENFIRE and SIRT are run with 150 iterations. ASTRA Toolbox58,59 is used to perform the SIRT reconstruction and IMOD60 is used for FBP. The R-factors are calculated to be 11.7, 23.9, 12.9 and 9.08% for FBP, SIRT GENFIRE, and RESIRE, respectively. By minimizing the least square through iteration, RESIRE and SIRT obtain lower \(R_F\) than FBP and GENFIRE. FBP has the largest \(R_F\) because it is a direct inversion method. Although its \(R_F\) is slightly larger, GENFIRE produces a more accurate reconstruction than SIRT as demonstrated below.
Figure 2 show 10-voxel-thick central slices of the 3D reconstructions in the XY (top row), XZ (middle row) and YZ (bottom row) planes by FBP, SIRT, GENFIRE, and RESIRE, respectively, where the z-axis is the missing wedge direction. The XY central slices from all four methods show a good agreement with the model. Although the missing wedge effects are present in all the four reconstructions of the XZ and YZ slices, RESIRE exhibits less peripheral noise, more easily detectable boundaries and reduces missing wedge effects than FBP, SIRT, and GENFIRE.
To quantify the results, we calculate the Fourier shell correlation (FSC) between the model and each reconstruction. Figure 2p confirms that that RESIRE outperforms the other three methods at all spatial frequencies.
Next, we perform the angular refinement test with the same simulation data, deviating the tilt angles by adding Gaussian noise with a normal distribution (standard deviation equals 1.0) to the ground truth \(\theta\) angles. The angular refinement starts with these initial guesses and searches for optimal values in the range of \(\pm 3^{\circ }\) within the initial ones. After a couple of rounds of refinement, the Root Mean Square Error (RMSE) between the ground truth angles and the refined ones is reduced from 1.00 to 0.16. The 3D image obtained by the angular refinements is almost indistinguishable from the regular RESIRE with the ground truth angles. At the same time, the two FSC curves are nearly identical (so the figures are omitted here).
RESIRE reconstruction of the experimental data of a biomineral
To test RESIRE with experimental data, we use a tomographic tilt series acquired from an aragonite coral sample with an annular dark-field (ADF) scanning transmission electron microscope (STEM). The tilt series consists of 69 projections with a tile range from \(-60^{\circ }\) to \(+76^{\circ }\) and an angular increment of \(2^{\circ }\). The projections were pre-processed with background subtraction, normalization, and alignment. Details of the experiment and pre-processing can be found elsewhere61.
Using the same experimental data set, we perform FBP, SIRT, GENFIRE, and RESIRE, reconstructions with \(R_F\) of 25.4%, 13.5%, 7.29% and 5.30%, respectively. RESIRE produces the smallest \(R_F\), indicating that its 3D reconstruction is most consistent the 69 experimental projections. Figure 3a–h shows the XZ and YZ projections of the four 3D reconstructions, where the z-axis (horizontal) is the missing wedge direction. Compared with FBP, SIRT and GENFIRE, RESIRE reduces the reconstruction artifacts in the missing wedge direction as indicated by arrows. Figure 3i–p shows the 30-nm-thick central slices of the four reconstructions in the XZ, and YZ planes. RESIRE exhibits less peripheral noise, more reduced missing wedge effects, and sharper fine features than the other three methods (arrows).
We also report the computational time of RESIRE in the case of GPU parallel computing. On an intel core i7-7800 computer equipped with a NVIDIA GeForce GTX1080Ti GPU, RESIRE needs 0.15 s per iteration to reconstruct the 3D volume with size \(243 \times 243 \times 243\) pixels. Totally, the entire reconstruction accomplishes 200 iterations in 30 s.
RESIRE reconstruction of the 3D atomic structure of an amorphous film
To perform the tomographic reconstruction of an extended object, we use an amorphous Ta thin film with an average thickness 5 nm, which was fabricated by physical vapor deposition. A tilt series of 46 ADF-STEM projections with a tilt range from \(-63.4^{\circ }\) to \(+55.5^{\circ }\) were acquired using the TEAM 1 microscope at NCEM and LBNL. The detailed experimental parameters and data pre-processing can be found elsewhere46.
The reconstruction of the amorphous thin film is performed by FBP, SIRT, GENFIRE, and RESIRE. Figure 4a–b shows the 3D atomic model of the amorphous thin film at two different orientations reconstructed by RESIRE, where the atomic tracing procedure is reported elsewhere46. Figure 4c–f, shows the 1.61-Å-thick central slices of the amorphous thin film in the XY plane, reconstructed by FBP, SIRT, GENFIRE and RESIRE, respectively, in which RESIRE produces better atomic features than the other methods. Figure 4g–j, (o–r) and their magnified regions (k–n) and (s–v) shows the 1.61-Å-thick central slices in the YZ and XZ planes by the four methods, where more missing wedge effects are clearly visible in the FBP and SIRT reconstructions. Due to the extended nature of the thin film, there are some the artifacts near the top and bottom edges of the slices in the YZ and XZ planes. Compared with FBP, SIRT and GENFIRE, RESIRE exhibits less peripheral noise, more easily detectable atomic features and reduced missing wedge effects.
Remark
In the first test, we compared the 3D reconstructions of FBP, SIRT, GENFIRE, and RESIRE from 41 projections. In the 3D reconstruction from the experimental data of a biomineral in the second test, we used 69 projections. In the reconstruction of the 3D atomic structure of an amorphous film in the last test, 46 projections were used. The range of the tilt angles is approximately between \(-65^{\circ }\) and \(65^{\circ }\). In all these cases, RESIRE outperforms other tomographic algorithms. Thus, on the performance of the RESIRE on the projection numbers, we conclude that RESIRE consistently performs more accurate 3D reconstructions than other existing tomographic algorithms when there are a different number of 2D projections.
Conclusion
We have developed a real-space iterative algorithm, termed RESIRE, for accurate tomographic reconstruction, which uses gradient descent to solve the least-square problem \(\Vert \Pi _{\theta }O-b \Vert ^2\). Compared with other real-space iterative algorithms such as ART, SIRT and SART, RESIRE implements a more accurate forward and back projection step in the iterative process. The forward projection step is calculated by either FST with oversampling or Radon transform with sub-voxel precision. The back projection step is computed by a linear transformation. Compared with Fourier-based iterative algorithms such as EST and GENFIRE, RESIRE can be used to reconstruction extended objects in parallel computing. Numerical simulations of a biological vesicle and experimental results of an aragonite coral sample and an amorphous thin film have shown that RESIRE outperforms FBP, SIRT, and GENFIRE with less peripheral noise, more easily detectable boundaries and features, and reduced missing wedge effects. As the power of RESIRE has been demonstrated with the successful reconstructions of amorphous solids and 3D vector ptychography of magnetic materials45,46, we expect that RESIRE can be applied to a wide range of imaging modalities in different disciplines, such as AET1,3,5,6,7,8,9,10, coherent diffractive imaging16,17,18,19,20,21,22,23,24,25,26,27,28,62,63, cryo-electron tomography14,15, x-ray absorption and phase contrast imaging2,42,64,65,66,67,68, and medical computed tomography30,34,41.
Data availability
The MATLAB source codes the simulated vesicle data, and the experimental data of the biomineral and the amorphous film are available at https://zenodo.org/record/7273314.
References
Miao, J., Ercius, P. & Billinge, S. J. L. Atomic electron tomography: 3D structures without crystals. Science aaf353, 2157 (2016).
Frank, J. Three-Dimensional Electron Microscopy of Macromolecular Assemblies: Visualization of Biological Molecules in Their Native State (Oxford University Press, Oxford, 2006).
Chen, C.-C. et al. Three-dimensional imaging of dislocations in a nanoparticle at atomic resolution. Nature 496, 74–77 (2013).
Momose, A., Takeda, T., Itai, Y. & Hirano, K. Phase-contrast x-ray computed tomography for observing biological soft tissues. Nat. Med. 2, 473–475 (1996).
Scott, M. et al. Electron tomography at 2.4-ångström resolution. Nature 483, 444–447 (2012).
Xu, R. et al. Three-dimensional coordinates of individual atoms in materials revealed by electron tomography. Nat. Mater. 14, 1099–1103 (2015).
Haberfehlner, G. et al. Formation of bimetallic clusters in superfluid helium nanodroplets analysed by atomic resolution electron tomography. Nat. Commun. 6, 8779 (2015).
Yang, Y. et al. Deciphering chemical order/disorder and material properties at the single-atom level. Nature 542, 75–79 (2017).
Zhou, J. et al. Observing crystal nucleation in four dimensions using atomic electron tomography. Nature 570, 500–503 (2019).
Tian, X. et al. Correlating the three-dimensional atomic defects and electronic properties of two- dimensional transition metal dichalcogenides. Nat. Mater. 19, 867–873 (2020).
Cheng, Y. Single-particle cryo-EM at crystallographic resolution. Cell 161, 450–457 (2015).
Bartesaghi, A. et al. 2.2 Å resolution cryo-EM structure of \(\beta\)-galactosidase in complex with a cell-permeant inhibitor. Science 348, 1147–1151 (2015).
Nogales, E. The development of cryo-EM into a mainstream structural biology technique. Nat. Methods 13, 24–27 (2016).
Fernandez-Leiro, R. & Scheres, S. H. W. Unravelling biological macromolecules with cryo-electron microscopy. Nature 537, 339–346 (2016).
Lučić, V., Förster, F. & Baumeister, W. Structural studies by electron tomography: From cells to molecules. Annu. Rev. Biochem. 74, 833–865 (2005).
Oikonomou, C. M. & Jensen, G. J. A new view into prokaryotic cell biology from electron cryotomography. Nat. Rev. Microbiol. 14, 205–220 (2016).
Miao, J., Charalambous, P., Kirz, J. & Sayre, D. Extending the methodology of X-ray crystallography to allow imaging of micrometre-sized non-crystalline specimens. Nature 400, 342–344 (1999).
Rodenburg, J. M. et al. Hard-x-ray lensless imaging of extended objects. Phys. Rev. Lett. 98, 034801 (2007).
Miao, J., Ishikawa, T., Robinson, I. K. & Murnane, M. M. Beyond crystallography: Diffractive imaging using coherent X-ray light sources. Science 348, 530–535 (2015).
Miao, J. et al. Three-dimensional GaN–\(\text{Ga}_{2}\text{O}_{3}\) core shell structure revealed by X-ray diffraction microscopy. Phys. Rev. Lett. 97, 215503 (2006).
Nishino, Y., Takahashi, Y., Imamoto, N., Ishikawa, T. & Maeshima, K. Three-dimensional visualization of a human chromosome using coherent X-ray diffraction. Phys. Rev. Lett. 102, 18101 (2009).
Jiang, H. et al. Quantitative 3D imaging of whole, unstained cells by using X-ray diffraction microscopy. Proc. Natl. Acad. Sci. 107, 11234–11239 (2010).
Dierolf, M. et al. Ptychographic X-ray computed tomography at the nanoscale. Nature 467, 436–439 (2010).
Jiang, H. et al. Three-dimensional coherent X-ray diffraction imaging of molten iron in mantle olivine at nanoscale resolution. Phys. Rev. Lett. 110, 205501 (2013).
Miao, J., Ishikawa, T., Robinson, I. K. & Murnane, M. M. Beyond crystallography: Diffractive imaging using coherent X-ray light sources. Science 348, 530–535 (2015).
Holler, M. et al. High-resolution non-destructive three-dimensional imaging of integrated circuits. Nature 543, 402–406 (2017).
Donnelly, C. Hard X-ray Tomography of Three Dimensional Magnetic Structures. Ph.D. Thesis, ETH Zurich (2017).
Deng, J. et al. Correlative 3D X-ray fluorescence and ptychographic tomography of frozen-hydrated green algae. Sci. Adv. 4, eaau4548 (2018).
Michelson, A. et al. Three-dimensional visualization of nanoparticle lattices and multi-material frameworks. Science 376, 203–207 (2022).
Kak, A. C. & Slaney, M. Principles of Computerized Tomographic Imaging (Society for Industrial and Applied Mathematics, Philadelphia, 2001).
Gordon, R., Bender, R. & Herman, G. T. Algebraic reconstruction techniques (ART) for three-dimensional electron microscopy and X-ray photography. J. Theor. Biol. 29, 471–481 (1970).
Andersen, A. H. & Kak, A. C. Simultaneous algebraic reconstruction technique (SART): A superior implementation of the ART algorithm. Ultrason. Imaging 6, 81–94 (1984).
Gilbert, P. Iterative methods for the three-dimensional reconstruction of an object from projections. J. Theor. Biol. 36, 105–117 (1972).
Herman, G. T. Fundamentals of Computerized Tomography: Image Reconstruction from Projections (Springer Science & Business Media, Berlin, 2009).
Miao, J., Förster, F. & Levi, O. Equally sloped tomography with oversampling reconstruction. Phys. Rev. B 72, 52103 (2005).
Pryor, A. et al. GENFIRE: A generalized Fourier iterative reconstruction algorithm for high-resolution 3D imaging. Sci. Rep. 7, 1–12 (2017).
O’Connor, Y. Z. & Fessler, J. A. Fourier-based forward and back-projectors in iterative fan-beam tomographic image reconstruction. IEEE Trans. Med. Imaging 25, 582–589 (2006).
Scheres, S. H. RELION: Implementation of a Bayesian approach to cryo-EM structure determination. J. Struct. Biol. 180, 519–530 (2012).
Elmlund, D. & Elmlund, H. Simple: Software for ab initio reconstruction of heterogeneous single-particles. J. Struct. Biol. 180, 420–427 (2012).
Lee, E. et al. Radiation dose reduction and image enhancement in biological imaging through equally-sloped tomography. J. Struct. Biol. 164, 221–227 (2008).
Fahimian, B. P., Mao, Y., Cloetens, P. & Miao, J. Low-dose X-ray phase-contrast and absorption CT using equally sloped tomography. Phys. Med. Biol. 55, 5383 (2010).
Zhao, Y. et al. High-resolution, low-dose phase contrast X-ray tomography for 3D diagnosis of human breast cancers. Proc. Natl. Acad. Sci. 109, 18290–18294 (2012).
Fahimian, B. P. et al. Radiation dose reduction in medical X-ray CT via Fourier-based iterative reconstruction. Med. Phys. 40, 31914 (2013).
Lee, J., Jeong, C., Lee, T., Ryu, S. & Yang, Y. Direct observation of three-dimensional atomic structure of twinned metallic nanoparticles and their catalytic properties. Nano Lett. 22, 665–672 (2022).
Yang, Y. et al. Determining the three-dimensional atomic structure of an amorphous solid. Nature 592, 60–64 (2021).
Yuan, Y. et al. Three-dimensional atomic packing in amorphous solids with liquid-like structure. Nat. Mater. 21, 95–102 (2022).
Aubert, G. & Kornprobst, P. Mathematical Problems in Image Processing: Partial Differential Equations and the Calculus of Variations Vol. 147 (Springer Science & Business Media, Berlin, 2006).
Miao, J., Sayre, D. & Chapman, H. N. Phase retrieval from the magnitude of the Fourier transforms of nonperiodic objects. J. Opt. Soc. Am. A 15, 1662–1669 (1998).
Bracewell, R. N. Two-Dimensional Imaging 505–537 (Prentice-Hall Inc., Hoboken, 1995).
Lim, J. S. Two-Dimensional Signal and Image Processing 42–45 (Prentice-Hall Inc, Hoboken, 1990).
Boyd, S. et al. Distributed optimization and statistical learning via the alternating direction method of multipliers. Found. Trends Mach. Learn. 3, 1–122 (2011).
Eckstein, J. & Bertsekas, D. P. On the Douglas–Rachford splitting method and the proximal point algorithm for maximal monotone operators. Math. Program. 55, 293–318 (1992).
Zhu, M. & Chan, T. An efficient primal–dual hybrid gradient algorithm for total variation image restoration. Ucla Cam Report 34, 8–34 (2008).
Rosen, J. B. The gradient projection method for nonlinear programming. Part I Linear constraints. J. Soc. Ind. Appl. Math. 8, 181–217 (1960).
Gower, R. M. Convergence theorems for gradient descent. In Lecture Notes for Statistical Optimization (2018).
Cho, D. et al. High-throughput 3D ensemble characterization of individual core-shell nanoparticles with X-ray free electron laser single-particle imaging. ACS Nano 15, 4066–4076 (2021).
Loh, N.-T.D. & Elser, V. Reconstruction algorithm for single-particle diffraction imaging experiments. Phys. Rev. E 80, 026705 (2009).
Van Aarle, W. et al. Fast and flexible X-ray tomography using the ASTRA toolbox. Opt. Express 24, 25129–25147 (2016).
Van Aarle, W. et al. The ASTRA toolbox: A platform for advanced algorithm development in electron tomography. Ultramicroscopy 157, 35–47 (2015).
Mastronarde, D. N. Dual-axis tomography: An approach with alignment methods that preserve resolution. J. Struct. Biol. 120, 343–352 (1997).
Lo, Y. H. Developments in Mesoscale Correlative Multimodal X-ray Microscopy. Ph.D. thesis, University of California, Los Angeles (2019).
Rana, A. et al. Three-dimensional topological magnetic monopoles and their interactions in a ferromagnetic meta-lattice. Nat. Nanotechnol. 18, 227–232 (2023).
Sung, D. et al. Stochastic chromatin packing of 3D mitotic chromosomes revealed by coherent X-rays. Proc. Natl. Acad. Sci. 118, e2109921118 (2021).
Pfeiffer, F., Weitkamp, T., Bunk, O. & David, C. Phase retrieval and differential phase-contrast imaging with low-brilliance X-ray sources. Nat. Phys. 2, 258–261 (2006).
Larabell, C. A. & Nugent, K. A. Imaging cellular architecture with X-rays. Curr. Opin. Struct. Biol. 20, 623–631 (2010).
Gibbs, J. W. et al. The three-dimensional morphology of growing dendrites. Sci. Rep. 5, 11824 (2015).
Meirer, F. et al. Three-dimensional imaging of chemical phase transformations at the nanoscale with full-field transmission X-ray microscopy. J. Synchrotron Radiat. 18, 773–781 (2011).
Krenkel, M. et al. Phase-contrast zoom tomography reveals precise locations of macrophages in mouse lungs. Sci. Rep. 5, 9973 (2015).
Acknowledgements
This work was supported by the US Department of Energy (DOE), Office of Science, Basic Energy Sciences (BES), Division of Materials Sciences and Engineering under grant number DE-SC0010378 on the electron tomography experiments as well as supported by STROBE: A National Science Foundation Science and Technology Center under grant number DMR 1548924 on the development of the RESIRE algorithm. The ADF-STEM imaging with TEAM I was performed at the Molecular Foundry, Lawrence Berkeley National Laboratory, which is supported by the Office of Science, Office of Basic Energy Sciences of the US DOE under contract no. DE-AC02-05CH11231.
Author information
Authors and Affiliations
Contributions
J.M. directed the research; M.P. developed the main algorithm with help from Y.Y. and A.R.; M.P and Y.Y. performed the numerical simulation and the reconstruction of the experimental data; M.P., J.M. and Y.Y wrote the manuscript with contributions from all authors. All authors reviewed the manuscript.
Corresponding authors
Ethics declarations
Competing interests
The authors declare no competing interests.
Additional information
Publisher's note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/.
About this article
Cite this article
Pham, M., Yuan, Y., Rana, A. et al. Accurate real space iterative reconstruction (RESIRE) algorithm for tomography. Sci Rep 13, 5624 (2023). https://doi.org/10.1038/s41598-023-31124-7
Received:
Accepted:
Published:
DOI: https://doi.org/10.1038/s41598-023-31124-7
- Springer Nature Limited
This article is cited by
-
Atomic-scale identification of active sites of oxygen reduction nanocatalysts
Nature Catalysis (2024)
-
Real space iterative reconstruction for vector tomography (RESIRE-V)
Scientific Reports (2024)
-
Three-dimensional atomic insights into the metal-oxide interface in Zr-ZrO2 nanoparticles
Nature Communications (2024)
-
Maximum Lyapunov exponent-based multiple chaotic slime mold algorithm for real-world optimization
Scientific Reports (2023)
-
Three-dimensional atomic structure and local chemical order of medium- and high-entropy nanoalloys
Nature (2023)