Abstract
X-ray is widely applied for transmission imaging due to its stronger penetration than natural light. When rendering novel view X-ray projections, existing methods mainly based on NeRF suffer from long training time and slow inference speed. In this paper, we propose a 3D Gaussian splatting-based method, namely X-Gaussian, for X-ray novel view synthesis. Firstly, we redesign a radiative Gaussian point cloud model inspired by the isotropic nature of X-ray imaging. Our model excludes the influence of view direction when learning to predict the radiation intensity of 3D points. Based on this model, we develop a Differentiable Radiative Rasterization (DRR) with CUDA implementation. Secondly, we customize an Angle-pose Cuboid Uniform Initialization (ACUI) strategy that directly uses the parameters of the X-ray scanner to compute the camera information and then uniformly samples point positions within a cuboid enclosing the scanned object. Experiments show that our X-Gaussian outperforms state-of-the-art methods by 6.5 dB while enjoying less than 15% training time and over 73\(\times \) inference speed. The application on CT reconstruction also reveals the practical values of our method. Code is at https://github.com/caiyuanhao1998/X-Gaussian.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Andersen, A.H., Kak, A.C.: Simultaneous algebraic reconstruction technique (SART): a superior implementation of the art algorithm. Ultrasonic imaging (1984)
Armato, I.I.I., et al.: The lung image database consortium (LIDC) and image database resource initiative (IDRI): a completed reference database of lung nodules on CT scans. Med. Phys. 38, 915–931 (2011)
Barron, J.T., Mildenhall, B., Tancik, M., Hedman, P., Martin-Brualla, R., Srinivasan, P.P.: Mip-NeRF: a multiscale representation for anti-aliasing neural radiance fields. In: ICCV (2021)
Barron, J.T., Mildenhall, B., Verbin, D., Srinivasan, P.P., Hedman, P.: Mip-NeRF 360: unbounded anti-aliased neural radiance fields. In: CVPR (2022)
Barron, J.T., Mildenhall, B., Verbin, D., Srinivasan, P.P., Hedman, P.: Zip-NeRF: anti-aliased grid-based neural radiance fields. In: ICCV (2023)
Biguri, A., Dosanjh, M., Hancock, S., Soleimani, M.: TIGRE: a MATLAB-GPU toolbox for CBCT image reconstruction. Biomed. Phys. Eng. Express 2, 055010 (2016)
Boone, J., Shah, N., Nelson, T.: A comprehensive analysis of coefficients for pendant-geometry cone-beam breast computed tomography. Med. Phys. 31, 226–235 (2004)
Boone, J.M., Nelson, T.R., Lindfors, K.K., Seibert, J.A.: Dedicated breast CT: radiation dose and image quality evaluation. Radiology 221, 657–667 (2001)
Chen, A., Xu, Z., Geiger, A., Yu, J., Su, H.: TensoRF: tensorial radiance fields. In: Avidan, S., Brostow, G., Cissé, M., Farinella, G.M., Hassner, T. (eds) Computer Vision – ECCV 2022. ECCV 2022. LNCS, vol. 13692. Springer, Cham (2022). https://doi.org/10.1007/978-3-031-19824-3_20
Chen, B., Ning, R.: Cone-beam volume CT breast imaging: Feasibility study. Med. Phys. 29, 755–770 (2002)
Chen, Z., Funkhouser, T., Hedman, P., Tagliasacchi, A.: MobileNeRF: exploiting the polygon rasterization pipeline for efficient neural field rendering on mobile architectures. In: CVPR (2023)
Cormack, A.M.: Representation of a function by its line integrals, with some radiological applications. J. Appl. Phys. 34, 2722–2727 (1963)
Cormack, A.M.: Representation of a function by its line integrals, with some radiological applications. II. J. Appl. Phys. 35, 2908–2913 (1964)
Corona-Figueroa, A., Frawley, J., Bond-Taylor, S., Bethapudi, S., Shum, H.P., Willcocks, C.G.: MedNeRF: medical neural radiance fields for reconstructing 3D-aware CT-projections from a single x-ray. In: International Conference of the IEEE Engineering in Medicine & Biology Society (EMBC) (2022)
Elbakri, I.A., Fessler, J.A.: Segmentation-free statistical image reconstruction for polyenergetic x-ray computed tomography with experimental validation. Phys. Med. Biol. 48, 2453 (2003)
Feldkamp, L.A., Davis, L.C., Kress, J.W.: Practical cone-beam algorithm. Josa a 1, 612–619 (1984)
Guide, D.: CUDA C programming guide. NVIDIA (2013)
Hounsfield, G.N.: Computerized transverse axial scanning (tomography): Part 1. description of system. British J. Radiol. 46, 1016–1022 (1973)
Hounsfield, G.N.: Computed medical imaging. Science 210, 22–28 (1980)
Hu, S., Liu, Z.: GauHuman: articulated gaussian splatting from monocular human videos. arXiv preprint arXiv: (2023)
Hu, T., Liu, S., Chen, Y., Shen, T., Jia, J.: EfficientNeRF efficient neural radiance fields. In: CVPR (2023)
Hu, W., et al.: Tri-MipRF: Tri-Mip representation for efficient anti-aliasing neural radiance fields. In: ICCV (2023)
Jiang, Y., et al.: GaussianShader: 3D Gaussian splatting with shading functions for reflective surfaces. arXiv preprint arXiv:2311.17977 (2023)
Keetha, N., et al.: Splatam: Splat, track & map 3D gaussians for dense RGB-D SLAM. arXiv preprint arXiv:2312.02126 (2023)
Kerbl, B., Kopanas, G., Leimkühler, T., Drettakis, G.: 3D Gaussian splatting for real-time radiance field rendering. ACM Trans. Graph. 42, 1–14 (2023)
Kingma, D.P., Ba, J.L.: Adam: a method for stochastic optimization. In: ICLR (2015)
Klacansky, P.: Scientific visualization datasets (2022). https://klacansky.com/open-scivis-datasets/
Kocabas, M., Chang, J.H.R., Gabriel, J., Tuzel, O., Ranjan, A.: Hugs: human gaussian splats. arXiv preprint arXiv:2311.17910 (2023)
Kopanas, G., Philip, J., Leimkühler, T., Drettakis, G.: Point-based neural rendering with per-view optimization. In: Computer Graphics Forum (2021)
Li, R., Gao, H., Tancik, M., Kanazawa, A.: NerfAcc: efficient sampling accelerates nerfs. In: ICCV (2023)
Liang, Y., Yang, X., Lin, J., Li, H., Xu, X., Chen, Y.: LucidDreamer: towards high-fidelity text-to-3D generation via interval score matching. arXiv preprint arXiv:2311.11284 (2023)
Liang, Z., Zhang, Q., Feng, Y., Shan, Y., Jia, K.: GS-IR: 3D Gaussian splatting for inverse rendering. arXiv preprint arXiv:2311.16473 (2023)
Liu, X., et al.: HumanGaussian: Text-driven 3D human generation with gaussian splatting. arXiv preprint arXiv:2311.17061 (2023)
Luiten, J., Kopanas, G., Leibe, B., Ramanan, D.: Dynamic 3D gaussians: tracking by persistent dynamic view synthesis. arXiv preprint arXiv:2308.09713 (2023)
Manglos, S.H., Gagne, G.M., Krol, A., Thomas, F.D., Narayanaswamy, R.: Transmission maximum-likelihood reconstruction with ordered subsets for cone beam ct. Phys. Med. Biol. 40, 1225 (1995)
Matsuki, H., Murai, R., Kelly, P.H., Davison, A.J.: Gaussian splatting slam. arXiv preprint arXiv:2312.06741 (2023)
Mildenhall, B., Srinivasan, P., Tancik, M., Barron, J., Ramamoorthi, R., Ng, R.: NeRF: representing scenes as neural radiance fields for view synthesis. In: ECCV (2020)
Müller, T., Evans, A., Schied, C., Keller, A.: Instant neural graphics primitives with a multiresolution hash encoding. ACM ToG (2022)
Pan, J., Zhou, T., Han, Y., Jiang, M., et al.: Variable weighted ordered subset image reconstruction algorithm. Int. J. Biomed. Imaging 2006, 010398 (2006)
Paszke, A., et al.: PyTorch: an imperative style, high-performance deep learning library. In: NeurIPS (2019)
Reiser, C., et al.: MERF: memory-efficient radiance fields for real-time view synthesis in unbounded scenes. TOG (2023)
Rückert, D., Wang, Y., Li, R., Idoughi, R., Heidrich, W.: NeAT: neural adaptive tomography. TOG (2022)
Sauer, K., Bouman, C.: A local update strategy for iterative reconstruction from projections. TIP (1993)
Scarfe, W.C., Farman, A.G., Sukovic, P., et al.: Clinical applications of cone-beam computed tomography in dental practice. J. Can. Dent. Assoc. 72, 75–80 (2006)
Sidky, E.Y., Pan, X.: Image reconstruction in circular cone-beam computed tomography by constrained, total-variation minimization. Phys. Med. Biol. 53, 4777–4807 (2008)
Snavely, N., Seitz, S.M., Szeliski, R.: Photo tourism: exploring photo collections in 3D. In: SIGGRAPH (2006)
Tang, J., Ren, J., Zhou, H., Liu, Z., Zeng, G.: DreamGaussian: generative gaussian splatting for efficient 3D content creation. arXiv preprint arXiv:2309.16653 (2023)
Verbin, D., Hedman, P., Mildenhall, B., Zickler, T., Barron, J.T., Srinivasan, P.P.: Ref-NeRF: structured view-dependent appearance for neural radiance fields. In: CVPR (2022)
Wang, Z., Bovik, A.C., Sheikh, H.R., Simoncell, E.P.: Image quality assessment: from error visibility to structural similarity. TIP (2004)
Wu, G., et al.: 4D Gaussian splatting for real-time dynamic scene rendering. arXiv preprint arXiv:2310.08528 (2023)
Xie, T., et al.: PhysGaussian: physics-integrated 3D gaussians for generative dynamics. arXiv preprint arXiv:2311.12198 (2023)
Yan, C., Qu, D., Wang, D., Xu, D., Wang, Z., Zhao, B., Li, X.: GS-SLAM: dense visual slam with 3D gaussian splatting. arXiv preprint arXiv:2311.11700 (2023)
Yang, Z., Yang, H., Pan, Z., Zhu, X., Zhang, L.: Real-time photorealistic dynamic scene representation and rendering with 4D gaussian splatting. arXiv preprint arXiv:2310.10642 (2023)
Yariv, L., et al.: BakedSDF: meshing neural SDFs for real-time view synthesis. In: SIGGRAPH (2023)
Yi, T., et al.: GaussianDreamer: fast generation from text to 3D Gaussian splatting with point cloud priors. arXiv preprint arXiv:2310.08529 (2023)
Yu, L., Zou, Y., Sidky, E.Y., Pelizzari, C.A., Munro, P., Pan, X.: Region of interest reconstruction from truncated data in circular cone-beam CT. TMI (2006)
Yugay, V., Li, Y., Gevers, T., Oswald, M.R.: Gaussian-SLAM: photo-realistic dense slam with gaussian splatting. arXiv preprint arXiv:2312.10070 (2023)
Zang, G., Idoughi, R., Li, R., Wonka, P., Heidrich, W.: IntraTomo: self-supervised learning-based tomography via sinogram synthesis and prediction. In: CVPR (2021)
Zbijewski, W., Defrise, M., Viergever, M.A., Beekman, F.J.: Statistical reconstruction for x-ray CT systems with non-continuous detectors. Phys. Med. Biol. 52, 403 (2006)
Zha, R., Zhang, Y., Li, H.: NAF: neural attenuation fields for sparse-view CBCT reconstruction. In: MICCAI (2022)
Zhang, T., et al.: PhysDreamer: physics-based interaction with 3d objects via video generation. In: ECCV (2024)
Zwicker, M., Pfister, H., Van Baar, J., Gross, M.: EWA volume splatting. In: Proceedings Visualization, 2001. VIS 2001. IEEE (2001)
Acknowledgements
This work was supported by the Lustgarten Foundation for Pancreatic Cancer Research and the Patrick J. McGovern Foundation Award.
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2025 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Cai, Y. et al. (2025). Radiative Gaussian Splatting for Efficient X-Ray Novel View Synthesis. In: Leonardis, A., Ricci, E., Roth, S., Russakovsky, O., Sattler, T., Varol, G. (eds) Computer Vision – ECCV 2024. ECCV 2024. Lecture Notes in Computer Science, vol 15059. Springer, Cham. https://doi.org/10.1007/978-3-031-73232-4_16
Download citation
DOI: https://doi.org/10.1007/978-3-031-73232-4_16
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-73231-7
Online ISBN: 978-3-031-73232-4
eBook Packages: Computer ScienceComputer Science (R0)Springer Nature Proceedings Computer Science