Skip to main content

A Riemannian Approach for Spatiotemporal Analysis and Generation of 4D Tree-Shaped Structures

  • Conference paper
  • First Online:
Computer Vision – ECCV 2024 (ECCV 2024)

Abstract

We propose the first comprehensive approach for modeling and analyzing the spatiotemporal shape variability in tree-like 4D objects, i.e., 3D objects whose shapes bend, stretch and change in their branching structure over time as they deform, grow, and interact with their environment. Our key contribution is the representation of tree-like 3D shapes using Square Root Velocity Function Trees (SRVFT) [21]. By solving the spatial registration in the SRVFT space, which is equipped with an \(\mathbb {L}^2\) metric, 4D tree-shaped structures become time-parameterized trajectories in this space. This reduces the problem of modeling and analyzing 4D tree-like shapes to that of modeling and analyzing elastic trajectories in the SRVFT space, where elasticity refers to time warping. In this paper, we propose a novel mathematical representation of the shape space of such trajectories, a Riemannian metric on that space, and computational tools for fast and accurate spatiotemporal registration and geodesics computation between 4D tree-shaped structures. Leveraging these building blocks, we develop a full framework for modelling the spatiotemporal variability using statistical models and generating novel 4D tree-like structures from a set of exemplars. We demonstrate and validate the proposed framework using real 4D plant data.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

We’re sorry, something doesn't seem to be working properly.

Please try refreshing the page. If that doesn't work, please contact support so we can address the problem.

Institutional subscriptions

Similar content being viewed by others

References

  1. Besl, P., McKay, N.: A method for registration of 3-d shapes. IEEE Trans. Pattern Anal. 14, 239–256 (1992)

    Article  Google Scholar 

  2. Chebrolu, N., Läbe, T., Stachniss, C.: Spatio-temporal non-rigid registration of 3d point clouds of plants. In: 2020 IEEE International Conference on Robotics and Automation (ICRA), pp. 3112–3118. IEEE (2020)

    Google Scholar 

  3. Chebrolu, N., Magistri, F., Läbe, T., Stachniss, C.: Registration of spatio-temporal point clouds of plants for phenotyping. PLoS ONE 16(2), e0247243 (2021)

    Article  Google Scholar 

  4. Duncan, A., Klassen, E., Srivastava, A.: Statistical shape analysis of simplified neuronal trees. Ann. Appl. Stat. 12(3), 1385–1421 (2018)

    Article  MathSciNet  Google Scholar 

  5. Feragen, A., Hauberg, S., Nielsen, M., Lauze, F.: Means in spaces of tree-like shapes. In: 2011 International Conference on Computer Vision, pp. 736–746. IEEE (2011)

    Google Scholar 

  6. Feragen, A., Lauze, F., Lo, P., de Bruijne, M., Nielsen, M.: Geometries on spaces of treelike shapes. In: Kimmel, R., Klette, R., Sugimoto, A. (eds.) ACCV 2010. LNCS, vol. 6493, pp. 160–173. Springer, Heidelberg (2011). https://doi.org/10.1007/978-3-642-19309-5_13

    Chapter  Google Scholar 

  7. Feragen, A., Lo, P., de Bruijne, M., Nielsen, M., Lauze, F.: Toward a theory of statistical tree-shape analysis. IEEE Trans. Pattern Anal. Mach. Intell. 35(8), 2008–2021 (2012)

    Article  Google Scholar 

  8. Feragen, A., et al.: Tree-space statistics and approximations for large-scale analysis of anatomical trees. In: Gee, J.C., Joshi, S., Pohl, K.M., Wells, W.M., Zöllei, L. (eds.) IPMI 2013. LNCS, vol. 7917, pp. 74–85. Springer, Heidelberg (2013). https://doi.org/10.1007/978-3-642-38868-2_7

    Chapter  Google Scholar 

  9. Jiang, B., Ren, X., Dou, M., Xue, X., Fu, Y., Zhang, Y.: Lord: local 4d implicit representation for high-fidelity dynamic human modeling. In: European Conference on Computer Vision, pp. 307–326. Springer, Heidelberg (2022). https://doi.org/10.1007/978-3-031-19809-0_18

  10. Jiang, B., Zhang, Y., Wei, X., Xue, X., Fu, Y.: H4d: human 4d modeling by learning neural compositional representation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 19355–19365 (2022)

    Google Scholar 

  11. Laga, H.: A survey on nonrigid 3d shape analysis. Acad. Press Libr. Signal Process. 6, 261–304 (2018)

    Google Scholar 

  12. Laga, H., Padilla, M., Jermyn, I.H., Kurtek, S., Bennamoun, M., Srivastava, A.: 4d atlas: statistical analysis of the spatiotemporal variability in longitudinal 3d shape data. IEEE Trans. Pattern Anal. Mach. Intell. 45(2), 1335–1352 (2022)

    Article  Google Scholar 

  13. Lobefaro, L., Malladi, M.V., Vysotska, O., Guadagnino, T., Stachniss, C.: Estimating 4d data associations towards spatial-temporal mapping of growing plants for agricultural robots. In: 2023 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp. 4212–4218. IEEE (2023)

    Google Scholar 

  14. Magistri, F., Chebrolu, N., Stachniss, C.: Segmentation-based 4d registration of plants point clouds for phenotyping. In: 2020 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp. 2433–2439. IEEE (2020)

    Google Scholar 

  15. Pan, H., Hétroy-Wheeler, F., Charlaix, J., Colliaux, D.: Multi-scale space-time registration of growing plants. In: 2021 International Conference on 3D Vision (3DV), pp. 310–319. IEEE (2021)

    Google Scholar 

  16. Schunck, D., et al.: Pheno4d: a spatio-temporal dataset of maize and tomato plant point clouds for phenotyping and advanced plant analysis. PLoS ONE 16(8), e0256340 (2021)

    Article  Google Scholar 

  17. Srivastava, A., Klassen, E., Joshi, S.H., Jermyn, I.H.: Shape analysis of elastic curves in euclidean spaces. IEEE Trans. Pattern Anal. Mach. Intell. 33(7), 1415–1428 (2010)

    Article  Google Scholar 

  18. Tsumura, R., Morishima, Y., Koseki, Y., Yoshinaka, K.: Body surface registration considering individual differences with non-rigid iterative closest point. In: International Journal of Computer Assisted Radiology and Surgery, pp. 1–10 (2023)

    Google Scholar 

  19. Wang, D., Puttonen, E., Casella, E.: Plantmove: a tool for quantifying motion fields of plant movements from point cloud time series. Int. J. Appl. Earth Obs. Geoinf. 110, 102781 (2022)

    Google Scholar 

  20. Wang, G., Laga, H., Jia, J., Xie, N., Tabia, H.: Statistical modeling of the 3d geometry and topology of botanical trees. In: Computer Graphics Forum, vol. 37, pp. 185–198. Wiley Online Library (2018)

    Google Scholar 

  21. Wang, G., Laga, H., Srivastava, A.: Elastic shape analysis of tree-like 3d objects using extended srvf representation. IEEE Trans. Pattern Anal. Mach. Intell. (2023)

    Google Scholar 

  22. Wang, G., Laga, H., Xie, N., Jia, J., Tabia, H.: The shape space of 3d botanical tree models. ACM Trans. Graph. 37(1) (2018)

    Google Scholar 

  23. Zhang, J., Yao, Y., Deng, B.: Fast and robust iterative closest point. IEEE Trans. Pattern Anal. Mach. Intell. 44(7), 3450–3466 (2021)

    Google Scholar 

  24. Zhang, T., Elnashef, B., Filin, S.: Spatio-temporal registration of plants non-rigid 3-d structure. ISPRS J. Photogramm. Remote. Sens. 205, 263–283 (2023)

    Article  Google Scholar 

Download references

Acknowledgements

This work is supported by the Australian Research Council (ARC) Discovery Projects no. DP220102197, DP210101682, and DP210102674. Tahmina Khanam is funded by the Murdoch University International Postgraduate Scholarship. Guan Wang is supported by the Scientific Research Foundation for Yangtze Delta Region Institute of University of Electronic Science and Technology of China (Huzhou) under Grant U032200114. Anuj Srivastava is supported by NSF DMS 1953087 and NSF DMS 241374. The authors would like to thank Adam Duncan for sharing with us the code of [4]. The source code of this project is publicly available, for research purposes, at https://github.com/Tahmina979/4Dtreeshape_project.

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Hamid Laga .

Editor information

Editors and Affiliations

1 Electronic supplementary material

Below is the link to the electronic supplementary material.

Supplementary material 1 (pdf 16031 KB)

Rights and permissions

Reprints and permissions

Copyright information

© 2025 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Khanam, T. et al. (2025). A Riemannian Approach for Spatiotemporal Analysis and Generation of 4D Tree-Shaped Structures. In: Leonardis, A., Ricci, E., Roth, S., Russakovsky, O., Sattler, T., Varol, G. (eds) Computer Vision – ECCV 2024. ECCV 2024. Lecture Notes in Computer Science, vol 15125. Springer, Cham. https://doi.org/10.1007/978-3-031-72855-6_19

Download citation

Keywords

Publish with us

Policies and ethics

Profiles

  1. Guanjin Wang
  2. Ferdous Sohel