Abstract
This paper proposes an innovative approach for optical flow-based control of micro air vehicles (MAVs), addressing challenges inherent in the nonlinearity of optical flow observables. The proposed incremental nonlinear dynamic inversion (INDI) control scheme employs an efficient data-driven approach to directly estimate the inverse of the time-varying INDI control effectiveness in real-time. This method eliminates the constant effectiveness assumption typically made by traditional INDI methods and reduces the computational burden associated with inverting this variable at each time step. It effectively handles rapidly changing system dynamics, often encountered in optical flow-based control, particularly height-dependent control variables. Stability analysis of the proposed control scheme is conducted, and its robustness and efficiency are demonstrated through both numerical simulations and real-world flight tests. These tests include multiple landings of an MAV on a static, flat surface with several different tracking setpoints, as well as hovering and landings on moving and undulating surfaces. Despite the challenges posed by noisy optical flow estimates and lateral or vertical movements of the landing surfaces, the MAV successfully tracks or lands on the surface with an exponential decay of both height and vertical velocity almost simultaneously, aligning with the desired performance.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
Notes
Video demos can be found at https://youtu.be/QX-9Hkagwr4
Paparazzi Autopilot: http://wiki.paparazziuav.org
References
Bacon, B. J., Ostroff, A. J., & Joshi, S. M. (2001). Reconfigurable NDI controller using inertial sensor failure detection & isolation. IEEE Transactions on Aerospace and Electronic Systems, 37(4), 1373–1383.
Bouquet, J. Y. (2000). Pyramidal implementation of the Lucas Kanade feature tracker. Microprocessor Research Labs, Intel Corporation.
Cesetti, A., Frontoni, E., Mancini, A., Zingaretti, P., & Longhi, S. (2010). A vision-based guidance system for UAV navigation and safe landing using natural landmarks. Journal of Intelligent and Robotic Systems, 57, 233–257.
Ching, P. L., Tan, S. C., & Ho, H. W. (2022). Ultra-wideband localization and deep-learning-based plant monitoring using micro air vehicles. Journal of Aerospace Information Systems, 19(11), 717–728.
Collett, T. S. (2002). Insect vision: Controlling actions through optic flow. Current Biology, 12(18), R615–R617.
Curtis, Andrew G.., Strong, Billie, Steager, Edward, Yim, Mark, & Rubenstein, Michael. (2023). Autonomous 3D position control for a safe single motor micro aerial vehicle. IEEE Robotics and Automation Letters, 8(6), 3566–3573.
De Croon, G. C. H. E., Dupeyroux, J. J. G., De Wagter, C., Chatterjee, A., Olejnik, D. A., & Ruffier, F. (2022). Accommodating unobservability to control flight attitude with optic flow. Nature, 610(7932), 485–490.
de Croon, B., & G.C.H.E. (2016). Monocular distance estimation with optical flow maneuvers and efference copies: A stability-based strategy. Bioinspiration & Biomimetics, 11(1), 016004.
Dong, S., Lin, T., Nieh, J. C., & Tan, K. (2023). Social signal learning of the waggle dance in honey bees. Science, 379(6636), 1015–1018.
Falanga, D., Kleber, K., & Scaramuzza, D. (2020). Dynamic obstacle avoidance for quadrotors with event cameras. Science Robotics, 5(40), eaaz9712.
Fridovich-Keil, D., Bajcsy, A., Fisac, J. F., Herbert, S. L., Wang, S., Dragan, A. D., & Tomlin, C. J. (2020). Confidence-aware motion prediction for real-time collision avoidance1. The International Journal of Robotics Research, 39(2–3), 250–265.
Haykin, S. S. (2002). Adaptive filter theory. Pearson Education India.
Herissé, B., Hamel, T., Mahony, R., & Russotto, F.-X. (2011). Landing a VTOL unmanned aerial vehicle on a moving platform using optical flow. IEEE Transactions on Robotics, 28(1), 77–89.
Ho, H. W., & Zhou, Y. (2023). Incremental nonlinear dynamic inversion based optical flow control for flying robots: An efficient data-driven approach. In Proceedings of robotics: Science and systems, Daegu, Republic of Korea, July. https://doi.org/10.15607/RSS.2023.XIX.081.
Ho, H. W., de Croon, G. C. H. E., & Chu, Q. (2017). Distance and velocity estimation using optical flow from a monocular camera. International Journal of Micro Air Vehicles, 9(3), 198–208.
Ho, H. W., de Croon, G. C. H. E., van Kampen, E., Chu, Q. P., & Mulder, M. (2018). Adaptive gain control strategy for constant optical flow divergence landing. IEEE Transactions on Robotics, 34(2), 508–516.
Isidori, A. (2013). Nonlinear control systems. Berlin: Springer.
Kendoul, F. (2014). Four-dimensional guidance and control of movement using time-to-contact: Application to automated docking and landing of unmanned rotorcraft systems. The International Journal of Robotics Research, 33(2), 237–267.
Mahlknecht, F., Gehrig, D., Nash, J., Rockenbauer, F. M., Morrell, B., Delaune, J., & Scaramuzza, D. (2022). Exploring event camera-based odometry for planetary robots. IEEE Robotics and Automation Letters, 7(4), 8651–8658.
Mehdi Yadipour, Md., Billah, A., & Faruque, I. A. (2023). Optic flow enrichment via drosophila head and retina motions to support inflight position regulation. Journal of Theoretical Biology, 562, 111416.
O’Connell, M., Shi, G., Shi, X., Azizzadenesheli, K., Anandkumar, A., Yue, Y., & Chung, S.-J. (2022). Neural-fly enables rapid learning for agile flight in strong winds. Science Robotics, 7(66), eabm6597.
Rosten, E. & Drummond, T. (2006). Machine learning for high-speed corner detection. In Computer Vision–ECCV 2006 (pp. 430–443). Springer.
Ruffier, F., & Franceschini, N. (2015). Optic flow regulation in unsteady environments: A tethered MAV achieves terrain following and targeted landing over a moving platform. Journal of Intelligent & Robotic Systems, 79, 275–293.
Sieberling, S., Chu, Q. P., & Mulder, J. A. (2010). Robust flight control using incremental nonlinear dynamic inversion and angular acceleration prediction. Journal of Guidance, Control, and Dynamics, 33(6), 1732–1742.
Slotine, Jean-Jacques E., Li, Weiping, et al. (1991). Applied nonlinear control. Englewood Cliffs: Prentice Hall.
Smeur, E. J. J., de Croon, G. C. H. E., & Chu, Q. (2018). Cascaded incremental nonlinear dynamic inversion for MAV disturbance rejection. Control Engineering Practice, 73, 79–90.
Smith, P. (1998). A simplified approach to nonlinear dynamic inversion based flight control. In 23rd atmospheric flight mechanics conference (pp. 4461).
Soria, E. (2022). Swarms of flying robots in unknown environments. Science Robotics, 7(66), eabq2215.
Steffensen, Rasmus, Steinert, Agnes, & Smeur, Ewoud JJ. (2022). Nonlinear dynamic inversion with actuator dynamics: An incremental control perspective. Journal of Guidance, Control, and Dynamics (pp. 1–9).
van’t Veld, R., Van Kampen, E.-J. &d Chu, Q. (2018). Stability and robustness analysis and improvements for incremental nonlinear dynamic inversion control. In AIAA guidance. Navigation, and control conference, p.1127.
Wang, X., Van Kampen, E. J., Chu, Q., & Peng, L. (2019). Stability analysis for incremental nonlinear dynamic inversion control. Journal of Guidance, Control, and Dynamics, 42(5), 1116–1129.
Yu, Z., Zardini, G., Censi, A., & Fuller, S. (2022). Visual confined-space navigation using an efficient learned bilinear optic flow approximation for insect-scale robots. In 2022 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) (pp 4250–4256). IEEE.
Zhou, Y. (2023). Efficient online globalized dual heuristic programming with an associated dual network. IEEE Transactions on Neural Networks and Learning Systems, 34(12), 10079–10090. https://doi.org/10.1109/TNNLS.2022.3164727
Zhou, Y., Ho, H. W., & Chu, Q. (2021). Extended incremental nonlinear dynamic inversion for optical flow control of micro air vehicles. Aerospace Science and Technology, 116, 106889.
Acknowledgements
The first and corresponding authors would like to thank Malaysian Ministry of Higher Education (MOHE) for providing the Fundamental Research Grant Scheme (FRGS) (Grant number: FRGS/1/2020/TK0/USM/03/3) for conducting this research.
Author information
Authors and Affiliations
Contributions
H.W.H. and Y.Z wrote the main manuscript text. Y.Z. performed the simulations. H.W.H. carried out the implementation and experiments. Y.Z. and Y.F. conducted theoretical analysis. H.W.H., Y.Z, and G.C.H.E.d.C discussed the results. G.C.H.E.d.C. reviewed the manuscript. H.W.H. and Y.Z administrated the project and acquired the funding support.
Corresponding author
Ethics declarations
Ethical statements
Not applicable.
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.
About this article
Cite this article
Ho, H.W., Zhou, Y., Feng, Y. et al. Optical flow-based control for micro air vehicles: an efficient data-driven incremental nonlinear dynamic inversion approach. Auton Robot 48, 22 (2024). https://doi.org/10.1007/s10514-024-10174-4
Received:
Accepted:
Published:
Version of record:
DOI: https://doi.org/10.1007/s10514-024-10174-4