{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,11,2]],"date-time":"2025-11-02T02:50:21Z","timestamp":1762051821555,"version":"build-2065373602"},"reference-count":54,"publisher":"MDPI AG","issue":"5","license":[{"start":{"date-parts":[[2022,5,21]],"date-time":"2022-05-21T00:00:00Z","timestamp":1653091200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"British Telecom and Invest Northern Ireland"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Algorithms"],"abstract":"<jats:p>Concept drift, which refers to changes in the underlying process structure or customer behaviour over time, is inevitable in business processes, causing challenges in ensuring that the learned model is a proper representation of the new data. Due to factors such as seasonal effects and policy updates, concept drifts can occur in customer transitions and time spent throughout the process, either suddenly or gradually. In a concept drift context, we can discard the old data and retrain the model using new observations (sudden drift) or combine the old data with the new data to update the model (gradual drift) or maintain the model as unchanged (no drift). In this paper, we model a response to concept drift as a sequential decision making problem by combing a hierarchical Markov model and a Markov decision process (MDP). The approach can detect concept drift, retrain the model and update customer profiles automatically. We validate the proposed approach on 68 artificial datasets and a real-world hospital billing dataset, with experimental results showing promising performance.<\/jats:p>","DOI":"10.3390\/a15050174","type":"journal-article","created":{"date-parts":[[2022,5,21]],"date-time":"2022-05-21T09:18:08Z","timestamp":1653124688000},"page":"174","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":4,"title":["Detecting and Responding to Concept Drift in Business Processes"],"prefix":"10.3390","volume":"15","author":[{"ORCID":"https:\/\/orcid.org\/0000-0003-4991-6813","authenticated-orcid":false,"given":"Lingkai","family":"Yang","sequence":"first","affiliation":[{"name":"School of Computing, Ulster University, Jordanstown BT37 0QB, UK"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-6871-3504","authenticated-orcid":false,"given":"Sally","family":"McClean","sequence":"additional","affiliation":[{"name":"School of Computing, Ulster University, Jordanstown BT37 0QB, UK"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-1250-265X","authenticated-orcid":false,"given":"Mark","family":"Donnelly","sequence":"additional","affiliation":[{"name":"School of Computing, Ulster University, Jordanstown BT37 0QB, UK"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-8724-809X","authenticated-orcid":false,"given":"Kevin","family":"Burke","sequence":"additional","affiliation":[{"name":"MACSI, Mathematics Applications Consortium for Science and Industry, University of Limerick, Limerick V94 T9PX, Ireland"}]},{"given":"Kashaf","family":"Khan","sequence":"additional","affiliation":[{"name":"British Telecom, Ipswich IP5 3RE, UK"}]}],"member":"1968","published-online":{"date-parts":[[2022,5,21]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"154","DOI":"10.1109\/TNNLS.2013.2278313","article-title":"Dealing with concept drifts in process mining","volume":"25","author":"Bose","year":"2013","journal-title":"IEEE Trans. Neural Netw. Learn. Syst."},{"key":"ref_2","unstructured":"Van Der Aalst, W., Adriansyah, A., De Medeiros, A.K.A., Arcieri, F., Baier, T., Blickle, T., Bose, J.C., Van Den Brand, P., Brandtjen, R., and Buijs, J. (September, January 28). Process mining manifesto. Proceedings of the International Conference on Business Process Management, Clermont-Ferrand, France."},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"285","DOI":"10.1504\/IJBPIM.2017.088819","article-title":"The state-of-the-art of business process mining challenges","volume":"8","author":"Cho","year":"2017","journal-title":"Int. J. Bus. Process Integr. Manag."},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"441","DOI":"10.1186\/s40064-015-1183-4","article-title":"Process modeling and bottleneck mining in online peer-review systems","volume":"4","author":"Premchaiswadi","year":"2015","journal-title":"SpringerPlus"},{"key":"ref_5","doi-asserted-by":"crossref","unstructured":"Martjushev, J., Bose, R.J.C., and van der Aalst, W.M. (2015, January 26\u201328). Change point detection and dealing with gradual and multi-order dynamics in process mining. Proceedings of the International Conference on Business Informatics Research, Tartu, Estonia.","DOI":"10.1007\/978-3-319-21915-8_11"},{"key":"ref_6","doi-asserted-by":"crossref","unstructured":"Seeliger, A., Nolle, T., and M\u00fchlh\u00e4user, M. (2017, January 30\u201331). Detecting concept drift in processes using graph metrics on process graphs. Proceedings of the 9th Conference on Subject-Oriented Business Process Management, Darmstadt, Germany.","DOI":"10.1145\/3040565.3040566"},{"key":"ref_7","doi-asserted-by":"crossref","unstructured":"Stertz, F., and Rinderle-Ma, S. (2019, January 3\u20137). Detecting and Identifying Data Drifts in Process Event Streams Based on Process Histories. Proceedings of the International Conference on Advanced Information Systems Engineering, Rome, Italy.","DOI":"10.1007\/978-3-030-21297-1_21"},{"key":"ref_8","doi-asserted-by":"crossref","unstructured":"Nguyen, H., Dumas, M., La Rosa, M., and ter Hofstede, A.H. (2018, January 22\u201325). Multi-perspective comparison of business process variants based on event logs. Proceedings of the International Conference on Conceptual Modeling, Xi\u2019an, China.","DOI":"10.1007\/978-3-030-00847-5_32"},{"key":"ref_9","unstructured":"Hompes, B., Buijs, J.C., van der Aalst, W.M., Dixit, P.M., and Buurman, J. (2015, January 9\u201311). Detecting changes in process behavior using comparative case clustering. Proceedings of the International Symposium on Data-Driven Process Discovery and Analysis, Vienna, Austria."},{"key":"ref_10","unstructured":"Hompes, B., Buijs, J., Van der Aalst, W., Dixit, P., and Buurman, J. (2015, January 5\u20136). Discovering deviating cases and process variants using trace clustering. Proceedings of the 27th Benelux Conference on Artificial Intelligence (BNAIC), Hasselt, Belgium."},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"127","DOI":"10.1016\/j.datak.2010.09.002","article-title":"Reinforcement learning based resource allocation in business process management","volume":"70","author":"Huang","year":"2011","journal-title":"Data Knowl. Eng."},{"key":"ref_12","doi-asserted-by":"crossref","unstructured":"Elkhawaga, G., Abuelkheir, M., Barakat, S.I., Riad, A.M., and Reichert, M. (2020). CONDA-PM\u2014A Systematic Review and Framework for Concept Drift Analysis in Process Mining. Algorithms, 13.","DOI":"10.3390\/a13070161"},{"key":"ref_13","doi-asserted-by":"crossref","unstructured":"Ter Hofstede, A.H., Van der Aalst, W.M., Adams, M., and Russell, N. (2009). Modern Business Process Automation: YAWL and Its Support Environment, Springer Science & Business Media.","DOI":"10.1007\/978-3-642-03121-2"},{"key":"ref_14","unstructured":"(2022, March 14). Test-Driven Data Analysis (Python TDDA library). Available online: https:\/\/github.com\/tdda\/tdda."},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"407","DOI":"10.1007\/s12369-015-0283-1","article-title":"The automation of taste: A theoretical exploration of mobile ICTs and social robots in the context of music consumption","volume":"7","author":"Barile","year":"2015","journal-title":"Int. J. Soc. Robot."},{"key":"ref_16","unstructured":"Liebman, E., Zavesky, E., and Stone, P. (2018, January 10\u201315). A stitch in time-autonomous model management via reinforcement learning. Proceedings of the 17th International Conference on Autonomous Agents and Multiagent Systems, Stockholm, Sweden."},{"key":"ref_17","doi-asserted-by":"crossref","unstructured":"Yang, L., McClean, S., Donnelly, M., Burke, K., and Khan, K. (2021, January 18\u201321). Process Duration Modelling and Concept Drift Detection for Business Process Mining. Proceedings of the 2021 IEEE SmartWorld, Ubiquitous Intelligence & Computing, Advanced & Trusted Computing, Scalable Computing & Communications, Internet of People and Smart City Innovation (SmartWorld\/SCALCOM\/UIC\/ATC\/IOP\/SCI), Atlanta, GA, USA.","DOI":"10.1109\/SWC50871.2021.00097"},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"560","DOI":"10.1287\/mnsc.45.4.560","article-title":"Solving semi-Markov decision problems using average reward reinforcement learning","volume":"45","author":"Das","year":"1999","journal-title":"Manag. Sci."},{"key":"ref_19","unstructured":"Tijms, H.C., and Tijms, H.C. (1994). Stochastic Models: An Algorithmic Approach, Wiley."},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1145\/2523813","article-title":"A survey on concept drift adaptation","volume":"46","author":"Gama","year":"2014","journal-title":"ACM Comput. Surv. (CSUR)"},{"key":"ref_21","doi-asserted-by":"crossref","unstructured":"Maisenbacher, M., and Weidlich, M. (2017, January 25\u201330). Handling Concept Drift in Predictive Process Monitoring. Proceedings of the IEEE International Conference on Services Computing (SCC), Honolulu, HI, USA.","DOI":"10.1109\/SCC.2017.10"},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1145\/3375398","article-title":"Robust drift characterization from event streams of business processes","volume":"14","author":"Ostovar","year":"2020","journal-title":"ACM Trans. Knowl. Discov. Data (TKDD)"},{"key":"ref_23","unstructured":"Ostovar, A., Maaradji, A., La Rosa, M., and ter Hofstede, A.H. (July, January 28). Characterizing drift from event streams of business processes. Proceedings of the International Conference on Advanced Information Systems Engineering, Melbourne, VIC, Australia."},{"key":"ref_24","doi-asserted-by":"crossref","unstructured":"Yeshchenko, A., Di Ciccio, C., Mendling, J., and Polyvyanyy, A. (2019, January 4\u20137). Comprehensive process drift detection with visual analytics. Proceedings of the International Conference on Conceptual Modeling, Salvador, Brazil.","DOI":"10.1007\/978-3-030-33223-5_11"},{"key":"ref_25","doi-asserted-by":"crossref","unstructured":"Adams, J.N., van Zelst, S.J., Quack, L., Hausmann, K., van der Aalst, W.M., and Rose, T. (2021). A Framework for Explainable Concept Drift Detection in Process Mining. arXiv.","DOI":"10.1007\/978-3-030-85469-0_25"},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"2140","DOI":"10.1109\/TKDE.2017.2720601","article-title":"Detecting sudden and gradual drifts in business processes from execution traces","volume":"29","author":"Maaradji","year":"2017","journal-title":"IEEE Trans. Knowl. Data Eng."},{"key":"ref_27","doi-asserted-by":"crossref","unstructured":"Bose, R.J.C., and van der Aalst, W.M. (2009, January 7\u201310). Trace clustering based on conserved patterns: Towards achieving better process models. Proceedings of the International Conference on Business Process Management, Ulm, Germany.","DOI":"10.1007\/978-3-642-12186-9_16"},{"key":"ref_28","unstructured":"Hompes, B., Buijs, J.C., van der Aalst, W.M., Dixit, P.M., and Buurman, H. (2015, January 9\u201311). Detecting Change in Processes Using Comparative Trace Clustering. Proceedings of the 5th International Symposium on Data-driven Process Discovery and Analysis (SIMPDA 2015), Vienna, Austria."},{"key":"ref_29","doi-asserted-by":"crossref","unstructured":"Zheng, C., Wen, L., and Wang, J. (2017, January 23\u201328). Detecting process concept drifts from event logs. Proceedings of the OTM Confederated International Conferences \u201cOn the Move to Meaningful Internet Systems\u201d, Rhodes, Greece.","DOI":"10.1007\/978-3-319-69462-7_33"},{"key":"ref_30","doi-asserted-by":"crossref","unstructured":"Barbon Junior, S., Tavares, G.M., da Costa, V.G.T., Ceravolo, P., and Damiani, E. (2018, January 23\u201327). A framework for human-in-the-loop monitoring of concept-drift detection in event log stream. Proceedings of the Companion Proceedings of the the Web Conference 2018, Lyon, France.","DOI":"10.1145\/3184558.3186343"},{"key":"ref_31","doi-asserted-by":"crossref","unstructured":"Tavares, G.M., Ceravolo, P., Da Costa, V.G.T., Damiani, E., and Junior, S.B. (2019, January 8\u201313). Overlapping analytic stages in online process mining. Proceedings of the 2019 IEEE International Conference on Services Computing (SCC), Milan, Italy.","DOI":"10.1109\/SCC.2019.00037"},{"key":"ref_32","doi-asserted-by":"crossref","unstructured":"Spenrath, Y., and Hassani, M. (2019, January 26\u201329). Ensemble-Based Prediction of Business Processes Bottlenecks With Recurrent Concept Drifts. Proceedings of the EDBT\/ICDT Workshops, Lisbon, Portugal.","DOI":"10.7148\/2020-0190"},{"key":"ref_33","doi-asserted-by":"crossref","unstructured":"Grabski, F. (2015). Semi-Markov Processes: Applications in System Reliability and Maintenance, Elsevier.","DOI":"10.1016\/B978-0-12-800518-7.00004-1"},{"key":"ref_34","doi-asserted-by":"crossref","first-page":"45","DOI":"10.1016\/j.apm.2016.10.038","article-title":"A life-cycle optimization model using semi-markov process for highway bridge maintenance","volume":"43","author":"Wu","year":"2017","journal-title":"Appl. Math. Model."},{"key":"ref_35","unstructured":"Papadopoulou, A., McClean, S., and Garg, L. (2015). Discrete semi Markov patient pathways through hospital care via Markov modelling. Stochastic Modeling, Data Analysis and Statistical Applications, ISAST."},{"key":"ref_36","doi-asserted-by":"crossref","first-page":"361","DOI":"10.1109\/TAC.2019.2915141","article-title":"Sliding mode control for nonlinear stochastic singular semi-Markov jump systems","volume":"65","author":"Qi","year":"2019","journal-title":"IEEE Trans. Autom. Control"},{"key":"ref_37","doi-asserted-by":"crossref","unstructured":"Ferreira, D.R., Szimanski, F., and Ralha, C.G. (2012, January 3\u20136). A hierarchical Markov model to understand the behaviour of agents in business processes. Proceedings of the International Conference on Business Process Management, Tallinn, Estonia.","DOI":"10.1007\/978-3-642-36285-9_16"},{"key":"ref_38","doi-asserted-by":"crossref","first-page":"2529","DOI":"10.1080\/01605682.2020.1796540","article-title":"A joint likelihood approach to the analysis of length of stay data utilising the continuous-time hidden Markov model and Coxian phase-type distribution","volume":"72","author":"Mitchell","year":"2021","journal-title":"J. Oper. Res. Soc."},{"key":"ref_39","unstructured":"Gosavi, A. (2017). A Tutorial for Reinforcement Learning, The State University of New York at Buffalo."},{"key":"ref_40","unstructured":"Sutton, R.S., and Barto, A.G. (2018). Reinforcement Learning: An Introduction, MIT Press."},{"key":"ref_41","unstructured":"\u017bbikowski, K., Ostapowicz, M., and Gawrysiak, P. (2021). Deep Reinforcement Learning for Resource Allocation in Business Processes. arXiv."},{"key":"ref_42","doi-asserted-by":"crossref","first-page":"1238","DOI":"10.1177\/0278364913495721","article-title":"Reinforcement learning in robotics: A survey","volume":"32","author":"Kober","year":"2013","journal-title":"Int. J. Robot. Res."},{"key":"ref_43","doi-asserted-by":"crossref","unstructured":"Ye, D., Chen, G., Zhang, W., Chen, S., Yuan, B., Liu, B., Chen, J., Liu, Z., Qiu, F., and Yu, H. (2020). Towards playing full moba games with deep reinforcement learning. arXiv.","DOI":"10.1609\/aaai.v34i04.6144"},{"key":"ref_44","doi-asserted-by":"crossref","first-page":"653","DOI":"10.1109\/TNNLS.2016.2522401","article-title":"Deep direct reinforcement learning for financial signal representation and trading","volume":"28","author":"Deng","year":"2016","journal-title":"IEEE Trans. Neural Netw. Learn. Syst."},{"key":"ref_45","doi-asserted-by":"crossref","unstructured":"Gosavi, A. (2013, January 8\u201311). Relative value iteration for average reward semi-Markov control via simulation. Proceedings of the 2013 Winter Simulations Conference (WSC), Washington, DC, USA.","DOI":"10.1109\/WSC.2013.6721456"},{"key":"ref_46","doi-asserted-by":"crossref","first-page":"451","DOI":"10.1016\/S0031-3203(02)00060-2","article-title":"The global k-means clustering algorithm","volume":"36","author":"Likas","year":"2003","journal-title":"Pattern Recognit."},{"key":"ref_47","doi-asserted-by":"crossref","first-page":"261","DOI":"10.1177\/0049124104268644","article-title":"Multimodel inference: Understanding AIC and BIC in model selection","volume":"33","author":"Burnham","year":"2004","journal-title":"Sociol. Methods Res."},{"key":"ref_48","doi-asserted-by":"crossref","first-page":"524","DOI":"10.1002\/anac.200410015","article-title":"Efficient implementation of the Nelder\u2013Mead search algorithm","volume":"1","author":"Singer","year":"2004","journal-title":"Appl. Numer. Anal. Comput. Math."},{"key":"ref_49","unstructured":"Engels, B. (2022, March 14). XNomial: Exact Goodness-of-Fit Test for Multinomial Data with Fixed Probabilities. Available online: https:\/\/cran.r-project.org\/web\/packages\/XNomial\/index.html."},{"key":"ref_50","doi-asserted-by":"crossref","first-page":"185","DOI":"10.1037\/0033-2909.85.1.185","article-title":"Combining results of independent studies","volume":"85","author":"Rosenthal","year":"1978","journal-title":"Psychol. Bull."},{"key":"ref_51","doi-asserted-by":"crossref","unstructured":"Dumas, M., La Rosa, M., Mendling, J., and Reijers, H.A. (2013). Fundamentals of Business Process Management, Springer.","DOI":"10.1007\/978-3-642-33143-5"},{"key":"ref_52","unstructured":"Mannhardt, F. (2018, January 9\u201314). Multi-perspective Process Mining. Proceedings of the BPM (Dissertation\/Demos\/Industry), Sydney, Australia."},{"key":"ref_53","doi-asserted-by":"crossref","unstructured":"Mannhardt, F., de Leoni, M., Reijers, H.A., and van der Aalst, W.M. (2017, January 12\u201316). Data-driven process discovery-revealing conditional infrequent behavior from event logs. Proceedings of the International Conference on Advanced Information Systems Engineering, Essen, Germany.","DOI":"10.1007\/978-3-319-59536-8_34"},{"key":"ref_54","doi-asserted-by":"crossref","unstructured":"Klein, J.P., and Moeschberger, M.L. (2003). Survival Analysis: Techniques for Censored and Truncated Data, Springer.","DOI":"10.1007\/b97377"}],"container-title":["Algorithms"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1999-4893\/15\/5\/174\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,10]],"date-time":"2025-10-10T23:16:16Z","timestamp":1760138176000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1999-4893\/15\/5\/174"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022,5,21]]},"references-count":54,"journal-issue":{"issue":"5","published-online":{"date-parts":[[2022,5]]}},"alternative-id":["a15050174"],"URL":"https:\/\/doi.org\/10.3390\/a15050174","relation":{},"ISSN":["1999-4893"],"issn-type":[{"type":"electronic","value":"1999-4893"}],"subject":[],"published":{"date-parts":[[2022,5,21]]}}}