{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T01:19:29Z","timestamp":1760145569666,"version":"build-2065373602"},"reference-count":72,"publisher":"MDPI AG","issue":"8","license":[{"start":{"date-parts":[[2024,7,27]],"date-time":"2024-07-27T00:00:00Z","timestamp":1722038400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"Flemish Government under the Onderzoeksprogramma Artifici\u00eble Intelligentie (AI) Vlaanderen"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Entropy"],"abstract":"<jats:p>We present contrast information, a novel application of some specific cases of relative entropy, designed to be useful for the cognitive modelling of the sequential perception of continuous signals. We explain the relevance of entropy in the cognitive modelling of sequential phenomena such as music and language. Then, as a first step to demonstrating the utility of constrast information for this purpose, we empirically show that its discrete case correlates well with existing successful cognitive models in the literature. We explain some interesting properties of constrast information. Finally, we propose future work toward a cognitive architecture that uses it.<\/jats:p>","DOI":"10.3390\/e26080638","type":"journal-article","created":{"date-parts":[[2024,7,29]],"date-time":"2024-07-29T12:27:43Z","timestamp":1722256063000},"page":"638","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":0,"title":["Contrast Information Dynamics: A Novel Information Measure for Cognitive Modelling"],"prefix":"10.3390","volume":"26","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-1515-1093","authenticated-orcid":false,"given":"Steven T.","family":"Homer","sequence":"first","affiliation":[{"name":"Computational Creativity Lab, Artificial Research Group, Vrije Universiteit Brussel, 1050 Etterbeek, Belgium"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-3181-5010","authenticated-orcid":false,"given":"Nicholas","family":"Harley","sequence":"additional","affiliation":[{"name":"Computational Creativity Lab, Artificial Research Group, Vrije Universiteit Brussel, 1050 Etterbeek, Belgium"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-1587-112X","authenticated-orcid":false,"given":"Geraint A.","family":"Wiggins","sequence":"additional","affiliation":[{"name":"Computational Creativity Lab, Artificial Research Group, Vrije Universiteit Brussel, 1050 Etterbeek, Belgium"},{"name":"Cognitive Science Research Group, School of Electronic Engineering and Computer Science, Queen Mary University of London, London E1 4NS, UK"}]}],"member":"1968","published-online":{"date-parts":[[2024,7,27]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","unstructured":"Shannon, C. (1948). A mathematical theory of communication. Bell Syst. Tech. J., 27, 379\u2013423, 623\u2013656.","DOI":"10.1002\/j.1538-7305.1948.tb00917.x"},{"key":"ref_2","unstructured":"Pearce, M.T. (2005). The Construction and Evaluation of Statistical Models of Melodic Structure in Music Perception and Composition. [Ph.D. Thesis, Department of Computing]."},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"625","DOI":"10.1111\/j.1756-8765.2012.01214.x","article-title":"Auditory Expectation: The Information Dynamics of Music Perception and Cognition","volume":"4","author":"Pearce","year":"2012","journal-title":"Top. Cogn. Sci."},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"43","DOI":"10.1111\/cogs.12477","article-title":"Information-Theoretic Properties of Auditory Sequences Dynamically Influence Expectation and Memory","volume":"42","author":"Agres","year":"2018","journal-title":"Cogn. Sci."},{"key":"ref_5","unstructured":"Abdallah, S.A., and Plumbley, M.D. (2010). A Measure of Statistical Complexity Based on Predictive Information. arXiv."},{"key":"ref_6","unstructured":"MacKay, D.J.C. (2003). Information Theory, Inference, and Learning Algorithms, Cambridge University Press."},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"127","DOI":"10.1038\/nrn2787","article-title":"The free-energy principle: A unified brain theory?","volume":"11","author":"Friston","year":"2010","journal-title":"Nat. Rev. Neurosci."},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"230","DOI":"10.1109\/TAMD.2010.2056368","article-title":"Formal Theory of Creativity, Fun, and Intrinsic Motivation (1990\u20132010)","volume":"2","author":"Schmidhuber","year":"2010","journal-title":"Auton. Ment. Dev. IEEE Trans."},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"181","DOI":"10.1017\/S0140525X12000477","article-title":"Whatever next? Predictive brains, situated agents, and the future of cognitive science","volume":"36","author":"Clark","year":"2013","journal-title":"Behav. Brain Sci."},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1016\/j.plrev.2018.05.001","article-title":"Creativity, Information, and Consciousness: The Information Dynamics of Thinking","volume":"34\u201335","author":"Wiggins","year":"2020","journal-title":"Phys. Life Rev."},{"key":"ref_11","doi-asserted-by":"crossref","unstructured":"Wiggins, G.A. (2021). Artificial Musical Intelligence: Computational creativity in a closed cognitive world. Artificial Intelligence and the Arts: Computational Creativity in the Visual Arts, Music, 3D, Games, and Artistic Perspectives, Springer International Publishing. Computational Synthesis and Creative Systems.","DOI":"10.1007\/978-3-030-59475-6_4"},{"key":"ref_12","doi-asserted-by":"crossref","unstructured":"Huron, D. (2006). Sweet Anticipation: Music and the Psychology of Expectation, MIT Press. Bradford Books.","DOI":"10.7551\/mitpress\/6575.001.0001"},{"key":"ref_13","unstructured":"Conklin, D. (1990). Prediction and Entropy of Music. [Master\u2019s Thesis, Department of Computer Science, University of Calgary]."},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"51","DOI":"10.1080\/09298219508570672","article-title":"Multiple Viewpoint Systems for Music Prediction","volume":"24","author":"Conklin","year":"1995","journal-title":"J. New Music. Res."},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"378","DOI":"10.1111\/nyas.13654","article-title":"Statistical learning and probabilistic prediction in music cognition: Mechanisms of stylistic enculturation","volume":"1423","author":"Pearce","year":"2018","journal-title":"Ann. N. Y. Acad. Sci."},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"1917","DOI":"10.1109\/26.61469","article-title":"Implementing the PPM data compression scheme","volume":"38","author":"Moffat","year":"1990","journal-title":"IEEE Trans. Commun."},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"76","DOI":"10.1093\/comjnl\/40.2_and_3.76","article-title":"Semantically Motivated Improvements for PPM Variants","volume":"40","author":"Bunton","year":"1997","journal-title":"Comput. J."},{"key":"ref_18","doi-asserted-by":"crossref","unstructured":"Wiil, U.K. (2005). Methods for Combining Statistical Models of Music. Computer Music Modelling and Retrieval, Springer.","DOI":"10.1007\/b105507"},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"377","DOI":"10.1525\/mp.2006.23.5.377","article-title":"Expectation in Melody: The Influence of Context and Learning","volume":"23","author":"Pearce","year":"2006","journal-title":"Music. Percept."},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"303","DOI":"10.1016\/j.neuroimage.2009.12.019","article-title":"Unsupervised Statistical Learning Underpins Computational, Behavioural and Neural Manifestations of Musical Expectation","volume":"50","author":"Pearce","year":"2010","journal-title":"NeuroImage"},{"key":"ref_21","doi-asserted-by":"crossref","unstructured":"Hansen, N.C., and Pearce, M.T. (2014). Predictive uncertainty in auditory sequence processing. Front. Psychol.","DOI":"10.3389\/fpsyg.2014.01052"},{"key":"ref_22","unstructured":"Cardoso, A., and Wiggins, G.A. (2007, January 17\u201319). Evaluating cognitive models of musical composition. Proceedings of the 4th International Joint Workshop on Computational Creativity, London, UK."},{"key":"ref_23","unstructured":"Dubnov, S. (2021). Deep Music Information Dynamics. arXiv."},{"key":"ref_24","unstructured":"Tishby, N., Pereira, F.C., and Bialek, W. (2000). The Information Bottleneck Method. arXiv."},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"1367","DOI":"10.1068\/p6507","article-title":"The role of expectation and probabilistic learning in auditory boundary perception: A model comparison","volume":"39","author":"Pearce","year":"2010","journal-title":"Perception"},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"307","DOI":"10.1177\/10298649100140S217","article-title":"Cue Abstraction, Paradigmatic Analysis and Information Dynamics: Towards Music Analysis by Cognitive Model","volume":"14","author":"Wiggins","year":"2010","journal-title":"Music. Sci."},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"81","DOI":"10.1037\/h0043158","article-title":"The Magical Number Seven, Plus or Minus Two: Some Limits on Our Capacity for Processing Information","volume":"63","author":"Miller","year":"1956","journal-title":"Pyschological Rev."},{"key":"ref_28","unstructured":"Rebuschat, P., and Williams, J. (2012). \u201cI let the music speak\u201d: Cross-domain application of a cognitive model of musical learning. Statistical Learning and Language Acquisition, Mouton De Gruyter."},{"key":"ref_29","unstructured":"Griffiths, S.S., McGinity, M.M., Forth, J., Purver, M., and Wiggins, G.A. (2015, January 28\u201329). Information-Theoretic Segmentation of Natural Language. Proceedings of the Third International Workshop on Artificial Intelligence and Cognition, Turin, Italy."},{"key":"ref_30","doi-asserted-by":"crossref","first-page":"533","DOI":"10.3758\/BF03202347","article-title":"Harmonic structure as a determinant of melodic organization","volume":"9","author":"Tan","year":"1981","journal-title":"Mem. Cogn."},{"key":"ref_31","doi-asserted-by":"crossref","first-page":"254","DOI":"10.3758\/BF03209402","article-title":"Phrasing influences the recognition of melodies","volume":"4","author":"Chiappe","year":"1997","journal-title":"Psychon. Bull. Rev."},{"key":"ref_32","doi-asserted-by":"crossref","unstructured":"Wiggins, G.A., and Sanjekdar, A. (2019). Learning and consolidation as re-representation: Revising the meaning of memory. Front. Psychol. Cogn. Sci., 10.","DOI":"10.3389\/fpsyg.2019.00802"},{"key":"ref_33","doi-asserted-by":"crossref","first-page":"3:1","DOI":"10.1145\/2967506","article-title":"Evaluation of Musical Creativity and Musical Metacreation Systems","volume":"14","author":"Agres","year":"2016","journal-title":"ACM Comput. Entertain."},{"key":"ref_34","unstructured":"Nuttall, A.L., Ren, T., Gillespie, P., Grosh, K., and de Boer, E. (2006). A generic nonlinear model for auditory perception. Auditory Mechanisms: Processes and Models, World Scientific."},{"key":"ref_35","unstructured":"Spivey, M. (2008). The Continuity of Mind, Oxford University Press."},{"key":"ref_36","doi-asserted-by":"crossref","unstructured":"Forth, J., Agres, K., Purver, M., and Wiggins, G.A. (2016). Entraining IDyOT: Timing in the information dynamics of thinking. Front. Psychol., 7.","DOI":"10.3389\/fpsyg.2016.01575"},{"key":"ref_37","doi-asserted-by":"crossref","unstructured":"Kraus, N., and Nicol, T. (2019). Brainstem Encoding of Speech and Music Sounds in Humans. The Oxford Handbook of the Auditory Brainstem, Oxford University Press.","DOI":"10.1093\/oxfordhb\/9780190849061.013.26"},{"key":"ref_38","doi-asserted-by":"crossref","unstructured":"Bellier, L., Llorens, A., Marciano, D., Gunduz, A., Schalk, G., Brunner, P., and Knight, R.T. (2023). Music can be reconstructed from human auditory cortex activity using nonlinear decoding models. PLoS Biol., 21.","DOI":"10.1371\/journal.pbio.3002176"},{"key":"ref_39","doi-asserted-by":"crossref","unstructured":"Pasley, B.N., David, S.V., Mesgarani, N., Flinker, A., Shamma, S.A., Crone, N.E., Knight, R.T., and Chang, E.F. (2023). Reconstructing Speech from Human Auditory Cortex. PLoS Biol., 10.","DOI":"10.1371\/journal.pbio.1001251"},{"key":"ref_40","unstructured":"Homer, S.T., Harley, N., and Wiggins, G.A. The Discrete Resonance Spectrogram: A novel method for precise determination of spectral content, in preparation."},{"key":"ref_41","unstructured":"Cover, T.M., and Thomas, J.A. (2006). Elements of Information Theory, John Wiley & Sons. [2nd ed.]."},{"key":"ref_42","doi-asserted-by":"crossref","first-page":"328","DOI":"10.1016\/j.jmva.2005.08.004","article-title":"Multivariate Dynamic Information","volume":"98","author":"Ebrahimi","year":"2007","journal-title":"J. Multivar. Anal."},{"key":"ref_43","doi-asserted-by":"crossref","first-page":"432","DOI":"10.1016\/j.neucom.2018.09.092","article-title":"Adaptive Entropy-Based Learning with Dynamic Artificial Neural Network","volume":"338","author":"Pinto","year":"2019","journal-title":"Neurocomputing"},{"key":"ref_44","first-page":"768","article-title":"On the Notion of Entropy of a Dynamical System","volume":"124","author":"Sinai","year":"1959","journal-title":"Dokl. Russ. Acad. Sci."},{"key":"ref_45","doi-asserted-by":"crossref","unstructured":"Layek, G. (2015). An Introduction to Dynamical Systems and Chaos, Springer.","DOI":"10.1007\/978-81-322-2556-0"},{"key":"ref_46","doi-asserted-by":"crossref","unstructured":"Brockwell, P.J., and Davis, R.A. (1991). Time Series: Theory and Methods, Springer.","DOI":"10.1007\/978-1-4419-0320-4"},{"key":"ref_47","unstructured":"Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A.N., Kaiser, L., and Polosukhin, I. (2017). Attention Is All You Need. arXiv."},{"key":"ref_48","unstructured":"Gu, A., and Dao, T. (2024). Mamba: Linear-Time Sequence Modeling with Selective State Spaces. arXiv."},{"key":"ref_49","doi-asserted-by":"crossref","unstructured":"Delgado-Bonal, A., and Marshak, A. (2019). Approximate Entropy and Sample Entropy: A Comprehensive Tutorial. Entropy, 21.","DOI":"10.3390\/e21060541"},{"key":"ref_50","doi-asserted-by":"crossref","first-page":"110","DOI":"10.1063\/1.166092","article-title":"Approximate Entropy (ApEn) as a Complexity Measure","volume":"5","author":"Pincus","year":"1995","journal-title":"Chaos Interdiscip. J. Nonlinear Sci."},{"key":"ref_51","doi-asserted-by":"crossref","first-page":"H2039","DOI":"10.1152\/ajpheart.2000.278.6.H2039","article-title":"Physiological Time-Series Analysis Using Approximate Entropy and Sample Entropy","volume":"278","author":"Richman","year":"2000","journal-title":"Am. J. Physiol.-Heart Circ. Physiol."},{"key":"ref_52","doi-asserted-by":"crossref","first-page":"021906","DOI":"10.1103\/PhysRevE.71.021906","article-title":"Multiscale Entropy Analysis of Biological Signals","volume":"71","author":"Costa","year":"2005","journal-title":"Phys. Rev. E"},{"key":"ref_53","doi-asserted-by":"crossref","first-page":"63","DOI":"10.1162\/comj.2006.30.2.63","article-title":"Spectral Anticipations","volume":"30","author":"Dubnov","year":"2006","journal-title":"Comput. Music. J."},{"key":"ref_54","unstructured":"Bialek, W., and Tishby, N. (1999). Predictive Information. arXiv."},{"key":"ref_55","doi-asserted-by":"crossref","first-page":"275","DOI":"10.1016\/j.physleta.2011.10.066","article-title":"A Measure of Statistical Complexity Based on Predictive Information with Application to Finite Spin Systems","volume":"376","author":"Abdallah","year":"2012","journal-title":"Phys. Lett. A"},{"key":"ref_56","doi-asserted-by":"crossref","first-page":"89","DOI":"10.1080\/09540090902733756","article-title":"Information Dynamics: Patterns of Expectation and Surprise in the Perception of Music","volume":"21","author":"Abdallah","year":"2009","journal-title":"Connect. Sci."},{"key":"ref_57","doi-asserted-by":"crossref","first-page":"75","DOI":"10.1063\/1.1751358","article-title":"Relative Entropy and Inductive Inference","volume":"707","author":"Caticha","year":"2004","journal-title":"AIP Conf. Proc."},{"key":"ref_58","doi-asserted-by":"crossref","unstructured":"Amari, S.I. (2016). Information Geometry and Its Applications, Springer.","DOI":"10.1007\/978-4-431-55978-8"},{"key":"ref_59","doi-asserted-by":"crossref","first-page":"191","DOI":"10.1007\/BF00288979","article-title":"Rigorous and Extended Application of Information Theory to the Afferent Visual System of the Cat. I. Basic Concepts","volume":"16","author":"Eckhorn","year":"1974","journal-title":"Kybernetik"},{"key":"ref_60","doi-asserted-by":"crossref","first-page":"325","DOI":"10.1088\/0954-898X_10_4_303","article-title":"How to Measure the Information Gained from One Symbol","volume":"10","author":"DeWeese","year":"1999","journal-title":"Network Comput. Neural Syst."},{"key":"ref_61","doi-asserted-by":"crossref","unstructured":"Dubnov, S., Assayag, G., and Cont, A. (2011, January 18\u201321). Audio Oracle Analysis of Musical Information Rate. Proceedings of the 2011 IEEE Fifth International Conference on Semantic Computing, Palo Alto, CA, USA.","DOI":"10.1109\/ICSC.2011.106"},{"key":"ref_62","unstructured":"Good, I.J. (1983). Good Thinking: The Foundations of Probability and Its Applications, University of Minnesota Press."},{"key":"ref_63","unstructured":"Braverman, M., Chen, X., Kakade, S., Narasimhan, K., Zhang, C., and Zhang, Y. (2020, January 13\u201318). Calibration, Entropy Rates, and Memory in Language Models. Proceedings of the International Conference on Machine Learning, PMLR, Virtual Event."},{"key":"ref_64","doi-asserted-by":"crossref","first-page":"396","DOI":"10.1109\/TCOM.1984.1096090","article-title":"Data Compression Using Adaptive Coding and Partial String Matching","volume":"32","author":"Cleary","year":"1984","journal-title":"IEEE Trans. Commun."},{"key":"ref_65","doi-asserted-by":"crossref","first-page":"314","DOI":"10.1080\/09298215.2016.1205632","article-title":"The Prediction of Merged Attributes with Multiple Viewpoint Systems","volume":"45","author":"Hedges","year":"2016","journal-title":"J. New Music. Res."},{"key":"ref_66","doi-asserted-by":"crossref","unstructured":"Anderson, W.J. (1991). Continuous-Time Markov Chains, Springer.","DOI":"10.1007\/978-1-4612-3038-0"},{"key":"ref_67","unstructured":"Soch, J., Monticone, P., Faulkenberry, T.J., Kipnis, A., Petrykowski, K., Allefeld, C., Atze, H., Knapp, A., and McInerney, C.D. (2024, July 23). StatProofBook\/StatProofBook.github.io: StatProofBook 2023. Available online: https:\/\/zenodo.org\/records\/10495684."},{"key":"ref_68","first-page":"63","article-title":"Continuous-Time Gaussian Autoregression","volume":"17","author":"Brockwell","year":"2007","journal-title":"Stat. Sin."},{"key":"ref_69","doi-asserted-by":"crossref","first-page":"43","DOI":"10.1016\/S0167-2789(02)00432-3","article-title":"Information Transfer in Continuous Processes","volume":"166","author":"Kaiser","year":"2002","journal-title":"Phys. D Nonlinear Phenom."},{"key":"ref_70","unstructured":"Massey, J.L. (1990). Causality, Feedback, and Directed Information. Proc. Int. Symp. Inf. Theory Applic. (ISITA-90), 2."},{"key":"ref_71","unstructured":"Alemi, A.A., Fischer, I., Dillon, J.V., and Murphy, K. (2019). Deep Variational Information Bottleneck. arXiv."},{"key":"ref_72","doi-asserted-by":"crossref","unstructured":"Fischer, I. (2020). The Conditional Entropy Bottleneck. Entropy, 22.","DOI":"10.3390\/e22090999"}],"container-title":["Entropy"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1099-4300\/26\/8\/638\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,10]],"date-time":"2025-10-10T15:25:04Z","timestamp":1760109904000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1099-4300\/26\/8\/638"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,7,27]]},"references-count":72,"journal-issue":{"issue":"8","published-online":{"date-parts":[[2024,8]]}},"alternative-id":["e26080638"],"URL":"https:\/\/doi.org\/10.3390\/e26080638","relation":{},"ISSN":["1099-4300"],"issn-type":[{"type":"electronic","value":"1099-4300"}],"subject":[],"published":{"date-parts":[[2024,7,27]]}}}