{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,2]],"date-time":"2026-04-02T18:09:20Z","timestamp":1775153360612,"version":"3.50.1"},"reference-count":118,"publisher":"Association for Computing Machinery (ACM)","issue":"1","license":[{"start":{"date-parts":[[2015,2,17]],"date-time":"2015-02-17T00:00:00Z","timestamp":1424131200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"funder":[{"DOI":"10.13039\/501100002341","name":"Academy of Finland","doi-asserted-by":"crossref","award":["#133021"],"award-info":[{"award-number":["#133021"]}],"id":[{"id":"10.13039\/501100002341","id-type":"DOI","asserted-by":"crossref"}]}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Trans. Inf. Syst."],"published-print":{"date-parts":[[2015,2,17]]},"abstract":"<jats:p>Evaluation is central in research and development of information retrieval (IR). In addition to designing and implementing new retrieval mechanisms, one must also show through rigorous evaluation that they are effective. A major focus in IR is IR mechanisms\u2019 capability of ranking relevant documents optimally for the users, given a query. Searching for information in practice involves searchers, however, and is highly interactive. When human searchers have been incorporated in evaluation studies, the results have often suggested that better ranking does not necessarily lead to better search task, or work task, performance. Therefore, it is not clear which system or interface features should be developed to improve the effectiveness of human task performance. In the present article, we focus on the evaluation of task-based information interaction (TBII). We give special emphasis to learning tasks to discuss TBII in more concrete terms. Information interaction is here understood as behavioral and cognitive activities related to task planning, searching information items, selecting between them, working with them, and synthesizing and reporting. These five generic activities contribute to task performance and outcome and can be supported by information systems. In an attempt toward task-based evaluation, we introduce program theory as the evaluation framework. Such evaluation can investigate whether a program consisting of TBII activities and tools works and how it works and, further, provides a causal description of program (in)effectiveness. Our goal in the present article is to structure TBII on the basis of the five generic activities and consider the evaluation of each activity using the program theory framework. Finally, we combine these activity-based program theories in an overall evaluation framework for TBII. Such an evaluation is complex due to the large number of factors affecting information interaction. Instead of presenting tested program theories, we illustrate how the evaluation of TBII should be accomplished using the program theory framework in the evaluation of systems and behaviors, and their interactions, comprehensively in context.<\/jats:p>","DOI":"10.1145\/2699660","type":"journal-article","created":{"date-parts":[[2015,3,19]],"date-time":"2015-03-19T12:13:49Z","timestamp":1426767229000},"page":"1-30","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":54,"title":["Task-Based Information Interaction Evaluation"],"prefix":"10.1145","volume":"33","author":[{"given":"Kalervo","family":"J\u00e4rvelin","sequence":"first","affiliation":[{"name":"University of Tampere, Finland"}]},{"given":"Pertti","family":"Vakkari","sequence":"additional","affiliation":[{"name":"University of Tampere, Finland"}]},{"given":"Paavo","family":"Arvola","sequence":"additional","affiliation":[{"name":"University of Tampere, Finland"}]},{"given":"Feza","family":"Baskaya","sequence":"additional","affiliation":[{"name":"University of Tampere, Finland"}]},{"given":"Anni","family":"J\u00e4rvelin","sequence":"additional","affiliation":[{"name":"University of Tampere, Finland"}]},{"given":"Jaana","family":"Kek\u00e4l\u00e4inen","sequence":"additional","affiliation":[{"name":"University of Tampere, Finland"}]},{"given":"Heikki","family":"Keskustalo","sequence":"additional","affiliation":[{"name":"University of Tampere, Finland"}]},{"given":"Sanna","family":"Kumpulainen","sequence":"additional","affiliation":[{"name":"University of Tampere, Finland"}]},{"given":"Miamaria","family":"Saastamoinen","sequence":"additional","affiliation":[{"name":"University of Tampere, Finland"}]},{"given":"Reijo","family":"Savolainen","sequence":"additional","affiliation":[{"name":"University of Tampere, Finland"}]},{"given":"Eero","family":"Sormunen","sequence":"additional","affiliation":[{"name":"University of Tampere, Finland"}]}],"member":"320","published-online":{"date-parts":[[2015,3,17]]},"reference":[{"key":"e_1_2_1_1_1","doi-asserted-by":"publisher","DOI":"10.1145\/274644.274679"},{"key":"e_1_2_1_2_1","volume-title":"Uncertainty in action: Observing information seeking within the creative processes of scholarly research. Inf. Res. 12, 1","author":"Anderson T. D.","year":"2006","unstructured":"T. D. Anderson . 2006. Uncertainty in action: Observing information seeking within the creative processes of scholarly research. Inf. Res. 12, 1 ( 2006 ), Article 283. T. D. Anderson. 2006. Uncertainty in action: Observing information seeking within the creative processes of scholarly research. Inf. Res. 12, 1 (2006), Article 283."},{"key":"e_1_2_1_3_1","volume-title":"Workshop Proceedings of the 33rd Annual International ACM SIGIR Conference: Simulation of Interaction, Automated Evaluation of Interactive IR. ACM","author":"Arvola P.","unstructured":"P. Arvola and J. Kek\u00e4l\u00e4inen . 2010. Simulating user interaction in result document browsing . In Workshop Proceedings of the 33rd Annual International ACM SIGIR Conference: Simulation of Interaction, Automated Evaluation of Interactive IR. ACM , New York, NY, 27--28. P. Arvola and J. Kek\u00e4l\u00e4inen. 2010. Simulating user interaction in result document browsing. In Workshop Proceedings of the 33rd Annual International ACM SIGIR Conference: Simulation of Interaction, Automated Evaluation of Interactive IR. ACM, New York, NY, 27--28."},{"key":"e_1_2_1_4_1","doi-asserted-by":"publisher","DOI":"10.1007\/s10791-010-9133-9"},{"key":"e_1_2_1_5_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.ipm.2011.02.006"},{"key":"e_1_2_1_6_1","doi-asserted-by":"publisher","DOI":"10.1108\/00220410310463860"},{"key":"e_1_2_1_7_1","doi-asserted-by":"publisher","DOI":"10.1016\/S0306-4573(97)00078-2"},{"key":"e_1_2_1_8_1","doi-asserted-by":"publisher","DOI":"10.1108\/eb024320"},{"key":"e_1_2_1_9_1","doi-asserted-by":"publisher","DOI":"10.1016\/0306-4573(90)90103-9"},{"key":"e_1_2_1_10_1","volume-title":"What is browsing\u2014really&quest","author":"Bates M. J.","year":"2007","unstructured":"M. J. Bates . 2007. What is browsing\u2014really&quest ; A model drawing from behavioural science research. Inf. Res. 12, 4 ( 2007 ), Article 330. M. J. Bates. 2007. What is browsing\u2014really&quest; A model drawing from behavioural science research. Inf. Res. 12, 4 (2007), Article 330."},{"key":"e_1_2_1_11_1","first-page":"133","article-title":"Anomalous states of knowledge as a basis for information retrieval","volume":"5","author":"Belkin N. J.","year":"1980","unstructured":"N. J. Belkin . 1980 . Anomalous states of knowledge as a basis for information retrieval . Can. J. Inform. Lib. Sci. 5 (1980), 133 -- 143 . N. J. Belkin. 1980. Anomalous states of knowledge as a basis for information retrieval. Can. J. Inform. Lib. Sci. 5 (1980), 133--143.","journal-title":"Can. J. Inform. Lib. Sci."},{"key":"e_1_2_1_12_1","volume-title":"The Janus Faced Scholar. A Festschrift in Honor of Peter Ingwersen","author":"Belkin N. J.","unstructured":"N. J. Belkin . 2010. On the evaluation of interactive information retrieval systems . In The Janus Faced Scholar. A Festschrift in Honor of Peter Ingwersen . Det Informationsvidenskabelige Akademi (Royal School of Library and Information Science , Copenhagen ); ISSI (International Society for Scientometrics and Informetrics), Copenhagen, Denmark, 13--22. N. J. Belkin. 2010. On the evaluation of interactive information retrieval systems. In The Janus Faced Scholar. A Festschrift in Honor of Peter Ingwersen. Det Informationsvidenskabelige Akademi (Royal School of Library and Information Science, Copenhagen); ISSI (International Society for Scientometrics and Informetrics), Copenhagen, Denmark, 13--22."},{"key":"e_1_2_1_13_1","doi-asserted-by":"publisher","DOI":"10.1177\/016555158200500501"},{"key":"e_1_2_1_14_1","volume-title":"Clinical Decision Support Systems","author":"Berner E. S.","unstructured":"E. S. Berner . 2007. Clinical Decision Support Systems . Springer, New York , NY. E. S. Berner. 2007. Clinical Decision Support Systems. Springer, New York, NY."},{"key":"e_1_2_1_15_1","doi-asserted-by":"publisher","DOI":"10.1177\/000494418803200201"},{"key":"e_1_2_1_16_1","doi-asserted-by":"crossref","unstructured":"A. Blandford and S. Attfield. 2010. Interacting with Information. Morgan and Claypool San Rafael CA.   A. Blandford and S. Attfield. 2010. Interacting with Information. Morgan and Claypool San Rafael CA.","DOI":"10.2200\/S00227ED1V01Y200911HCI006"},{"key":"e_1_2_1_17_1","unstructured":"B. S. Bloom M. D. Engelhart E. J. Furst W. H. Hill and D. R. Krathwohl. 1956. Taxonomy of Educational Objectives: The Classification of Educational Goals. David McKay New York NY.  B. S. Bloom M. D. Engelhart E. J. Furst W. H. Hill and D. R. Krathwohl. 1956. Taxonomy of Educational Objectives: The Classification of Educational Goals. David McKay New York NY."},{"key":"e_1_2_1_18_1","doi-asserted-by":"publisher","DOI":"10.1002\/asi.4630340504"},{"key":"e_1_2_1_19_1","doi-asserted-by":"publisher","DOI":"10.1145\/253495.253533"},{"key":"e_1_2_1_20_1","doi-asserted-by":"publisher","DOI":"10.1108\/EUM0000000007110"},{"key":"e_1_2_1_21_1","doi-asserted-by":"publisher","DOI":"10.1002\/asi.10286"},{"key":"e_1_2_1_22_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.lisr.2009.09.005"},{"key":"e_1_2_1_23_1","doi-asserted-by":"publisher","DOI":"10.1145\/2348283.2348342"},{"key":"e_1_2_1_24_1","doi-asserted-by":"publisher","DOI":"10.1145\/1998076.1998136"},{"key":"e_1_2_1_25_1","doi-asserted-by":"publisher","DOI":"10.1002\/asi.v56:10"},{"key":"e_1_2_1_26_1","doi-asserted-by":"publisher","DOI":"10.1016\/0306-4573(94)00041-Z"},{"key":"e_1_2_1_27_1","doi-asserted-by":"publisher","DOI":"10.1037\/0012-1649.25.5.765"},{"key":"e_1_2_1_28_1","doi-asserted-by":"publisher","DOI":"10.1023\/A:1022908820784"},{"key":"e_1_2_1_29_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.compedu.2014.02.006"},{"key":"e_1_2_1_30_1","volume-title":"Proceedings of the 4th International Conference on Conceptions of Library and Information Science (CoLIS4)","author":"Cool C.","unstructured":"C. Cool and N. J. Belkin . 2002. A classification of interactions with information. In Emerging Frameworks and Methods , Proceedings of the 4th International Conference on Conceptions of Library and Information Science (CoLIS4) . Greenwood Village, CO, Libraries Unlimited, 1--15. C. Cool and N. J. Belkin. 2002. A classification of interactions with information. In Emerging Frameworks and Methods, Proceedings of the 4th International Conference on Conceptions of Library and Information Science (CoLIS4). Greenwood Village, CO, Libraries Unlimited, 1--15."},{"key":"e_1_2_1_31_1","doi-asserted-by":"publisher","DOI":"10.1177\/1098214010389321"},{"key":"e_1_2_1_32_1","doi-asserted-by":"publisher","DOI":"10.1016\/S0306-4573(99)00072-2"},{"key":"e_1_2_1_33_1","unstructured":"A. Dillon and D. Turnbull. 2005. Information architecture. In Encyclopedia of Library and Information Sciences (3rd ed.). 1 1 (2005) 2361--2368.  A. Dillon and D. Turnbull. 2005. Information architecture. In Encyclopedia of Library and Information Sciences (3rd ed.). 1 1 (2005) 2361--2368."},{"key":"e_1_2_1_34_1","doi-asserted-by":"publisher","DOI":"10.1037\/0003-066X.34.10.906"},{"key":"e_1_2_1_35_1","doi-asserted-by":"publisher","DOI":"10.2307\/356600"},{"key":"e_1_2_1_36_1","doi-asserted-by":"publisher","DOI":"10.1145\/1076034.1076110"},{"key":"e_1_2_1_37_1","unstructured":"S. Goldman K. Lawless and F. Manning. 2013. Research and development of multiple source comprehension assessment. In Reading - From Words to Multiple Texts M. A. Britt S. R. Goldman and J.-F. Rouet (Eds.). Routledge New York NY 180--199.  S. Goldman K. Lawless and F. Manning. 2013. Research and development of multiple source comprehension assessment. In Reading - From Words to Multiple Texts M. A. Britt S. R. Goldman and J.-F. Rouet (Eds.). Routledge New York NY 180--199."},{"key":"e_1_2_1_38_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.ipm.2004.02.004"},{"key":"e_1_2_1_39_1","volume-title":"Task-Based Information Seeking and Retrieval in the Patent Domain: Processes and Relationships. Acta Universitatis Tamperensis: 1631","author":"Hansen P.","unstructured":"P. Hansen . 2011. Task-Based Information Seeking and Retrieval in the Patent Domain: Processes and Relationships. Acta Universitatis Tamperensis: 1631 , Tampere University Press , Tampere, Finland . P. Hansen. 2011. Task-Based Information Seeking and Retrieval in the Patent Domain: Processes and Relationships. Acta Universitatis Tamperensis: 1631, Tampere University Press, Tampere, Finland."},{"key":"e_1_2_1_40_1","doi-asserted-by":"publisher","DOI":"10.1108\/00220410510585205"},{"key":"e_1_2_1_41_1","volume-title":"Fast surfing for availability or deep diving into quality\u2014motivation and information seeking among middle and high school students. Inform. Res. 11, 4","author":"Heinstr\u00f6m J.","year":"2006","unstructured":"J. Heinstr\u00f6m . 2006. Fast surfing for availability or deep diving into quality\u2014motivation and information seeking among middle and high school students. Inform. Res. 11, 4 ( 2006 ), Article 433. J. Heinstr\u00f6m. 2006. Fast surfing for availability or deep diving into quality\u2014motivation and information seeking among middle and high school students. Inform. Res. 11, 4 (2006), Article 433."},{"key":"e_1_2_1_42_1","doi-asserted-by":"publisher","DOI":"10.1002\/(SICI)1097-4571(199404)45:3%3C201::AID-ASI9%3E3.0.CO;2-W"},{"key":"e_1_2_1_43_1","doi-asserted-by":"publisher","DOI":"10.1002\/(SICI)1097-4571(199601)47:1%3C50::AID-ASI5%3E3.3.CO;2-Y"},{"key":"e_1_2_1_44_1","doi-asserted-by":"publisher","DOI":"10.1558\/wap.v2i2.177"},{"key":"e_1_2_1_45_1","doi-asserted-by":"publisher","DOI":"10.1145\/1840784.1840788"},{"key":"e_1_2_1_46_1","volume-title":"The Turn: Integration of Information Seeking and Retrieval in Context","author":"Ingwersen P.","year":"2005","unstructured":"P. Ingwersen and K. J\u00e4rvelin . 2005 . The Turn: Integration of Information Seeking and Retrieval in Context . Springer, Dordrecht , The Netherlands . P. Ingwersen and K. J\u00e4rvelin. 2005. The Turn: Integration of Information Seeking and Retrieval in Context. Springer, Dordrecht, The Netherlands."},{"key":"e_1_2_1_47_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-642-04417-5_28"},{"key":"e_1_2_1_48_1","volume-title":"Evaluation","author":"J\u00e4rvelin K.","unstructured":"K. J\u00e4rvelin . 2011. Evaluation . In Interactive Information Seeking, Behaviour and Retrieval, I. Ruthven and D. Kelly (Eds.). Facet Publishing , London, UK , 113--138. K. J\u00e4rvelin. 2011. Evaluation. In Interactive Information Seeking, Behaviour and Retrieval, I. Ruthven and D. Kelly (Eds.). Facet Publishing, London, UK, 113--138."},{"key":"e_1_2_1_49_1","first-page":"12","article-title":"Using metacognitive theories to design user interfaces for computer-based learning","volume":"35","author":"Jones M. G.","year":"1995","unstructured":"M. G. Jones , J. D. Farquhar , and D. D. Surry . 1995 . Using metacognitive theories to design user interfaces for computer-based learning . Educational Technol. 35 , 4 (1995), 12 -- 22 . M. G. Jones, J. D. Farquhar, and D. D. Surry. 1995. Using metacognitive theories to design user interfaces for computer-based learning. Educational Technol. 35, 4 (1995), 12--22.","journal-title":"Educational Technol."},{"key":"e_1_2_1_50_1","doi-asserted-by":"publisher","DOI":"10.1145\/2637002.2637047"},{"key":"e_1_2_1_51_1","volume-title":"Proceedings of the 4th CoLIS Conference. Libraries Unlimited","author":"Kek\u00e4l\u00e4inen J.","unstructured":"J. Kek\u00e4l\u00e4inen and K. J\u00e4rvelin . 2002. Evaluating information retrieval systems under the challenges of interaction and multi-dimensional dynamic relevance . In Proceedings of the 4th CoLIS Conference. Libraries Unlimited , Greenwood Village, CO, 253--270. J. Kek\u00e4l\u00e4inen and K. J\u00e4rvelin. 2002. Evaluating information retrieval systems under the challenges of interaction and multi-dimensional dynamic relevance. In Proceedings of the 4th CoLIS Conference. Libraries Unlimited, Greenwood Village, CO, 253--270."},{"key":"e_1_2_1_52_1","doi-asserted-by":"publisher","DOI":"10.1177\/1086296X12457166"},{"key":"e_1_2_1_53_1","doi-asserted-by":"publisher","DOI":"10.1145\/1840784.1840839"},{"key":"e_1_2_1_54_1","doi-asserted-by":"publisher","DOI":"10.1207\/s15430421tip4104_2"},{"key":"e_1_2_1_55_1","doi-asserted-by":"publisher","DOI":"10.1002\/(SICI)1097-4571(199106)42:5<361::AID-ASI6>3.0.CO;2-#"},{"key":"e_1_2_1_56_1","unstructured":"C. Kuhlthau. 1993. Seeking Meaning. Ablex Norwood NJ.  C. Kuhlthau. 1993. Seeking Meaning. Ablex Norwood NJ."},{"key":"e_1_2_1_57_1","doi-asserted-by":"publisher","DOI":"10.1002\/(SICI)1097-4571(1999)50:5%3C399::AID-ASI3%3E3.0.CO;2-L"},{"key":"e_1_2_1_58_1","doi-asserted-by":"publisher","DOI":"10.1108\/EUM0000000007076"},{"key":"e_1_2_1_60_1","doi-asserted-by":"publisher","DOI":"10.1108\/JD-06-2013-0082"},{"key":"e_1_2_1_61_1","doi-asserted-by":"publisher","DOI":"10.1145\/1840784.1840800"},{"key":"e_1_2_1_62_1","doi-asserted-by":"publisher","DOI":"10.1002\/asi.21672"},{"key":"e_1_2_1_63_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.ipm.2008.07.005"},{"key":"e_1_2_1_64_1","doi-asserted-by":"publisher","DOI":"10.1002\/asi.v61:9"},{"key":"e_1_2_1_65_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.jslw.2012.03.002"},{"key":"e_1_2_1_66_1","doi-asserted-by":"publisher","DOI":"10.1002\/meet.14504901134"},{"key":"e_1_2_1_67_1","doi-asserted-by":"publisher","DOI":"10.1145\/1935826.1935875"},{"key":"e_1_2_1_68_1","doi-asserted-by":"publisher","DOI":"10.1145\/1121949.1121979"},{"key":"e_1_2_1_69_1","doi-asserted-by":"publisher","DOI":"10.1145\/1164820.1164844"},{"key":"e_1_2_1_70_1","volume-title":"Reading and Writing the Electronic Book. Morgan and Claypool","author":"Marshall C. C.","unstructured":"C. C. Marshall . 2009. Reading and Writing the Electronic Book. Morgan and Claypool , San Rafael, CA . C. C. Marshall. 2009. Reading and Writing the Electronic Book. Morgan and Claypool, San Rafael, CA."},{"key":"e_1_2_1_71_1","doi-asserted-by":"publisher","DOI":"10.1007\/BF03178760"},{"key":"e_1_2_1_72_1","doi-asserted-by":"publisher","DOI":"10.1177\/1086296X12450445"},{"key":"e_1_2_1_73_1","doi-asserted-by":"publisher","DOI":"10.1145\/1118178.1118182"},{"key":"e_1_2_1_74_1","doi-asserted-by":"publisher","DOI":"10.1145\/2009916.2009966"},{"key":"e_1_2_1_75_1","doi-asserted-by":"publisher","DOI":"10.1145\/1031607.1031626"},{"key":"e_1_2_1_76_1","volume-title":"Proceedings of the 64th Annual Conference of the American Society for Information Science and Technology (ASIST\u201901)","author":"Oard D. W.","unstructured":"D. W. Oard and J. Kim . 2001. Modeling information content using observable behavior . In Proceedings of the 64th Annual Conference of the American Society for Information Science and Technology (ASIST\u201901) . 481--488. D. W. Oard and J. Kim. 2001. Modeling information content using observable behavior. In Proceedings of the 64th Annual Conference of the American Society for Information Science and Technology (ASIST\u201901). 481--488."},{"key":"e_1_2_1_77_1","doi-asserted-by":"crossref","unstructured":"J. Pearson G. Buchanan and H. Thimbleby. 2013. Designing for Digital Reading. Morgan and Claypool San Rafael CA.  J. Pearson G. Buchanan and H. Thimbleby. 2013. Designing for Digital Reading. Morgan and Claypool San Rafael CA.","DOI":"10.1007\/978-3-031-02330-9"},{"key":"e_1_2_1_78_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.ipm.2007.09.006"},{"key":"e_1_2_1_79_1","doi-asserted-by":"publisher","DOI":"10.1002\/ev.1177"},{"key":"e_1_2_1_80_1","volume-title":"Evaluation: A Systematic Approach","author":"Rossi P. H.","year":"2004","unstructured":"P. H. Rossi , M. W. Lipsey , and H. Freeman . 2004 . Evaluation: A Systematic Approach . Sage, Thousand Oaks , CA. P. H. Rossi, M. W. Lipsey, and H. Freeman. 2004. Evaluation: A Systematic Approach. Sage, Thousand Oaks, CA."},{"key":"e_1_2_1_81_1","volume-title":"The Skills of Document Use: From Text Comprehension to Web-Based Learning","author":"Rouet J.-F.","unstructured":"J.-F. Rouet . 2006. The Skills of Document Use: From Text Comprehension to Web-Based Learning . Erlbaum , Mahwah, NJ . J.-F. Rouet. 2006. The Skills of Document Use: From Text Comprehension to Web-Based Learning. Erlbaum, Mahwah, NJ."},{"key":"e_1_2_1_82_1","doi-asserted-by":"publisher","DOI":"10.1002\/aris.144.v42:1"},{"key":"e_1_2_1_83_1","doi-asserted-by":"publisher","DOI":"10.1145\/2362724.2362747"},{"key":"e_1_2_1_84_1","doi-asserted-by":"publisher","DOI":"10.1145\/2362724.2362759"},{"key":"e_1_2_1_85_1","doi-asserted-by":"publisher","DOI":"10.1145\/2484028.2484031"},{"key":"e_1_2_1_86_1","doi-asserted-by":"publisher","DOI":"10.1561\/1500000009"},{"key":"e_1_2_1_87_1","doi-asserted-by":"publisher","DOI":"10.1002\/asi.4630260604"},{"key":"e_1_2_1_88_1","first-page":"3","article-title":"Modeling interaction in information retrieval (IR): A review and proposal","volume":"33","author":"Saracevic T.","year":"1996","unstructured":"T. Saracevic . 1996 a. Modeling interaction in information retrieval (IR): A review and proposal . Proceedings of the American Society for Information Science 33 (1996), 3 -- 9 . T. Saracevic. 1996a. Modeling interaction in information retrieval (IR): A review and proposal. Proceedings of the American Society for Information Science 33 (1996), 3--9.","journal-title":"Proceedings of the American Society for Information Science"},{"key":"e_1_2_1_89_1","volume-title":"Proceedings of the 2nd Conference on Conceptions of Library and Information Science. The Royal School of Librarianship, Copenhagen, DK, 201--218","author":"Saracevic T.","year":"1996","unstructured":"T. Saracevic . 1996 b. Relevance reconsidered \u201996. In Information science: Integration in Perspectives . Proceedings of the 2nd Conference on Conceptions of Library and Information Science. The Royal School of Librarianship, Copenhagen, DK, 201--218 . T. Saracevic. 1996b. Relevance reconsidered \u201996. In Information science: Integration in Perspectives. Proceedings of the 2nd Conference on Conceptions of Library and Information Science. The Royal School of Librarianship, Copenhagen, DK, 201--218."},{"key":"e_1_2_1_90_1","doi-asserted-by":"publisher","DOI":"10.1023\/A:1003044231033"},{"key":"e_1_2_1_91_1","volume-title":"Evaluation Thesaurus","author":"Scriven M.","unstructured":"M. Scriven . 1991. Evaluation Thesaurus ( 4 th ed.). Sage , Newbury Park, CA . M. Scriven. 1991. Evaluation Thesaurus (4th ed.). Sage, Newbury Park, CA.","edition":"4"},{"key":"e_1_2_1_92_1","doi-asserted-by":"publisher","DOI":"10.1023\/B:ESLL.0000033847.00732.af"},{"key":"e_1_2_1_93_1","volume-title":"Collaborative Information Seeking: The Art and Science of Making the Whole Greater Than the Sum of All","author":"Shah C.","unstructured":"C. Shah . 2012. Collaborative Information Seeking: The Art and Science of Making the Whole Greater Than the Sum of All . Springer, Heidelberg , Germany . C. Shah. 2012. Collaborative Information Seeking: The Art and Science of Making the Whole Greater Than the Sum of All. Springer, Heidelberg, Germany."},{"key":"e_1_2_1_94_1","doi-asserted-by":"publisher","DOI":"10.1145\/1390334.1390362"},{"key":"e_1_2_1_95_1","unstructured":"E. Sormunen and L. Lehti\u00f6. 2011. Authoring Wikipedia articles as an information literacy assignment -- copy-pasting or expressing new understanding in one's own words&quest; Inf. Res. 16 4 (2011) Article 503.  E. Sormunen and L. Lehti\u00f6. 2011. Authoring Wikipedia articles as an information literacy assignment -- copy-pasting or expressing new understanding in one's own words&quest; Inf. Res. 16 4 (2011) Article 503."},{"key":"e_1_2_1_96_1","unstructured":"E. Sormunen J. Heinstr\u00f6m L. Romu and R. Turunen. 2012. A method for the analysis of information use in source-based writing. Inf. Res. 17 4 (2012) Article 535.  E. Sormunen J. Heinstr\u00f6m L. Romu and R. Turunen. 2012. A method for the analysis of information use in source-based writing. Inf. Res. 17 4 (2012) Article 535."},{"key":"e_1_2_1_97_1","doi-asserted-by":"publisher","DOI":"10.1002\/asi.23032"},{"key":"e_1_2_1_98_1","volume-title":"The Constructivist Metaphor. Reading, Writing, and the Making of Meaning","author":"N. N.","unstructured":"N. N. Spivey 1997. The Constructivist Metaphor. Reading, Writing, and the Making of Meaning . Academic Press, San Diego , CA. N. N. Spivey 1997. The Constructivist Metaphor. Reading, Writing, and the Making of Meaning. Academic Press, San Diego, CA."},{"key":"e_1_2_1_99_1","volume-title":"Plans and Situated Actions: The Problem of Human-Machine Communication","author":"Suchman L.","unstructured":"L. Suchman . 1987. Plans and Situated Actions: The Problem of Human-Machine Communication . Cambridge University Press , New York . L. Suchman. 1987. Plans and Situated Actions: The Problem of Human-Machine Communication. Cambridge University Press, New York."},{"key":"e_1_2_1_100_1","doi-asserted-by":"publisher","DOI":"10.1086\/620653"},{"key":"e_1_2_1_101_1","doi-asserted-by":"publisher","DOI":"10.1145\/62437.62477"},{"key":"e_1_2_1_102_1","doi-asserted-by":"publisher","DOI":"10.1108\/00220410810912442"},{"key":"e_1_2_1_103_1","volume-title":"From information to knowledge: Charting and measuring changes in students","author":"Todd R.","year":"2006","unstructured":"R. Todd . 2006. From information to knowledge: Charting and measuring changes in students \u2019 knowledge of a curriculum topic. Inf. Res. 11, 4 ( 2006 ), Article 264. R. Todd. 2006. From information to knowledge: Charting and measuring changes in students\u2019 knowledge of a curriculum topic. Inf. Res. 11, 4 (2006), Article 264."},{"key":"e_1_2_1_104_1","doi-asserted-by":"publisher","DOI":"10.1145\/1148170.1148176"},{"key":"e_1_2_1_105_1","doi-asserted-by":"publisher","DOI":"10.1016\/S0306-4573(99)00028-X"},{"key":"e_1_2_1_106_1","doi-asserted-by":"publisher","DOI":"10.1145\/345508.345512"},{"key":"e_1_2_1_107_1","doi-asserted-by":"publisher","DOI":"10.1108\/EUM0000000007075"},{"key":"e_1_2_1_108_1","doi-asserted-by":"crossref","first-page":"3","DOI":"10.1023\/A:1016089224008","article-title":"Changes in search tactics and relevance judgments in preparing a research proposal: A summary of findings of a longitudinal study","volume":"4","author":"Vakkari P.","year":"2001","unstructured":"P. Vakkari . 2001 b. Changes in search tactics and relevance judgments in preparing a research proposal: A summary of findings of a longitudinal study . Inf. Retrieval 4 , 3 -- 4 (2001), 295--310. P. Vakkari. 2001b. Changes in search tactics and relevance judgments in preparing a research proposal: A summary of findings of a longitudinal study. Inf. Retrieval 4, 3--4 (2001), 295--310.","journal-title":"Inf. Retrieval"},{"key":"e_1_2_1_109_1","article-title":"Task based information searching. Annu","volume":"37","author":"Vakkari P.","year":"2003","unstructured":"P. Vakkari . 2003 . Task based information searching. Annu . Rev. Inform. Sci. 37 , 1 (2003). Information Today, Medford, NJ. P. Vakkari. 2003. Task based information searching. Annu. Rev. Inform. Sci. 37, 1 (2003). Information Today, Medford, NJ.","journal-title":"Rev. Inform. Sci."},{"key":"e_1_2_1_110_1","volume-title":"Proceedings of 4th Workshop on Human-Computer Interaction and Information Retrieval","author":"Vakkari P.","year":"2010","unstructured":"P. Vakkari . 2010 . Exploratory searching as conceptual exploration . In Proceedings of 4th Workshop on Human-Computer Interaction and Information Retrieval . Rutgers University, New Brunswick, NJ, USA, 24--27. P. Vakkari. 2010. Exploratory searching as conceptual exploration. In Proceedings of 4th Workshop on Human-Computer Interaction and Information Retrieval. Rutgers University, New Brunswick, NJ, USA, 24--27."},{"key":"e_1_2_1_111_1","doi-asserted-by":"publisher","DOI":"10.1108\/EUM0000000007127"},{"key":"e_1_2_1_112_1","doi-asserted-by":"publisher","DOI":"10.1002\/asi.21683"},{"key":"e_1_2_1_113_1","doi-asserted-by":"publisher","DOI":"10.1016\/S0306-4573(02)00031-6"},{"key":"e_1_2_1_114_1","doi-asserted-by":"publisher","DOI":"10.1002\/(SICI)1097-4571(1998)49:2%3C115::AID-ASI3%3E3.0.CO;2-1"},{"key":"e_1_2_1_115_1","doi-asserted-by":"publisher","DOI":"10.1002\/asi.10367"},{"key":"e_1_2_1_116_1","doi-asserted-by":"publisher","DOI":"10.1002\/(SICI)1097-4571(199509)46:8%3C590::AID-ASI7%3E3.0.CO;2-#"},{"key":"e_1_2_1_117_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.ipm.2007.11.005"},{"key":"e_1_2_1_118_1","volume-title":"Interactive Information Retrieval in Digital Environments","author":"Xie I.","unstructured":"I. Xie . 2008. Interactive Information Retrieval in Digital Environments . IGI , New York, NY . I. Xie. 2008. Interactive Information Retrieval in Digital Environments. IGI, New York, NY."},{"key":"e_1_2_1_119_1","doi-asserted-by":"publisher","DOI":"10.1007\/s11528-011-0467-y"}],"container-title":["ACM Transactions on Information Systems"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/2699660","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/2699660","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,18]],"date-time":"2025-06-18T06:16:59Z","timestamp":1750227419000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/2699660"}},"subtitle":["The Viewpoint of Program Theory"],"short-title":[],"issued":{"date-parts":[[2015,2,17]]},"references-count":118,"journal-issue":{"issue":"1","published-print":{"date-parts":[[2015,2,17]]}},"alternative-id":["10.1145\/2699660"],"URL":"https:\/\/doi.org\/10.1145\/2699660","relation":{},"ISSN":["1046-8188","1558-2868"],"issn-type":[{"value":"1046-8188","type":"print"},{"value":"1558-2868","type":"electronic"}],"subject":[],"published":{"date-parts":[[2015,2,17]]},"assertion":[{"value":"2014-03-01","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2014-11-01","order":1,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2015-03-17","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}