{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,10,8]],"date-time":"2025-10-08T22:05:32Z","timestamp":1759961132262},"reference-count":32,"publisher":"Wiley","issue":"1","license":[{"start":{"date-parts":[[2011,10,24]],"date-time":"2011-10-24T00:00:00Z","timestamp":1319414400000},"content-version":"vor","delay-in-days":0,"URL":"http:\/\/onlinelibrary.wiley.com\/termsAndConditions#vor"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["J. Am. Soc. Inf. Sci."],"published-print":{"date-parts":[[2012,1]]},"abstract":"<jats:title>Abstract<\/jats:title><jats:p>Evaluation of machine translation output is an important task. Various human evaluation techniques as well as automatic metrics have been proposed and investigated in the last decade. However, very few evaluation methods take the linguistic aspect into account. In this article, we use an objective evaluation method for machine translation output that classifies all translation errors into one of the five following linguistic levels: orthographic, morphological, lexical, semantic, and syntactic. Linguistic guidelines for the target language are required, and human evaluators use them in to classify the output errors. The experiments are performed on English\u2010to\u2010Catalan and Spanish\u2010to\u2010Catalan translation outputs generated by four different systems: 2 rule\u2010based and 2 statistical. All translations are evaluated using the 3 following methods: a standard human perceptual evaluation method, several widely used automatic metrics, and the human linguistic evaluation. Pearson and Spearman correlation coefficients between the linguistic, perceptual, and automatic results are then calculated, showing that the semantic level correlates significantly with both perceptual evaluation and automatic metrics.<\/jats:p>","DOI":"10.1002\/asi.21674","type":"journal-article","created":{"date-parts":[[2011,10,24]],"date-time":"2011-10-24T14:31:23Z","timestamp":1319466683000},"page":"174-184","source":"Crossref","is-referenced-by-count":14,"title":["Study and correlation analysis of linguistic, perceptual, and automatic machine translation evaluations"],"prefix":"10.1002","volume":"63","author":[{"given":"Mireia","family":"Farre\u00fas","sequence":"first","affiliation":[]},{"given":"Marta R.","family":"Costa\u2010juss\u00e0","sequence":"additional","affiliation":[]},{"given":"Maja Popovi\u0107","family":"Morse","sequence":"additional","affiliation":[]}],"member":"311","published-online":{"date-parts":[[2011,10,24]]},"reference":[{"issue":"4","key":"e_1_2_8_2_1","first-page":"577","article-title":"Machine translation: An introductory guide","volume":"21","author":"Arnold D.","year":"1995","journal-title":"Computational Linguistics"},{"key":"e_1_2_8_3_1","doi-asserted-by":"publisher","DOI":"10.3115\/1117586.1117594"},{"issue":"2","key":"e_1_2_8_4_1","first-page":"263","article-title":"The mathematics of statistical machine translation","volume":"19","author":"Brown P.","year":"1993","journal-title":"Computational Linguistics"},{"key":"e_1_2_8_5_1","doi-asserted-by":"publisher","DOI":"10.3115\/1626431.1626433"},{"key":"e_1_2_8_6_1","first-page":"249","volume-title":"Proceedings of the 11th Conference of the European Chapter of the Association for Computational Linguistics","author":"Callison\u2010Burch C.","year":"2006"},{"key":"e_1_2_8_7_1","doi-asserted-by":"publisher","DOI":"10.1109\/ASRU.2001.1034664"},{"key":"e_1_2_8_8_1","volume-title":"An empirical study of smoothing techniques for language modelling","author":"Chen S.F.","year":"1998"},{"key":"e_1_2_8_9_1","doi-asserted-by":"publisher","DOI":"10.1037\/h0026256"},{"issue":"4","key":"e_1_2_8_10_1","first-page":"670","article-title":"Machine translation: A view from the lexicon","volume":"20","author":"Dorr B.J.","year":"1994","journal-title":"Computational Linguistics"},{"key":"e_1_2_8_11_1","first-page":"36","volume-title":"An introduction to linear regression and correlation","author":"Edwards A.L.","year":"1976"},{"key":"e_1_2_8_12_1","unstructured":"Farr\u00fas M. Costa\u2010juss\u00e0 M. Mari\u00f1o J. &Fonollosa J.A.R.(2010).Linguistic\u2010based evaluation criteria to identify statistical machine translation errors. Proceedings of the 14th Annual Meeting of the European Association for Machine Translation (pp. 167\u2013173). Saint\u2010Rapha\u00ebl France."},{"key":"e_1_2_8_13_1","first-page":"52","volume-title":"Proceedings of the 13th Annual Meeting of the European Association for Machine Translation","author":"Farr\u00fas M.","year":"2009"},{"key":"e_1_2_8_14_1","doi-asserted-by":"crossref","unstructured":"Farr\u00fas M. Costa\u2010juss\u00e0 M. Mari\u00f1o J. Poch M. Hern\u00e1ndez A. Henr\u00edquez C. &Fonollosa J.A.R.(2011).Overcoming statistical machine translation limitations: Error analysis and proposed solutions for the Catalan\u2010Spanish language pair. Language Resources and Evaluation (forthcoming).","DOI":"10.1007\/s10579-011-9137-0"},{"key":"e_1_2_8_15_1","first-page":"65","volume-title":"Proceedings of the Association for Machine Translation in the Americas","author":"Flanagan M.A.","year":"1994"},{"key":"e_1_2_8_16_1","unstructured":"Forcada M.L. Tyers F.M. &Ram\u00edrez G.(2009).The apertium machine translation platform: Five years on. First International Workshop on Free\/Open\u2010Source Rule\u2010Based Machine Translation. Alacant Spain."},{"key":"e_1_2_8_17_1","doi-asserted-by":"publisher","DOI":"10.3115\/1626355.1626393"},{"key":"e_1_2_8_18_1","volume-title":"Machine translation: Past, present, future","author":"Hutchins W.J.","year":"1986"},{"key":"e_1_2_8_19_1","first-page":"127","volume-title":"Proceedings of the Human Language Technology Conference (HLT\u2010NAACL)","author":"Koehn P.","year":"2003"},{"key":"e_1_2_8_20_1","doi-asserted-by":"publisher","DOI":"10.2307\/2529310"},{"key":"e_1_2_8_21_1","first-page":"65","volume-title":"Proceedings of the Second Workshop on Statistical Machine Translation","author":"Lavie A.","year":"2007"},{"key":"e_1_2_8_22_1","doi-asserted-by":"crossref","unstructured":"Lopez A.(2007).A survey of statistical machine translation. A survey of statistical machine translation. (Tech Rep. No. A033664). Maryland University College Park Department of Computer Science.","DOI":"10.21236\/ADA466330"},{"key":"e_1_2_8_23_1","doi-asserted-by":"publisher","DOI":"10.1162\/coli.2006.32.4.527"},{"key":"e_1_2_8_24_1","volume-title":"On the use of information retrieval measures for speech recognition evaluation","author":"McCowan I.","year":"2004"},{"key":"e_1_2_8_25_1","first-page":"160","volume-title":"Proceedings of the 41th Annual Meeting of the Association for Computational Linguistics","author":"Och F.","year":"2003"},{"key":"e_1_2_8_26_1","unstructured":"Olive J.(2005).Global autonomous language exploitation (GALE) DARPA\/IPTO Proposer Information Pamphlet. Retrieved fromhttp:\/\/projects.ldc.upenn.edu\/gale\/"},{"key":"e_1_2_8_27_1","first-page":"311","volume-title":"Proceedings of the 40th Annual Meeting of the Association for Computational Linguistics","author":"Papineni K.","year":"2002"},{"key":"e_1_2_8_28_1","unstructured":"Popovi\u0107 M.(2009).Machine translation: Statistical approach with additional linguistic knowledge. Unpublished doctoral dissertation RWTH Aachen University Aachen Germany."},{"key":"e_1_2_8_29_1","doi-asserted-by":"publisher","DOI":"10.3115\/1626431.1626435"},{"issue":"2","key":"e_1_2_8_30_1","first-page":"117","article-title":"TER\u2010plus\u2010paraphrase, semantic, and alignment enhancements to translation edit rate","volume":"23","author":"Snover M.","year":"2010","journal-title":"Machine Translation"},{"key":"e_1_2_8_31_1","doi-asserted-by":"publisher","DOI":"10.2307\/1412159"},{"key":"e_1_2_8_32_1","first-page":"697","volume-title":"Proceedings of the International Conference on Language Resources and Evaluation (LREC 2006)","author":"Vilar D.","year":"2006"},{"key":"e_1_2_8_33_1","volume-title":"KI\u20102002: Advances in Artificial Intelligence (Vol. 2479, pp. 18\u201332)","author":"Zens R.","year":"2002"}],"container-title":["Journal of the American Society for Information Science and Technology"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/api.wiley.com\/onlinelibrary\/tdm\/v1\/articles\/10.1002%2Fasi.21674","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/onlinelibrary.wiley.com\/doi\/pdf\/10.1002\/asi.21674","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2023,10,11]],"date-time":"2023-10-11T18:28:51Z","timestamp":1697048931000},"score":1,"resource":{"primary":{"URL":"https:\/\/onlinelibrary.wiley.com\/doi\/10.1002\/asi.21674"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2011,10,24]]},"references-count":32,"journal-issue":{"issue":"1","published-print":{"date-parts":[[2012,1]]}},"alternative-id":["10.1002\/asi.21674"],"URL":"https:\/\/doi.org\/10.1002\/asi.21674","archive":["Portico"],"relation":{},"ISSN":["1532-2882","1532-2890"],"issn-type":[{"value":"1532-2882","type":"print"},{"value":"1532-2890","type":"electronic"}],"subject":[],"published":{"date-parts":[[2011,10,24]]}}}