Abstracts
Résumé
L’indice de détection de patrons de réponses inappropriés lz (Drasgow, Levine & Williams, 1985) a été appliqué à un test d’habileté en lecture en langue seconde de 64 items soumis à 171 étudiants universitaires. L’objectif était de confronter un rejet intuitif de données de recherche à une élimination suggérée par lz. En outre, lz a été mis à l’épreuve pour détecter 12 participants additionnels ayant répondu par pseudo-hasard. Les résultats suggèrent que, bien que lz détecte efficacement des patrons de réponses aberrants pour de grands groupes et qu’il soit préférable à l’élimination intuitive, cet indice présente des limites pour l’analyse de plus petites matrices de données.
Mots-clés :
- indice lz,
- élimination de données,
- technique de vérification de phrases,
- tests SVT
Abstract
With the intent to detect inappropriate response patterns, the lz index (Drasgow, Levine & Williams, 1985) was applied to a 64-item test of second language reading ability administered to 171 university students. Our goal was to compare intuitive rejection of research data to data elimination suggested by lz. In addition, lz was challenged to detect 12 additional participants who had responded by pseudo-chance. Results suggest that although lz detects efficiently aberrant response patterns for large groups and that it proves superior to intuitive rejection, that index has limitations when it comes to analyzing smaller data matrices.
Keywords:
- lz index,
- data elimination,
- sentence verification technique,
- SVT tests
Resumo
O índice de deteção de respostas inapropriadas lz (Drasgow, Levine & Williams, 1985) foi aplicado a um teste de competência de leitura em segunda língua com 64 itens administrado a 171 estudantes universitários. O objetivo era confrontar uma rejeição intuitiva de dados de investigação com uma eliminação sugerida pelo lz. Além disso, lz foi testado para detetar 12 participantes adicionais que responderam de modo pseudo-aleatório. Os resultados sugerem que, apesar de lz detetar eficazmente padrões de respostas aberrantes por grandes grupos e que é preferível à eliminação intuitiva, este índice tem limitações para a análise das matrizes mais pequenas de dados.
Palavras chaves:
- índice lz,
- eliminação de dados,
- técnica de verificação de frase,
- teste SVT
Appendices
Bibliographie
- Bernhardt, E. B. (1991). Reading development in a second language: Theoretical, empirical, and classroom perspectives. Norwood, NJ: Ablex.
- Bertrand, R. & Blais, J.-G. (2004). Modèle de mesure : l’apport de la théorie de la réponse aux items. Sainte-Foy, Québec : Presses de l’Université du Québec.
- Bolger, D. J., Balass, M., Landen, E., & Perfetti, C. A. (2008). Context variation and definitions in learning the meanings of words: An instance-based learning approach. Discourse Processes, 45, 122-159. doi: 10.1080/01638530701792826
- Borghi A., Glenberg A., & Kaschak, M. (2004). Putting words in perspective. Memory and Cognition, 32, 863-873. Retrieved from http://scalab.cnrs.fr/CNCC09/PuttingWordsInPerspective.pdf
- Brassard, P. D. (2011). Identification des stratégies de sous-classement intentionnel aux tests de classement en anglais, langue seconde, au collégial (Mémoire de maîtrise non publié). Montréal, Québec : Université du Québec à Montréal. Récupéré de http://www.archipel.uqam.ca/4275/
- Cohen, A. D. (1992-1993). Test-taking strategies on language tests. Journal of English and Foreign Languages, 10-11, 90-105.
- Cronbach, L. J. (1946). Response sets and test validity. Educational and Psychological Measurement, 6, 475-494. doi: 10.1177/001316444600600405
- Dodeen, H., & Darabi, M. (2009). Person-fit: Relationship with four personality tests in mathematics. Research Papers in Education, 24, 115-126.
- Drasgow, F., & Levine, M. V. (1986). Optimal detection of certain forms of inappropriate test scores. Applied Psychological Measurement, 10(1), 59-67. doi: 10.1177/014662168601000105
- Drasgow, F., Levine, M. V., & Williams, E. A. (1985). Appropriateness measurement with polychotomous item response models and standardized indices. British Journal of Mathematical and Statistical Psychology, 38, 67-86. doi: 10.1111/j.2044-8317.1985.tb00817.x
- Giasson, J. (2007). La compréhension en lecture. Paris : De Boeck.
- Glenberg, A. M., Sato, M., Cattaneo, L., Riggio, L., Palumbo, D., & Buccino, G. (2008). Processing abstract language modulates motor system activity. Quarterly Journal of Experimental Psychology, 61, 905-919. doi: 10.1080/17470210701625550
- Grabe, W. (2009). Reading in a second language: Moving from theory to practice. New York, NY: Columbia University Press.
- Guasch, M., Sanchez-Casas, R., Ferre, P., & García-Albea, J. E. (2011). Effects of the degree of meaning similarity on cross-language semantic priming in highly proficient bilinguals. Journal of Cognitive Psychology, 23(8), 942-961. doi: 10.1080/ 20445911. 2011.589382
- Johnson, E. M. (1998). A taxonomy of person misfit on affective measures (Unpublished doctoral dissertation). Denver, CO: University of Denver.
- Karabatsos, G. (2003). Comparing the aberrant response detection performance of thirty-six person-fit statistics. Applied Measurement in Education, 16, 277-298. doi: 10.1207/S15324818AME1604_2
- Levine, M. B., & Rubin, D. B. (1979). Measuring the appropriateness of multiple choice test scores. Journal of Educational Statistics, 4, 269-290. doi: 10.3102/10769986 004004269
- Li, M. F., & Olejnik, S. (1997). The power of Rasch person-fit statistics in detecting unusual response patterns. Applied Psychological Measurement, 21, 215-231. doi: 10.1177/01466216970213002
- Magis, D., Béland, S. & Raîche, G. (2013). Un processus itératif pour réduire l’impact de réponses aberrantes sur l’identification de patrons de réponses inappropriés. Mesure et évaluation en éducation, 36(2), 87-110. doi: 10.7202/1024416ar
- Marchant, H. G., Royer, J. M., & Greene, B. A. (1988). Superior reliability and validity for a new form of the Sentence Verification Technique for measuring comprehension. Educational and Psychological Measurement, 48, 827-834. doi: 10.1177/0013164488483032
- Meijer, R. R. (1996). Person-fit research: An introduction. Applied Measurement in Education, 9(1), 3-8. doi: 10.1207/s15324818ame0901_2
- Meijer, R. R. (2003). Diagnosing item score pattern on a test using item response theory-based person-fit statistics. Psychological Methods, 8(1), 72-87. doi: 10.1037/1082-989X.8.1.72
- Nering, M. L. (1997). The distribution of indexes of person fit within the computerized adaptive testing environment. Applied PsychologicalMeasurement, 21, 115-127. doi: 10.1177/01466216970212002
- Nering, M. L., & Meijer, R. R. (1998). A comparison of the person response function and the lz person fit statistic. Applied Psychological Measurement, 22, 53-69. doi: 10.1177/01466216980221004
- Partchev, I. (2011). Irtoys: Simple interface to the estimation and plotting of IRT models. R package (version 0.1.4). Retrieved from http://cRan.R-project.org/package=irtoys
- Pichette, F., Béland, S., Jolani, S., & Leśniewska, J. (2015). The handling of missing binary data in language research. Studies in Second Language Learning and Teaching, 5(1), 153-172. doi: http://dx.doi.org/10.14746/ssllt.2015.5.1.8
- Pichette, F., Lafontaine, M., & de Serres, L. (2009). A new tool for measuring L2 reading comprehension ability. Paper presented at the 20th EuroSLA Conference, Cork, Ireland.
- Pothos, E. M., Chater, N., & Ziori, E. (2006). Does stimulus appearance affect learning? American Journal of Psychology, 119(2), 277-301. Retrieved from http://www.dectech.co.uk/publications/LinksNick/CategorizationPerceptionAndMemory/Does%20stimulus%20appearance%20affect%20learning.pdf
- Raîche, G. (2002). Le dépistage de sous-classement aux tests de classement en anglais, langue seconde, au collégial. Gatineau, Québec : Collège de l’Outaouais.
- Raîche, G., Magis, D., Béland, S. & Blais, J.-G. (2011). Conditions d’efficacité de la détection des patrons de réponses inappropriés lors de l’administration d’épreuves adaptatives. Dans J.-G. Blais & J.-L. Gilles (dir.), Évaluation des apprentissages et technologies de l’information et de la communication : le futur est à notre porte (pp. 339-354). Québec, Québec : Presses de l’Université Laval.
- Raîche, G., Magis, D., Blais, J.-G., & Brochu, P. (2013). Taking atypical response pattern into account: A multidimensional measurement model from item response theory. In M. Simon, K. Ercikan & M. Rousseau (Eds.), Improving large-scale education assessment (pp. 238-259). New York, NY: Taylor & Francis.
- Reise, S. P., & Due, A. M. (1991). Test characteristics and their influence on the detection of aberrant response patterns. Applied Psychological Measurement, 15, 217-226. doi: 10.1177/014662169101500301
- Reise, S. P., & Flannerey, W. P. (1996). Assessing person-fit on measures of typical performance. Applied Measurement in Education, 9(1), 9-26. doi: 10.1207/s15324818ame0901_3
- Ro, S. (2001). Characteristics of a likelihood-based person-fit index under the graded response model (Unpublished doctoral dissertation). University of Minnesota, Minneapolis, MN.
- Royer, J. M. (2004). Uses for the Sentence Verification Technique for measuring language comprehension. Amherst, MA: Reading Success Lab. Retrieved from http://www.readingsuccesslab.com/publications/Svt%20Review%20PDF%20version.pdf
- Royer, J. M., Hastings, C. N., & Hook, C. (1979). A sentence verification technique for measuring reading comprehension. Journal of Reading Behavior, 11, 355-363.
- Scharnagl, T. L. (2005). The effects of test-taking strategies on students’ reading achievement. (Unpublished doctoral dissertation). University of Michigan, Ann Arbor, MI.
- Snijders, T. A. B. (2001). Asymptotic null distribution of person fit statistics with estimated person parameter. Psychometrika, 66(3), 331-342. doi: 10.1007/BF02294437
- Yanguas, I. (2009). Multimedia glosses and their effect on L2 text comprehension and vocabulary learning. Language Learning & Technology, 13(2), 48-67. Retrieved from http://llt.msu.edu/vol13num2/yanguas.pdf
10.7202/1024416ar