Résumés
Abstract
Despite the importance of replication research in scientific fields, very few replications are conducted in applied linguistics (AL). To enhance language researchers’ awareness of replications and provide a systematic evaluation of current replications, this study analyzed replication studies published in 92 AL leading journals from 1970 to 2021 based on five themes of replication labels, methodological orientations, research trends, authorship, and citation counts of replicators. The results reveal that replication labels have explicitly been mentioned since 2002, the replication of quantitative studies has predominately been raised, studies on second language acquisition were frequently replicated, collaborative authorship has increased in replications, and influential AL scholars tend to conduct replication research. The study highlights the need for a well-established replication classification and calls for replication research in the areas and methodological orientations marginalized in AL. It is also recommended that prominent figures perform more replication research to consolidate its status in AL.
Keywords:
- Applied Linguistics,
- methodological orientation,
- replication studies,
- replication labels,
- systematic review,
- research trends
Résumé
Malgré l'importance de la recherche en réplication dans les domaines scientifiques, très peu de réplications sont menées en linguistique appliquée (LA). Afin de sensibiliser les chercheurs en linguistique à la nécessité des réplications et de fournir une évaluation systématique des réplications actuelles, cette étude a analysé les études de réplication publiées dans 92 revues de premier plan en LA de 1970 à 2021, en se basant sur cinq thèmes : les étiquettes de réplication, les orientations méthodologiques, les tendances de recherche, l'auteurship et les comptes de citations des réplicateurs. Les résultats révèlent que les étiquettes de réplication ont été explicitement mentionnées depuis 2002, que les réplications d'études quantitatives ont été principalement abordées, que les études sur l'acquisition d'une langue seconde ont été fréquemment répliquées, que la collaboration entre auteurs a augmenté dans les réplications, et que les chercheurs influents en LA ont tendance à mener des recherches en réplication. Cette étude met en évidence la nécessité d'un cadre bien établi avec des descriptions détaillées des différentes étiquettes de réplication, servant de référence pour les études de réplication, et appelle à la recherche en réplication dans les domaines et orientations méthodologiques marginalisés en LA. Il est également recommandé que des personnalités éminentes effectuent davantage de recherches en réplication pour consolider son statut en LA.
Veuillez télécharger l’article en PDF pour le lire.
Télécharger
Parties annexes
Bibliography
- Abbuhl, R. (2012a). Practical methods for teaching replication to applied linguistics studies. In G. Porte (Ed.), Replication research in applied linguistics (pp. 135-150). Cambridge University Press.
- Abbuhl, R. (2012b). Why, when, and how to replicate research. In A. Mackey & S. Gass (Eds.), Research methods in second language acquisition (pp. 296-312). Wiley-Blackwell. https://doi.org/10.1002/9781444347340.ch15
- Abbuhl, R. (2018). Replication research. In A. Phakiti, A., P. De Costa, L. Plonsky, & S. Starfield (Eds.), The Palgrave handbook of applied linguistics research methodology (pp. 145-162). Palgrave MacMillan.
- Allwright, D. (2003). Exploratory practice: Rethinking practitioner research in language teaching. Language Teaching Research, 7(2), 113-141. https://doi.org/10.1191/1362168803lr118oa
- Amjad, T., Daud, A., & Aljohani, N. R. (2018). Ranking authors in academic social networks: A survey. Library Hi Tech, 36(1), 97-128. https://doi.org/10.1108/LHT-05-2017-0090
- Bochynska, A., Keeble, L., Halfacre, C. A. E., Casillas, J. V., Champagne, I., Chen, K., Röthlisberger, M., Buchanan, E. M., Roettger, T. B. (2023). Reproducible research practices and transparency across linguistics. MetaArXiv. https://doi.org/10.31222/osf.io/rcews
- Borry, P., Schotsmans, P., & Dierickx, K. (2006). Author, contributor or just a signer? A quantitative analysis of authorship trends in the field of bioethics. Bioethics, 20(4), 213-220. https://doi.org/10.1111/j.1467-8519.2006.00496.x
- Casanave, C. (2012). Heading in the wrong direction? A response to Porte and Richards. Journal of Second Language Writing, 21(3), 296-297. https://doi.org/1060374312000422
- Coretta, S., Casillas, J. V., Roessig, S., Franke, M., Ahn, B., Al-Hoorie, A. H., Al-Tamimi, J., Alotaibi, N. E., AlShakhori, M. K., Altmiller, R. M., Arantes, P., Athanasopoulou, A., Baese-Berk, M. M., Bailey, G., Sangma, Ch. B. A., Beier, E. J., Benavides, G. M., Benker, N., BensonMeyer, E. P., … & Roettger, T. B. (2023). Multidimensional signals and analytic flexibility: Estimating degrees of freedom in human speech analyses. Advances in Methods and Practices in Psychological Science, 6(2). https://doi.org/10.1177/25152459231162567
- Creswell, J. W. (2014). Research design. qualitative, quantitative and mixed method approaches (4th ed.). SAGE Publications.
- Del Ben, R., Brouillard, M., Gonzalez-Barrero, A.M., Killam, H., Kremin, L.V., Quirk, E., Sander-Montan, A., Schott., E., Tsui, R., & Byers-Heinlein, K. (2022). How open science can benefit bilingualism research: A lesson in six tales. Bilingualism: Language and Cognition,25(5), 913-920. https://doi.org/10.1017/S1366728922000256
- Denscombe, M. (2008). Communities of practice: A research paradigm for the mixed methods approach. Journal of Mixed Methods Research, 2(3), 270-283. https://doi.org/10.1177/1558689808316807
- Egbert, J. (2007). Quality analysis of journals in TESOL and applied linguistics. TESOL Quarterly, 41(1), 157-171. https://www.jstor.org/stable/40264335
- Exaly (2023). Applied Linguistics: Top Authors. Retrieved on May 9, 2023, from https://exaly.com/journal/19954/applied-linguistics/top-authors
- Gatbonton, E. (2000). Investigating experienced ESL teachers' pedagogical knowledge. Canadian Modern Language Review, 56(4), 585-616. https://doi.org/10.3138/cmlr.56.4.585
- Golden, M. (1995). Replication and non-quantitative research. Political Science and Politics, 28(3), 481-483. https://doi.org/10.2307/420313
- Hashemi, M. R., & Babaii, E. (2013). Mixed methods research: Toward new research designs in applied linguistics. The Modern Language Journal, 97(4), 828-852. https://doi.org/10.1111/j.1540-4781.2013.12049.x
- Heirene, R. M. (2021). A call for replications of addiction research: Which studies should we replicate and what constitutes a ‘successful’ replication? Addiction Research & Theory, 29(2), 89-97. https://doi.org/10.1080/16066359.2020.1751130
- Held, L., Micheloud, C., & Pawel, S. (2022). The assessment of replication success based on relative effect size. The Annals of Applied Statistics, 16(2), 706-720. https://doi.org/10.1214/21-AOAS1502
- Henriksen, D. (2018). What factors are associated with increasing co-authorship in the social sciences? A case study of Danish economics and political science. Scientometrics, 114(3), 1395-1421. https://doi.org/10.1007/s11192-017-2635-0
- Hiver, P., Al-Hoorie, A. H., Vitta, J. P., & Wu, J. (2021). Engagement in language learning: A systematic review of 20 years of research methods and definitions. Language Teaching Research, 28(1), 201-230. https://doi.org/10.1177/13621688211001289
- Hyland, K. (2015). Academic publishing: Issues and challenges in the construction of knowledge. Oxford University Press.
- Ioannidis, J. P., & Trikalinos, T. A. (2007). An exploratory test for an excess of significant findings. Clinical Trials, 4(3), 245-253. https://doi.org/10.1177%2F1740774507079441
- Irvine, E. (2021). The role of replication studies in theory building. Perspectives on Psychological Science, 16(4), 844-853. https://doi.org/10.1177/1745691620970558
- Isbell, D. R., Brown, D., Chen, M., Derrick, D. J., Ghanem, R., Gutierrez Arvizu, M. N., Schnur, E., Zhang, M., & Plonsky, L. (2022). Misconduct and questionable research practices: The ethics of quantitative data handling and reporting in applied linguistics. The Modern Language Journal. 106(1), 172-195. https://doi.org/10.1111/modl.12760
- John, L., Loewenstein, G., & Prelec, D. (2012). Measuring the prevalence of questionable research practices with incentives for truth telling. Psychological Science, 23(5), 524-532. https://doi.org/10.1177%2F0956797611430953
- Jordan, G. (2005). Theory construction in second language acquisition. John Benjamins.
- Jung, U. O. H. (2004). Paris in London revisited or the foreign language teacher’s topmost journals. System, 32(3), 357-361.
- Kelly, C. D. (2019). Rate and success of study replication in ecology and evolution. PeerJ, 7. https://doi.org/10.7717/peerj.7654
- Khany, R., & Tazik, K. (2019). Levels of statistical use in applied linguistics research articles: From 1986-2015. Journal of Quantitative Linguistics, 26(1), 48-65. https://doi.org/10.1080/09296174.2017.1421498
- King, K., & Mackey, A. (2016). Research methodology in second language studies: Trends, concerns, and new directions. Modern Language Journal, 100(s1), 209-227. https://doi.org/10.1111/modl.12309
- Kobrock, K., & Roettger, T. B. (2023). Assessing the replication landscape in experimental linguistics. Glossa Psycholinguistics, 2(1). http://dx.doi.org/10.5070/G6011135
- Language Teaching Review Panel. (2008). Replication studies in language learning and teaching: Questions and answers. Language Teaching, 41(1), 1-14. https://doi.org/10.1017/S0261444807004727
- Larson-Hall, J., & Plonsky, L. (2015). Reporting and interpreting quantitative research findings: What gets reported and recommendations for the field. Language Learning, 65(s1), 127-159. https://doi.org/10.1111/lang.12115
- Lykken, D. T. (1968). Statistical significance in psychological research. Psychological Bulletin, 70(3), 151-159. https://psycnet.apa.org/doi/10.1037/h0026141
- Mackey, A., & Marsden, E. (2016). Advancing methodology and practice: The IRIS repository of instruments for research into second languages. Routledge.
- Magnan, S.S. (2006). From the editor: The MLJ turns 90 in a digital age. The Modern Language Journal, 90(1), 1-5. https://www.jstor.org/stable/3588810
- Makel, M. C., Meyer, M. S., Simonsen, M. A., Roberts, A. M., & Plucker, J. A. (2022). Replication is relevant to qualitative research. Educational Research and Evaluation, 27(1-2), 215-219. https://doi.org/10.1080/13803611.2021.2022310
- Makel, M., & Plucker, J. (2014). Facts are more important than novelty: Replication in the education sciences. Educational Researcher, 43(6), 304-316. https://doi.org/10.3102/0013189X14545513
- Makel, M., Plucker, J., & Hegarty, B. (2012). Replications in psychology research: How often do they really occur? Perspectives on Psychological Science, 7(6), 532-542. https://doi.org/10.1177%2F1745691612460688
- Markee, N. (2017). Are replication studies possible in qualitative second/foreign language classroom research? A call for comparative re-production research. Language Teaching, 50(3), 367-383. https://doi.org/10.1017/S0261444815000099
- Marsden, E. J. (2020). Methodological transparency in applied linguistics and its consequences for the quality and scope of research. In J. McKinley, & H. Rose (Eds.), Routledge handbook of research methods in applied linguistics (pp. 15-28). Routledge.
- Marsden, E., & Mackey, A. (2014). IRIS: A new resource for second language research. Linguistic Approaches to Bilingualism, 4(1), 125-130. https://doi.org/10.1075/lab.4.1.05mar
- Marsden, E., Mackey, A., & Plonsky, L. (2016). The IRIS repository: Advancing research practice and methodology. In A. Mackey, & E. Marsden (Eds.), Advancing methodology and practice: The IRIS repository of instruments for research into second languages (pp. 1-21). Routledge. https://doi.org/10.4324/9780203489666
- Marsden, E., & Morgan-Short, K. (2023) (Why) are open research practices the future for the study of language learning? Language Learning, 73(s2), 344-387. https://doi.org/10.1111/lang.12568
- Marsden, E., Morgan-Short, K., Thompson, S., & Abugaber, D. (2018). Replication in second language research: Narrative and systematic reviews and recommendations for the field. Language Learning, 68(2), 321-391. https://doi.org/10.1111/lang.12286
- Matsuda, P. (2012). On the nature of second language writing: Replication in a postmodern field. Journal of Second Language Writing, 21(3), 300-302. http://dx.doi.org/10.1016%2Fj.jslw.2012.05.006
- McHugh, M. L. (2012). Interrater reliability: The kappa statistic. Biochemia Medica, 22(3), 276-282. https://pmc.ncbi.nlm.nih.gov/articles/PMC3900052/
- McManus, K (2021). Are replication studies infrequent because of negative attitudes? Insights from a survey of attitudes and practices in second language research. Studies in Second Language Acquisition, 44(5), 1410-1423 https://doi.org/10.1017/S0272263121000838
- Moreau, D., & Wiebels, K. (2023). Ten simple rules for designing and conducting undergraduate replication projects. PLoS Computer Biology, 19(3). https://doi.org/10.1371/journal.pcbi.1010957
- Mullock, B. (2006). The pedagogical knowledge base of four TESOL teachers. The Modern Language Journal, 90(1), 48-66. https://doi.org/10.1111/j.1540-4781.2006.00384.x
- Nassaji, H. (2012). Significance tests and generalizability of research results: A case for replication. In G. Porte (Ed.), Replication research in applied linguistics (pp. 92-115). Cambridge University Press.
- Nassaji, H. (2021). Effect sizes in quantitative and qualitative research. Language Teaching Research, 25(5), 681-684. https://doi.org/10.1177%2F13621688211040882
- Nosek, B.A., & Errington, T. M. (2020). The best time to argue about what a replication means? Before you do it. Nature, 583, 518-520. https://doi.org/10.1038/d41586-020-02142-6
- Pashler, H., & Harris, C. (2012). Is the replicability crisis overblown? Three arguments examined. Perspectives on Psychological Science, 7(6), 531-536. https://doi.org/10.1177%2F1745691612463401
- Plonsky, L. (2023). Sampling and generalizability in Lx research: A second-order synthesis. Languages, 8(1), 75. https://doi.org/10.3390/languages8010075
- Polio, C. (2012). Replication in published applied linguistics research: A historical perspective. In G. Porte (Ed.), Replication research in applied linguistics (pp. 47-91). Cambridge University Press.
- Polio, C., & Gass, S. (1997). Replication and reporting: A commentary. Studies in Second Language Acquisition, 19(4), 499-508. https://www.jstor.org/stable/44487991
- Popper, K. R. (1959). The logic of scientific discovery. Hutchinson.
- Porte, G. (2012). Replication research in applied linguistics. Cambridge University Press.
- Porte, G., & McManus, K. (2019). Doing replication research in applied linguistics. Routledge. https://doi.org/10.4324/9781315621395
- Porte, G., & Richards, K. (2012). Focus article: Replication in second language writing research. Journal of Second Language Writing, 21(3), 284-293. https://doi.org/10.1016/j.jslw.2012.05.002
- Riazi, A. M., Ghanbar, H., & Fazel, I. (2020). The contexts, theoretical and methodological orientation of EAP research: Evidence from empirical articles published in the Journal of English for Academic Purposes. Journal of English for Academic Purposes, 48. https://doi.org/10.1016/j.jeap.2020.100925
- Robinson, L. M., & Adler, R. D. (2004). Business research in eight business disciplines. Journal of College Teaching & Learning, 1(2), 69-80. https://doi.org/10.19030/tlc.v1i2.1916
- Santos, T. (1989). Replication in applied linguistics research. TESOL Quarterly, 23(4), 699-702. https://doi.org/10.2307/3587548
- Sasaki, M. (2012). An alternative approach to replication studies in second language writing: An ecological perspective. Journal of Second Language Writing, 21(3), 303-305. https://doi.org/10.1016/j.jslw.2012.05.004
- Schmidt, S. (2009). Shall we really do it again? The powerful concept of replication is neglected in the social sciences. Review of General Psychology, 13(2), 90-100. https://doi.org/10.1037/a0015108
- Schofield, J. (2002). Increasing the generalizability of qualitative research. In A. Huberman, & M. Miles (Eds.), The qualitative researcher’s companion (pp. 171-203). SAGE. https://doi.org/10.4135/9781412986274
- Silberzahn, R., Uhlmann, E. L., Martin, D. P., Anselmi, P., Aust, F., Awtrey, E., Bahník, Š., Bai, F., Bannard, C., Bonnier, E., Carlsson, R., Cheung, F., Christensen, G., Clay, R., Craig, M. A., Dalla Rosa, A., Dam, L., Evans, M. H., Flores Cervantes, I., Fong, N., … & Nosek, B. A. (2018). Many analysts, one data set: Making transparent how variations in analytic choices affect results. Advances in Methods and Practices in Psychological Science, 1(3), 337-356. https://doi.org/10.1177/2515245917747646
- Sönning, L., & Werner, V. (2021). The replication crisis, scientific revolutions, and linguistics. Linguistics, 59(5), 1179-1206. https://doi.org/10.1515/ling-2019-0045
- Stapleton, P., & Shao, Q. (2017). Research in language teaching over two decades: A retrospective of the first 20 volumes of Language Teaching Research. Language Teaching Research, 22(3), 350-369. https://doi.org/10.1177/1362168816688929
- Vandergrift, L., & Cross, J. (2017). Replication research in L2 listening comprehension: A conceptual replication of Graham & Macaro (2008) and an approximate replication of Vandergrift & Tafaghodtari (2010) and Brett (1997). Language Teaching, 50(1), 80-89. https://doi.org/10.1017/S026144481500004X
- Weber, M., & Campbell, C. M. (2004). In other professional journals. Modern Language Journal, 88(1), 130-139. https://doi.org/10.1111/0026-7902.00010
- Zou, D., Luo, S., Xie, H., & Hwang, G. (2020). A systematic review of research on flipped language classrooms: Theoretical foundations, learning activities, tools, research topics and findings. Computer Assisted Language Learning, 35(8) 1811-1837. https://doi.org/10.1080/09588221.2020.1839502