Linguistically Balanced Tool for Reading Skills Assessment in Russian-Speaking Adults
Abstract
Reading impairments substantially reduce quality of life, that is why it is necessary to detect them timely, provide intervention programs, or provide special learning or working environment for people with reading impairments in case the intervention was not effective. The diagnostics of reading impairments and evaluation of intervention efficiency is impossible without reliable and valid assessment tools. Whereas Russian-language tools for reading skills assessment in children are being developed and standardized systematically, there still was a lack of tools for reading skills assessment in Russian-speaking adults.
We conducted a study aiming to construct the linguistically balanced tool for reading skills assessment in adults. We developed a test including two parallel versions of texts with no significant differences in several psycholinguistics parameters. Each text is accompanied with a set of questions that allows to asses both reading skills and written text comprehension skills. The test passes an approbation in a group of 111 Russian-speaking young adults. During the approbation, two versions were proved to be parallel: no significant differences were revealed neither in reading speed nor in number of correct answers to the comprehension questions. Since it was impossible to evaluate the external validity of an instrument due to the absence of another standardized reading test for adults, the comparison with the widely used standardized reading test for children was made, using the most difficult variant of that test.
The paper provides the description of a new test, the results of comparison of parallel versions, the results of external validity evaluation, and the computed cur-off levels for diagnosing reading impairments in Russian-speaking young-adults.
Downloads
References
Alexeeva S., Slioussar N., Chernova D. (2018) StimulStat: A Lexical Database for Russian. Behavior Research Methods, vol. 50, no 6, pp. 2305–2315. https://doi.org/10.3758/s13428-017-0994-3.
Altpeter E.K., Marx T., Nguyen N.X., Naumann A., Trauzettel-Klosinski S. (2015) Measurement of Reading Speed with Standardized Texts: A Comparison of Single Sentences and Paragraphs. Graefe's Archive for Clinical and Experimental Ophthalmology, vol. 253, no 8, pp. 1369–1375. https://doi.org/10.1007/s00417-015-3065-4
Burdyna A.V., Koltuntseva V.E., Nikiforova N.V., Marchenko E.N., Sergeeva O.A., Alekseeva S.V. (2023) Assessment of Word- and Nonword Reading Skills in Russian-Speaking Adolescents Age 14–17 Years. Proceedings of the Conference "Cognitive Science in Moscow: New Researches" (Moscow, 2023, 21–22 June), pp. 558–563 (In Russian).
Catts H.W., Kamhi A.G. (1999) Language and Reading Disabilities. Boston: Allyn and Bacon. Cavalli E., Colé P., Leloup G., Poracchia-George F., Sprenger-Charolles L., El Ahmadi A. (2017) Screening for Dyslexia in French-Speaking University Students: An Evaluation of the Detection Accuracy of the Alouette Test. Journal of Learning Disabilities, vol. 51, no 3, pp. 268–282. https://doi.org/10.1177/0022219417704637
Cavalli E., Colé P., Leloup G., Poracchia-George F., Sprenger-Charolles L., El Ahmadi A. (2017) Screening for Dyslexia in French-Speaking University Students: An Evaluation of the Detection Accuracy of the Alouette Test. Journal of Learning Disabilities, vol. 51, no 3, pp. 268–282. https://doi.org/10.1177/0022219417704637
Cline F., Johnstone C.J., King T.K. (2006) Focus Group Reactions to Three Definitions of Reading (As Originally Developed in Support of NARAP Goal 1). Minneapolis, MN: National Accessible Reading Assessment Projects.
Desrochers A., Glickman V. (2009) Criteria for the Evaluation of Reading Assessment Tools. Encyclopedia of Language and Literacy Development. London, ON: Canadian Language and Literacy Research Network, pp. 1–9.
Dobrego A.S., Petrova T.E. (2016) Perception Patterns of Static and Dynamic Texts: An Experimental Study of Russian. Aktualnye problemy yazykoznaniya, vol. 1, pp. 33–34 (In Russian).
Dorofeeva S.V., Grinʹko I.Ju., Perevoshchikova T.D.., Dragoy O.V. (2021) Constructing of Word and Non-Word Reading Tests for Assessment of Reading Ability of Russian-Speaking Children. Proceedings of the Conference "Cognitive Science in Moscow: New Researches" (Moscow, 2021, 23–24 June), pp. 502–507 (In Russian).
Dorofeeva S. V., Reshetnikova V., Serebryakova M., Goranskaya D., Akhutina T. V., Dragoy O. (2019) Assessing the Validity of the Standardized Assessment of Reading Skills in Russian and Verifying the Relevance of Available Normative Data. The Russian Journal of Cognitive Science, vol. 6, no 1, pp. 4–24.
Driscoll D.P. (2012) Reading Framework for the 2013. National Assessment of Educational Process. Available at: https://www.nagb.gov/content/dam/nagb/en/documents/publications/frameworks/reading/2013-reading-framework.pdf (accessed 5.05.2025)
Filippov K.A. (2003) Linguistics of Text. Saint-Petersburg: St. Petersburg University (In Russian).
Flesch R. (1948) A New Readability Yardstick. Journal of Applied Psychology, vol. 32, no 3, pp. 221–233. https://doi.org/10.1037/h0057532
Harris J.L., Rogers W.A., Qualls C.D. (1998) Written Language Comprehension in Younger and Older Adults. Journal of Speech Language and Hearing Research, vol. 41, no 3, pp. 603–616. https://doi.org/10.1044/jslhr.4103.603
Hasbrouck J.E., Tindal G.A. (2006) Oral Reading Fluency Norms: A Valuable Assessment Tool for Reading Teachers. The Reading Teacher, vol. 59, iss. 7, pp. 636–644. https://doi.org/10.1598/RT.59.7.3
Hasbrouck J.E., Tindal G.A. (1992) Curriculum-Based Oral Reading Fluency Norms for Students in Grades 2 through 5. TEACHING Exceptional Children, vol. 24, iss. 3, pp. 41–44. https://doi.org/10.1177/004005999202400310
Henning G. (1987) A Guide to Language Testing: Development, Evaluation, Research. Cambridge, MA: Newberry House.
Hoover W.A., Gough P.B. (1990) The Simple View of Reading. Reading and Writing, vol. 2, no 2, pp. 127–160. https://doi.org/10.1007/bf00401799
Ivanova M.V., Akinina Yu.S., Soloukhina O.A., Iskra E.V., Buivolova O.V., Chrabaszcz A.V., Stupina E.A., Khudyakova M.V., Akhutina T.V., Dragoy O.V. (2021) The Russian Aphasia Test: The First Comprehensive, Quantitative, Standardized, and Computerized Aphasia Language Battery in Russian. PLOS One, vol. 16, no 11, Article no e0258946. https://doi.org/10.1371/journal.pone.0258946
Johann V.E., Könen T., Karbach J. (2020) The Unique Contribution of Working Memory, Inhibition, Cognitive Flexibility, and Intelligence to Reading Comprehension and Reading Speed. Child Neuropsychology, vol. 26, no 3, pp. 324–344. https://doi.org/10.1080/09297049.2019.1649381
Keenan J.M., Betjemann R.S., Olson R.K. (2008) Reading Comprehension Tests Vary in the Skills They Assess: Differential Dependence on Decoding and Oral Comprehension. Scientific Studies of Reading, vol. 12, no 3, pp. 281–300. https://doi.org/10.1080/10888430802132279
Khamaganova V.M. (2002) Structural, Semantic and Lexical Model of a Text of the "Description" Type: Problems of Semiotics and Ontology (PhD Thesis). Moscow: Moscow Pedagogical State University (In Russian).
Kincaid J.P., Fishburne R.P., Rogers R.L., Chissom B.S. (1975) Derivation of New Readability Formulas (Automated Readability Index, Fog Count, and Flesch Reading Ease Formula) for Navy Enlisted Personnel. Research Branch Report no 8–75. Orlando, FL: Institute for Simulation and Training. https://doi.org/10.21236/ADA006655
Kliegl R., Grabner E., Rolfs M., Engbert R. (2004) Length, Frequency, and Predictability Effects of Words on Eye Movements in Reading. European Journal of Cognitive Psychology, vol. 16, no 1–2, pp. 262–284. https://doi.org/10.1080/09541440340000213
Klingner J.K. (2004) Assessing Reading Comprehension. Assessment for Effective Intervention, vol. 29, no 4, pp. 59–70. https://doi.org/10.1177/073724770402900408
Konur O. (2002) Assessment of Disabled Students in Higher Education: Current Public Policy Issues. Assessment & Evaluation in Higher Education, vol. 27, no 2, pp. 131–152. https://doi.org/10.1080/02602930220128715
Kornev A.N. (1997) Reading and Writing Disorders in Children. Saint-Petersburg: M&M (In Russian).
Kornev A.N., Ishimova O.A. (2010) Method of Diagnostic of Dyslexia in Children. Saint-Petersburg: Peter the Great St. Petersburg Polytechnic University (In Russian).
Laposhina A.N., Lebedeva M.Ju. (2021) Textometr: An Online Tool for Automated Complexity Level Assessment of Texts for Russian Language Learners. Russian Language Studies, vol. 19, no 3, pp. 331–345 (In Russian). https://doi.org/10.22363/2618-8163-2021-19-3-331-345
Laposhina A.N., Lebedeva M.Ju., Berlin Khenis A.A. (2022) Word Frequency and Text Complexity: An Eye-Tracking Study of Young Russian Readers. Russian Journal of Linguistics, vol. 26, no 2, pp. 493–514. https://doi.org/10.22363/2687-0088-30084
Laurinavichyute A.K., Sekerina I.A., Alexeeva S., Bagdasaryan K., Kliegl R. (2019) Russian Sentence Corpus: Benchmark Measures of Eye Movements in Reading in Russian. Behaviour Research Methods, vol. 51, iss. 3, pp. 1161–1178. https://doi.org/10.3758/s13428-018-1051-6
Lezina A.D., Nikonova Ya.E., Cherevik A. S., Shatkovskaya I.G., Alekseeva S.V. (2023) Reading Aloud in Russian Language as Tool for Assessment of Risk of Dyslexia in Adolescents Age 14–17. Proceedings of the Conference "Cognitive Science in Moscow: New Researches" (Moscow, 2023, 21–22 June), pp. 599–603 (In Russian).
Magliano J., Millis K., Ozuru Y., McNamara D. (2007) A Multidimensional Framework to Evaluate Reading Assessment Tools. Reading Comprehension Strategies: Theories, Interventions, and Technologies (ed. D.S. McNamara), New York, NY: Psychology Press, pp. 107–136. https://doi.org/10.4324/9780203810033
Mastitskiy S.E., Shitikov V.K. (2015) Statistical Analysis and Data Visualization via R. Moscow: DMK.
McArthur G., Jones K., Anandakumar T., Larsen L., Castles A., Coltheart, M. (2013) A Test of Everyday Reading Comprehension (TERC). Australian Journal of Learning Difficulties, vol. 18, no 1, pp. 35–85. https://doi.org/10.1080/19404158.2013.779588
Murzin L.N., Štern A.S. (1991) Text and Its Perception. Sverdlovsk: Ural University (In Russian).
Nelson H.E., Willison J.R. (1991) The Revised National Adult Reading Test — Test Manual. Windsor, UK: NFER-Nelson.
Nicolson R.I., Fawcett A.J. (2008) Dyslexia, Learning, and the Brain. Cambridge, MA: MIT. https://doi.org/10.7551/mitpress/9780262140997.001.0001
Ottosen H.F., Bønnerup K.H., Weed E., Parrila R. (2022) Identifying Dyslexia at the University: Assessing Phonological Coding Is Not Enough. Annals of Dyslexia, vol. 72, no 1, pp. 147–170. https://doi.org/10.1007/s11881-021-00247-9
Phillips S.M., Kelly K., Symes L. (2013) Assessment of Learners with Dyslexic-Type Difficulties. Los Angeles, CA: Sage.
Poulsen M., Gravgaard A.K.D. (2016) Who Did What to Whom? The Relationship between Syntactic Aspects of Sentence Comprehension and Text Comprehension. Scientific Studies of Reading, vol. 20, no 4, pp. 325–338. https://doi.org/10.1080/10888438.2016.1180695
Şahi̇n A. (2013) The Effect of Text Types on Reading Comprehension. Mevlana International Journal of Education, vol. 3, no 2, pp. 57–67. http://dx.doi.org/10.13054/mije.13.27.3.2
Share D.L. (2021) Common Misconceptions about the Phonological Deficit Theory of Dyslexia. Brain Sciences, vol. 11, no 11, Article no 1510. https://doi.org/10.3390/brainsci11111510
Sharma B., Jain R. (2014) Right Choice of a Method for Determination of Cut-Off Values: A Statistical Tool for a Diagnostic Test. Asian Journal of Medical Sciences, vol. 5, no 3, pp. 30–34. https://doi.org/10.3126/ajms.v5i3.9296
Slioussar N., Harchevnik M. (2024) Word Order and Context in Sentence Processing: Evidence from L1 and L2 Russian. Frontiers in Psychology, vol. 15, March, Article no 1344366. https://doi.org/10.3389/fpsyg.2024.1344366
Slioussar N., Makarchuk I. (2022) SOV in Russian: A Corpus Study. Journal of Slavic Linguistics, vol. 30, no 29, pp. 1–14. https://doi.org/10.1353/jsl.2022.a923076
Spector J.E. (1992) Predicting Progress in Beginning Reading: Dynamic Assessment of Phonemic Awareness. Journal of Educational Psychology, vol. 84, no 3, pp. 353–363. doi:10.1037/0022-0663.84.3.353
Tops W., Jansen D., Ceulemans E., Petry K., Hilton N. H., Baeyens D. (2022) Participation Problems and Effective Accommodations in Students with Dyslexia in Higher Education. European Journal of Special Needs Education, vol. 38, no 3, pp. 317–333. https://doi.org/10.1080/08856257.2022.2089507
Trauzettel-Klosinski S., Dietz K. (2012) Standardized Assessment of Reading Performance: The New International Reading Speed Texts IReST. Investigative Ophthalmology & Visual Science, vol. 53, no 9, pp. 5452–5461. https://doi.org/10.1167/iovs.11-8284
Uysal P.K., Bilge H. (2019) An Investigation on the Relationship between Reading Fluency and Level of Reading Comprehension according to the Type of Texts. International Electronic Journal of Elementary Education, vol. 11, no 2, pp. 161–172. https://doi.org/10.26822/iejee.2019248590
Valgina N.S. (2003) Theory of Text. Moscow: Logos (In Russian).
Weiner I.B., Graham J.R., Naglieri J.A. (eds) (2012) Handbook of Psychology. Vol. 10. Assessment Psychology. Hoboken, NJ: John Wiley & Sons.
Wiederholt J.L., Bryant B.R. (2001) GORT 4 Gray Oral Reading Tests Examiner's Manual. Austin, TX: Pro-Ed.
Zdorova N.S., Staroverova V.N., Lopukhina A.A., Dzhonboboeva D.B., Pareshina E.A., Shestakova E.R., Gmyrina V.A., Dmitrova E.P., Dragoy O.V. (2025) Reading Assessment Tool "LexiMetr" for Russian-Speaking Primary School Students. Voprosy obrazovaniya / Educational Studies Moscow, no 2, pp. ???? (In Russian) https://doi.org/10.17323/vo-2025-21693
.jpg){kind=small}
