Digital Literacy Assessment: Methodology, Conceptual Model and Measurement Tool
Abstract
The development of new technologies, the extensive implementation of digital devices in all spheres of society contributed to the appearance of a new type of human literacy - digital literacy. Today, it ranks alongside other basic literacies such as numeracy, writing, and reading. Digital literacy has become an indicator to the new conditions of life, involving the use of digital technologies in education, family and professional activities; to some extent, it has become an indicator of the socialization of a modern person.
Based on the analysis of international approaches to the assessment of digital literacy and related constructs, a conceptual model of its measurement is proposed, in which digital literacy is considered as a complex latent construct.
The process of developing digital literacy assessment tools based on the Principled Assessment Design paradigm and the Evidence-Centered Design method is presented.
An analysis of the results of digital literacy studies of students in grades 7-8 made it possible to confirm the quality of the developed tools for measuring digital literacy based on authentic performance-based tasks, to confirm the reliability and validity of the results of its measurement.
Downloads
References
Almond R.G., Kim Y.J., Velasquez G., Shute V.J. (2014) How Task Features Impact Evidence from Assessments Embedded in Simulations and Games. Measurement: Interdisciplinary Research & Perspectives, vol. 12, no 1–2, pp. 1–33. https://doi.org/10.1080/15366367.2014.910060
Behrens J., Mislevy R., DiCerbo K., Levy R. (2010) An Evidence Centered Design for Learning and Assessment in the Digital World (CRESST Report no 778). Los Angeles, CA: University of California, National Center for Research on Evaluation, Standards, and Student Testing (CRESST).
Carretero S., Vuorikari R., Punie Y. (2017) DigComp 2.1: The Digital Competence Framework for Citizens. With Eight Proficiency Levels and Examples of Use. Luxembourg: Publications Office of the European Union. https://doi.org/10.2760/38842
Chetty K., Liu Q., Gcora N. et al. (2018) Bridging the Digital Divide: Measuring Digital Literacy. Economics E-Journal, vol. 12, no 1. http://dx.doi.org/10.5018/economics-ejournal.ja.2018-23
De Klerk S., Veldkamp B.P., Eggen T.J.H.M. (2015) Psychometric Analysis of the Performance Data of Simulation-Based Assessment: A Systematic Review and a Bayesian Network Example. Computers & Education, vol. 85, no 12, pp. 23–34. https://doi.org/10.1016/j.compedu.2014.12.020
Eshet-Alkalai Y. (2012) Thinking in the Digital Era: A Revised Model for Digital Literacy. Issues in Informing Science and Information Technology, vol. 9, no 2, pp. 267–276. https://doi.org/10.28945/1621
Ferrara S., Lai E., Nichols P. (2016) Principled Approaches to Assessment Design, Development, and Implementation. The Handbook of Cognition and Assessment: Frameworks, Methodologies, and Applications (eds A.A. Rupp, J.P. Leighton). Chichester, West Sussex, UK: Wiley-Blackwell, pp. 41–74. https://doi.org/10.1002/9781118956588.ch3
Ferrari A. (2013) DIGCOMP: A Framework for Developing and Understanding Digital Competence in Europe. Luxembourg: Publications Office of the European Union.
Fraillon J., Ainley J., Schulz W., Duckworth D., Friedman T. (2019) IEA International Computer and Information Literacy Study 2018. Assessment Framework. Cham, Switzerland: Springer Open. https://doi.org/10.1007/978-3-030-19389-8
Gee J.P. (2015) The New Literacy Studies. The Routledge Handbook of Literacy Studies (eds J. Rowsell, K. Pahl), Abingdon: Routledge, pp. 35–48.
Gee J.P., Hull G., Lankshear C. (2018) The New Work Order: Behind the Language of the New Capitalism. New York, London: Routledge. https://doi.org/10.4324/9780429496127
Gilster P. (1997) Digital Literacy. New York, NY: Wiley Computer Pub.
Glaser R., Chudowsky N., Pellegrino J.W. (eds) (2001) Knowing What Students Know. The Science and Design of Educational Assessment. Washington, DC: National Academy Press.
Goodfellow R. (2011) Literacy, Literacies and the Digital in Higher Education. Teaching in Higher Education, vol. 16, no 1, pp. 131–144. https://doi.org/10.1080/13562517.2011.544125
Green F. (2011) What Is Skill? An Inter-Disciplinary Synthesis. Centre for Learning and Life Chances in Knowledge Economies and Societies Research Paper no 20. London, England: LLAKES.
Haas L., Tussey J. (eds) (2021) Connecting Disciplinary Literacy and Digital Storytelling in K-12 Education. Hershey, PA: IGI Global. https://doi.org/10.4018/978-1-7998-5770-9
He Q., von Davier M., Greiff S., Steinhauer E.W., Borysewicz P.B. (2017) Collaborative Problem Solving Measures in the Programme for International Student Assessment (PISA). Innovative Assessment of Collaboration (eds A.A. von Davier, M. Zhu, P.C. Kyllonen), Cham: Springer, pp. 95–111 https://doi.org/10.1007/978-3-319-33261-1_7
IEEE (2021) IEEE Standard for Digital Intelligence (DQ)—Framework for Digital Literacy, Skills, and Readiness. https://doi.org/10.1109/IEEESTD.2021.9321783
Marion S., Landl E. (2017) Principled Assessment Design for the Performance Assessment of Competency Education. Available at: https://www.nciea.org/wp-content/uploads/2021/11/PACE-Principled-assessment-design_092417.pdf (accessed 20 May 2023).
Messick S. (1994) Alternative Modes of Assessment, Uniform Standards of Validity. ETS Research Report Series, iss. 2, pp. 1–22. https://doi.org/10.1002/j.2333-8504.1994.tb01634.x
Mislevy R.J. (2018) Sociocognitive Foundations of Educational Measurement. New York, NY: Routledge. https://doi.org/10.4324/9781315871691
Mislevy R.J. (1994) Evidence and Inference in Educational Assessment. Psychometrika, vol. 59, no 4, pp. 439–483. https://doi.org/10.1007/bf02294388
Mislevy R.J., Almond R.G., Lukas J.F. (2003) A Brief Introduction to Evidence‐Centered Design. ETS Research Report Series, no 1, pp. 1–29. https://doi.org/10.1002/j.2333-8504.2003.tb01908.x
Mislevy R.J., Behrens J.T., Dicerbo K.E., Levy R. (2012) Design and Discovery in Educational Assessment: Evidence-Centered Design, Psychometrics, and Educational Data Mining. Journal of Educational Data Mining, vol. 4, no1, pp. 11–48. https://doi.org/10.5281/zenodo.3554641
Mislevy R.J., Haertel G.D. (2006) Implications of Evidence‐Centered Design for Educational Testing. Educational Measurement: Issues and Practice, vol. 25, no 4, pp. 6–20, https://doi.org/10.1111/j.1745-3992.2006.00075.x
Moje E.B. (2009) Standpoints: A Call for New Research on New and Multi-Literacies. Research in the Teaching of English, vol. 43, no 4, pp. 348–362.
Street B. (2003) What's "New" in New Literacy Studies? Critical Approaches to Literacy in Theory and Practice. Current Issues in Comparative Education, vol. 5, no 2, pp. 77–91.
Toulmin S. (2003) The Uses of Argument. Updated Edition. Cambridge, UK: Cambridge University.
UNESCO (2019) Recommendations on Assessment Tools for Monitoring Digital Literacy within UNESCO’s Digital Literacy Global Framework. Information Paper no 56. Montreal, Quebec: UNESCO Institute for Statistics.
UNESCO (2018) A Global Framework of Reference on Digital Literacy Skills for Indicator 4.4.2. Information Paper no 51. Montreal, Quebec: UNESCO Institute for Statistics.
Vuorikari R., Kluzer S., Punie Y. (2022) DigComp. 2.2: The Digital Competence Framework for Citizens. With New Examples of Knowledge, Skills and Attitudes. Luxembourg: Publications Office of the European Union. https://dx.doi.org/10.2760/490274
Yu C., Muthen B. (2002) Evaluation of Model Fit Indices for Latent Variable Models with Categorical and Continuous Outcomes. Paper presented at the Annual conference of the American Educational Research Association (4 April 2002, New Orleans).
Zieky M.J. (2013) An Introduction to the Use of Evidence-Centered Design in Test Development. Psicología Educativa, vol. 20, no 2, pp. 79–87. https://doi.org/10.1016/j.pse.2014.11.003
.jpg){kind=small}
