Измерение критического мышления студентов в открытой онлайн-среде: методология, концептуальная рамка и типология заданий
Аннотация
В условиях обилия и доступности различной информации, в том числе благодаря распространению электронных устройств и интернета, человеку необходимо уметь эффективно ориентироваться в информационном поле, оценивать информацию, делать собственные выводы на ее основе и использовать аргументы при принятии решений. Поэтому одной из ключевых компетенций современного человека, обеспечивающих его успешность в профессиональной и повседневной жизни, становится критическое мышление.
Авторы оценивают теоретическую возможность измерения сложного латентного конструкта «критическое мышление в онлайн-среде» с помощью методологии Evidence-Centered Design. Она позволяет выстроить цепочку аргументов, обосновывающих вывод об уровне развития критического мышления у респондента, и тем самым обеспечить справедливость процесса оценки. Оценивание выстраивается от теоретических предположений о природе конструкта к поиску эмпирических свидетельств — наблюдаемых в процессе тестирования действий, позволяющих сделать обоснованные выводы о респондентах.
Предложена теоретическая рамка оценивания критического мышления в онлайн-среде для студентов вузов и ее операционализация через релевантное наблюдаемое поведение целевой аудитории, позволяющая получить валидные данные относительно выраженности критического мышления. Тем самым сделан первый шаг к созданию русскоязычного инструмента для оценки критического мышления с подтвержденным психометрическим качеством. Ключевой особенностью инструмента является то, что студент работает не в симулированной среде, где источники информации подбираются разработчиками, а в открытой онлайн-среде и может пользоваться любыми доступными материалами для решения поставленной задачи.
Скачивания
Литература
Aloisi C., Callaghan A. (2018) Threats to the Validity of Peer Learning Assessment (CLA +) as an Indicator of Critical Thinking Skills and the Implications for Knowledge. Pedagogy of Higher Education, vol. 3, no 1, pp. 57-82. https://doi.org/10.1080/23752696.2018.1449128
American Educational Research Association (2018) Standards for Educational and Psychological Testing. Washington, DC: American Educational Research Association.
Bartram D. (2005) The Great Eight Competencies: A Criterion-Centric Approach to Validation. Journal of Applied Psychology, vol. 90, no 6, pp. 1185-1203. https://doi.org/10.1037/0021-9010.90.6.1185
Baturin N., Vuchetich E., Kostromina S. et al. (2015) Rossijskij standart testirovaniya personala (vremennaya versiya, sozdannaya dlya shirokogo obsuzhdeniya v 2015 g.) [Russian Standard for Personnel Testing (Interim Version, Designed for a Discussion)]. Organizational Psychology, vol. 5, no 2, pp. 67-138.
Beer de D., Matthee M. (2021) Approaches to Identify Fake News: A Systematic Literature Review. Integrated Science in Digital Age 2020 (ed. T. Antipova), Cham: Springer, pp. 13-22. https://doi.org/10.1007/978-3-030-49264-9_2
Behar-Horenstein L.S., Niu L. (2011) Teaching Critical Thinking Skills in Higher Education: A Review of the Literature. Journal of College Teaching & Learning, vol. 8, no 2, pp. 25-41. https://doi.org/10.19030/tlc.v8i2.3554
Bernard R.M., Zhang D., Abrami P.C., Sicoly F., Borokhovski E., Surkes M. A. (2008) Exploring the Structure of the Watson-Glaser Critical Thinking Appraisal: One Scale or Many Subscales? Thinking Skills and Creativity, vol. 3, no 1, pp. 15-22. https://doi.org/10.1016/j.tsc.2007.11.001
Blonskiy P.P. (1935) Razvitie myshleniya shkol'nika [The Development of the Schoolchild's Thinking]. M.: Uchpedgiz.
Boyatzis R. (2008) Kompetentny menedzher. Model' effektivnoj raboty [The Competent Manager], Moscow: HIPPO.
Braun H., Kirsch I., Yamamoto K. (2011) An Experimental Study of the Effects of Monetary Incentives on Performance on the 12th-Grade NAEP Reading Assessment. Teachers College Record, vol. 113, no 11, pp. 2309-2344. https://doi.org/10.1177/016146811111301101
Carnevale A.P., Smith N. (2013) Workplace Basics: The Skills Employees Need and Employers Want. Human Resource Development International, vol. 16, no 5, pp. 491-501. https://doi.org/10.1080/13678868.2013.821267
Davydov V.V. (1972) Vidy obobshchenij v obuchenii [Types of Generalizations in Learning], M.: Pedagogika.
Davydov V.V. (1996) Teoriya razvivayushchego obucheniya [Theory of Developmental Learning], Moscow: Intor.
Dewey J. (1909) Moral Principles in Education. Boston: Houghton Mifflin.
El'konin D.B. (1971) K probleme periodizatsii psikhicheskogo razvitiya v detskom vozraste [On the Problem of Periodization of Mental Development in Childhood]. Voprosy Psychologii, no 4, pp. 6-20.
Ennis R.H. (1993) Critical Thinking Assessment. Theory into Practice, vol. 32, no 3, pp. 179-186. https://doi.org/10.1080/00405849309543594
Ennis R.H., Millman J., Tomko T.N. (2005) Cornell Critical Thinking Tests. Seaside, CA: The Critical Thinking Co.
Ennis R.H., Weir E.E. (1985) The Ennis-Weir Critical Thinking Essay Test: An Instrument for Teaching and Testing. Pacific Grove, CA: Midwest Publications.
Facione P. (1990) Critical Thinking: A Statement of Expert Consensus for Purposes of Educational Assessment and Instruction. ERIC Doc. No ED 315 423. Fullerton, CA: California State University.
Fawkes D., O'Meara B., Weber D., Flage D. (2005) Examining the Exam: A Critical Look at the California Critical Thinking Skills Test. Science & Education, vol. 14, no 2, pp. 117-135. https://doi.org/10.1007/s11191-005-6181-4
Griffin P., Care E. (eds) (2014) Assessment and Teaching of 21st Century Skills: Methods and Approach. Dordrecht: Springer. https://doi.org/10.1007/978-94-017-9395-7
Hitchcock D. (2020) Critical Thinking. The Stanford Encyclopedia of Philosophy (ed. E.N. Zalta), Stanford, CA: Metaphysics Research Lab, Stanford University. Available at: https://plato.stanford.edu/archives/fall2020/entries/critical-thinking/ (accessed 2 September 2022).
Il'enkov E.V. (2002) Shkola dolzhna uchit' myslit' [School Should Teach Thinking], Moscow: Russian Academy of Education, Moscow Psychological and Social Institute.
Il'enkov E.V. (1979) Dialekticheskoe protivorechie [Dialectical Contradiction], Moscow: Politizdat.
Il'enkov E.V. (1974) Dialekticheskaya logika [Dialectical Logic]. Moscow: Politizdat.
Jackson S. (1965) The Growth of Logical Thinking in Normal and Subnormal Children. British Journal of Educational Psychology, vol. 35, no 2, pp. 255-258. https://doi.org/10.1111/j.2044-8279.1965.tb01811.x
Kennedy M., Fisher M.B., Ennis R.H. (1991) Critical Thinking: Literature Review and Needed Research. Educational Values and Cognitive Instruction: Implications for Reform (eds L. Idol, B. Fly Jones), Hillsdale, NJ: Lawrence Erlbaum, pp. 11-40.
Khan U. (2020) Developing Critical Thinking in Student Seafarers: An Exploratory Study. Journal of Applied Learning & Teaching, vol. 3, Sp. Iss. 1, pp. 40-50. https://doi.org/10.37074/jalt.2020.3.s1.15
Koreshnikova Yu.N., Froumin I.D., Pashchenko T.V. (2020) Bar'ery dlya sozdaniya pedagogicheskikh uslovij razvitiya kriticheskogo myshleniya v rossijskikh vuzakh [Barriers to Creating Pedagogical Conditions for the Development of Critical Thinking in Russian Universities]. Pedagogika, vol. 84, no 9, pp. 45-54.
Kurz R. (2009) The Structure of Work Effectiveness as Measured through the Saville Consulting Wave® Performance 360 'BA-G'Model of Behaviour, Ability and Global Performance. Assessment & Development Matters, vol. 1, no 1, pp. 15-18.
Lai E.R. (2011) Critical Thinking: A Literature Review Research Report. London: Parsons Publishing.
Lane S., Stone C. (2006) Performance Assessment. Educational Measurement (ed. R.L. Brennan), Westport, CT: American Council on Education and Praeger, pp. 387-431.
Liu O.L., Frankel L., Roohr K.C. (2014) Assessing Сritical Thinking in Higher Education: Current State and Directions for Next-Generation Assessment. ETS Research Report Series, no 2014(1). pp. 1-23. https://doi.org/10.1002/ets2.12009
Lovell K. (1961) A Follow-Up Study of Inhelder and Piaget's. The Growth of Logical Thinking. British Journal of Psychology, vol. 52, no 2, pp. 143-153. https://doi.org/10.1111/j.2044-8295.1961.tb00776.x
Messick S. (1994) The Interplay of Evidence and Consequences in the Validation of Performance Assessments. Educational Researcher, vol. 23, no 2, pp. 13-23. https://doi.org/10.3102/0013189X023002013
Mislevy R.J., Almond R.G., Lukas J.F. (2003) A Brief Introduction to Evidence-Centered Design. ETS Research Report Series, no 2003(1), pp. i-29. https://doi.org/10.1002/j.2333-8504.2003.tb01908.x
Nagel M.-T., Schäfer S., Zlatkin-Troitschanskaia O. et al. (2020) How Do University Students' Web Search Behavior, Website Characteristics, and the Interaction of Both Influence Students' Critical Online Reasoning. Frontiers in Education, no 5, Article no 565062. https://doi.org/10.3389/feduc.2020.565062
Norris S.P. (1985) Synthesis of Research on Critical Thinking. Educational Leadership, vol. 42, no 8, pp. 40-45.
Oliveri M.E., Mislevy R.J. (2019) Introduction to "Challenges and Opportunities in the Design of 'Next-Generation Assessments of 21st Century Skills'". International Journal of Testing. Special Issue, vol. 19, no 2, pp. 97-102. https://doi.org/10.1080/15305058.2019.1608551
Paul R., Elder L. (2011) Critical Thinking: Tools for Taking Charge of Your Learning and Your Life. Upper Saddle River, NJ: Prentice Hall.
Piaget J. (2008) Rech' i myshlenie rebenka [The Language and Thought of the Child]. Moscow: Rimis.
Possin K. (2014) Critique of the Watson-Glaser Critical Thinking Appraisal Test: The More You Know, the Lower Your Score. Informal Logic, vol. 34, no 4, pp. 393-416. https://doi.org/10.22329/il.v34i4.4141
Probierz B., Stefański P., Kozak J. (2021) Rapid Detection of Fake News Based on Machine Learning Methods. Procedia Computer Science, vol. 192, pp. 2893-2902. https://doi.org/10.1016/j.procs.2021.09.060
Rubinstein L.S. (2002) Osnovy obshchej psikhologii [Fundamentals of General Psychology]. Saint Petersburg: Piter.
Seaman M. (2011) Bloom's Taxonomy: Its Evolution, Revision, and Use in the Field of Education. Curriculum and Teaching Dialogue, vol. 13, no 1-2, pp. 29-131A.
Simonenko S.I. (2012). Model' effektivnogo rukovoditelya v ramkakh kontseptsii dinamicheskogo liderstva [Model of Effective Manager in Frame of Dynamic Leadership Concept]. Izvestiya of Saratov University. Philosophy. Psychology. Pedagogy, vol. 12, no 4, pp. 90-96. https://doi.org/10.18500/1819-7671-2012-12-4-90-96
Spencer L.M., Spencer S.M. (2005) Kompetentsii na rabote [Competence at Work. Models for Superior Performance]. Moscow: HIPPO.
Steedle J.T. (2012) Selecting Value-Added Models for Postsecondary Institutional Assessment. Assessment & Evaluation in Higher Education, vol. 37, no 6, pp. 637-652. https://doi.org/10.1080/02602938.2011.560720
Stepashkina E., Sukhodoev A., Gudgelya D. (2022) Issledovanie profilya nadprofessional'nykh kompetentsij, vostrebovannykh vedushchimi rabotodatelyami pri prieme na rabotu studentov i vypusknikov universitetov i molodykh spetsialistov [The Research on the Essential Range of Soft Skills Enquired by Leading Employers during the Process of Recruitment of University Graduates and Young Professionals]. Moscow: HSE.
Sternberg R.J. (1986) Critical Thinking: Its Nature, Measurement, and Improvement. ERIC no ED272882. Available at: https://files.eric.ed.gov/fulltext/ED272882.pdf (accessed 2 September 2022).
Teplov B. (1946) Psikhologiya [Psychology]. Moscow: Uchpedgiz.
Tremblay K. (2013) OECD Assessment of Higher Education Learning Outcomes (AHELO): Rationale, Challenges and Initial Insights from the Feasibility Study. Modeling and Measuring Competencies in Higher Education (eds S. Blömeke, O. Zlatkin-Troitschanskaia, Ch. Kuhn, Ju. Fege), Rotterdam, Boston, Taipei: Sense, pp. 113-126.
Uglanova I., Brun I., Vasin G. (2018) Metodologiya Evidence-Centered Design dlya izmereniya kompleksnykh psikhologicheskikh konstruktov [Evidence-Centered Design Method for Measuring Complex Psychological Constructs]. Journal of Modern Foreign Psychology, vol. 7, no 3, pp. 18-27. https://doi.org/10.17759/jmfp.2018070302
Uglanova I.L., Orel E.A., Brun I.V. (2020) Izmerenie kreativnosti i kriticheskogo myshleniya v nachal'noj shkole [Measuring Creativity and Critical Thinking in Primary School]. Psikhologicheskij zhurnal, vol. 41, no 6, pp. 96-107. https://doi.org/10.31857/S020595920011124-2
Uglanova I. L., Pogozhina I. N. (2021) Chto mozhet predlozhit' novaya metodologiya otsenki myshleniya shkol'nikov sovremennomu obrazovaniyu [What the New Measure of Thinking in School Students Has to Offer to Contemporary Education]. Voprosy obrazovaniya / Educational Studies Moscow, no 4, pp. 8-34. https://doi.org/10.17323/1814-9545-2021-4-8-34
Watson G., Glaser E.M. (1980) Watson-Glaser Critical Thinking Appraisal Manual. San Antonio, TX: Psychological Corporation.
Zeigarnik B.V. (1986) Patopsikhologiya [Pathopsychology], Moscow: Moscow University Publishing House.
Zlatkin-Troitschanskaia O., Shavelson R.J. (2019) Advantages and Challenges of Performance Assessment of Student Learning in Higher Education. British Journal of Educational Psychology, vol. 89, no 3, pp. 413-415. https://doi.org/10.1111/bjep.12314
Zlatkin-Troitschanskaia O., Toepper M., Pant H.A., Lautenbach C., Kuhn C. (2018) Assessment of Learning Outcomes in Higher Education: Cross-National Comparisons and Perspectives. Cham: Springer. https://doi.org/10.1007/978-3-319-74338-7
