Probabilistic Reasoning, Statistical Attitudes, and AI’s Role in Analyzing Students’ Explanations

Marija Kaplar1[0000-0002-0920-8276] and Aleksandra Stevanović 2[0000-0001-5408-608X]
1 Faculty of Technical Sciences, University of Novi Sad, Serbia,
2 Faculty of Information Technology, Belgrade Metropolitan University, Serbia,
marija.kaplar@uns.ac.rs,
aleksandra.stevanovic@metropolitan.ac.rs
DOI: 10.46793/eLearning2025.291K

Abstract. In contemporary society, the ability to understand and interpret data has become a fundamental skill. Regardless of STEM or non-STEM fields, all individuals is expected the minimum level of stochastic literacy. This study examined Non-STEM students’ base-rate reasoning, their attitudes toward statistics, and the potential of artificial intelligence (AI) to support the analysis of students’ explanations. There participated 105 students, from the University of Novi Sad (non-STEM: Law, Economics) during regular classes. They solved a base-rate task, provided written explanations, and completed an attitude questionnaire. Responses were coded into six categories reflecting correctness and explanation type. Overall, 41% of students answered the task correctly, with no significant differences by faculty or gender. However, almost half of the participants chose the same wrong option. Students of the Faculty of Economics were more likely to provide explanations (70%) compared to law students (25%). When considering only responses with explanations, a significant gender difference was found, with male students more likely to provide correct probability based reasoning, while female students more often relied on equiprobability explanations. Attitude measures showed generally positive orientations although self-efficacy was weaker. AI-based (ChatGPT) coding of explanations gives results comparable to human classification and shows its potential for identifying misconceptions, albeit with certain limitations. Findings emphasize the need to strengthen probabilistic reasoning and statistical self-efficacy among Non STEM students, while also pointing to AI’s potential as a research and pedagogical tool.

Keywords: probabilistic reasoning, base-rate reasoning, attitudes toward statistics, non-STEM students, AI-assisted coding of explanations.

 

References

  1. AlMuraie, E., Algarni, N. A., & Alahmad, N. Sh. (2021). Upper-secondary school science teachers’ perceptions of the integrating mechanisms and importance of stem education. Journal of Baltic Science 557.https://doi.org/10.33225/jbse/21.20.546 Education, 20(4), 546
  2. Attali, Y., Laitusis, C., & Stone, E. (2016). Differences in reaction to immediate feedback and opportunity to revise answers for multiple-choice and open-ended questions. Educational and Psychological https://doi.org/10.1177/0013164415612548. Measurement, 76(5), 787–802.
  3. Bílek, J., Nedoma, J., & Jirásek, M. (2018). Representativeness heuristics: A literature review of its impacts on the quality of decision-making. Scientific papers of the University of Pardubice. Series D, Faculty of Economics & Administration, 25(43), 29–38 Retrieved from https://www.scimagojr.com/journalsearch.php?q=21100390414&tip=sid&clean=0.
  4. Bramwell, R., West, H., & Salmon, P. (2006). Health professionals’ and service users’ interpretation of screening test results: Experimental study. British Medical Journal, 333(7562), 284–286. https://doi.org/10.1136/bmj.38884.663102.AE.
  5. Carvalho, C., & Solomon, Y. (2012). Supporting statistical literacy: What do culturally relevant/realistic tasks show us about the nature of pupil engagement with statistics? In ternational Journal of Educational https://doi.org/10.1016/j.ijer.2012.06.006 Research, 55, 57–65.
  6. Delmas, R., Garfield, J., Ooms, A., & Chance, B. (2007). Assessing students’ conceptual understanding after a first course in statistics. Statistics Education Research Journal, 6(2), 28–58.
  7. Dönmez, I., Idin, S., & Gürbüz, S. (2022). Determining lower-secondary students’ STEM motivation: A profile from Turkey. Journal of Baltic Science Education, 21(1), 38. https://doi.org/10.33225/jbse/22.21.38
  8. Faghmous, J. H., & Kumar, V. (2014). A big data guide to understanding climate change: The case for theory-guided data science. Big Data, 2(3), 155-163. http://doi.org/10.1089/big.2014.0026
  9. Garfield, J., & Ben-Zvi, D. (2007). How students learn statistics revisited: A current review of research on teaching and learning statistics. International Statistical Review, 75(3), 372–396. https://doi.org/10.1111/j.1751-5823.2007.00029.x.
  10. Gauvrit, N., & Morsanyi, K. (2014). The equiprobability bias from a mathematical and psychological perspective. Advances in Cognitive Psychology, 10(4), 119–130. https://doi.org/10.5709/acp-0163-9.
  11. Gigerenzer, G., Gaissmaier, W., Kurz-Milcke, E., Schwartz, L. M., & Woloshin, S.(2007). Helping doctors and patients make sense of health statistics. Psychological Science in the Public Interest, 6053.2008.00033.x. Supplement, 8(2), 53–96. https://doi.org/10.1111/j.1539
  12. Hirsch, L. S., & O’Donnell, A. M. (2001). Representativeness in statistical reasoning:Identifying and assessing misconceptions. Journal of Statistics Education, 9(2). https://doi.org/10.1080/10691898.2001.11910655.
  13. Kahneman, D., & Frederick, S. (2002). Representativeness revisited: Attribute substitution in intuitive judgment. In T. Gilovich, D. Griffin, & D. Kahneman (Eds.), Heuristic and bi ases: The psychology of intuitive judgment, (pp. 49–81). New York: Cambridge University Press. 306 17
  14. Kahneman, D., & Tversky, A. (1973). On the psychology of prediction. Psychological Review, 80(4), 237–251. https://doi.org/10.1037/h0034747.
  15. Kang, M., & Park, M. J. (2019). Employees’ judgment and decision making in the banking industry: The perspective of heuristics and biases. International Journal of Bank Market ing, 37(1), 382–400. https://doi.org/10.1108/IJBM-04-2 018-0111.
  16. Kaplar, M. (2022). Recognizing misconceptions in working with data as a basis to enhance mathematical literacy (Publication No. 29111073) [Doctoral dissertation, University of Novi Sad]. ProQuest Dissertations and Theses Global.
  17. Kaplar, M., Lužanin, Z. & Verbić, S. (2021). Evidence of probability misconception in engineering students —why even an inaccurate explanation is better than no explanation. International Journal of STEM Education, 8, Article
  18. https://doi.org/10.1186/s40594 021-00279-y 18. Khazanov, L., & Prado, L. (2010). Correcting students’ misconceptions about probability in an introductory college statistics course. ALM Internationa Journal, 5(1), 23–35.
  19. Lužanin, Z., Kaplar, M., & Hrubik-Vulanovic, T. (2022). Undergraduate STEM and non STEM students’ interpretation of mean in an infographic. Journal of Baltic Science Education, 21(4), 638-650. https://doi.org/10.33225/jbse/22.21.638
  20. Maass, K., Doorman, M., Jonker, V., & Wijers, M. (2019). Promoting active citizenship in mathematics teaching. ZDM, 51(6), 991–1003. https://doi.org/10.1007/s11858-019-01048 6
  21. Masel, J., Humphrey, P. T., Blackburn, B., & Levine, J. A. (2015). Evidence-based medicine as a tool for undergraduate probability and statistics education. CBE Life Sciences Education, 14(4), 1–10. https://doi.org/10.1187/cbe.15-04-0079.
  22. McClure, J., Smyslova, D., Hall, A., & Jiang, S. (2024). Deductive Coding’s Role in AI vs. Human Performance. In Proceedings of the 17th International Conference on Educational Data Mining (pp. 809-813).
  23. Park, S., Bekemeier, B., Flaxman, A. D. (2021). Understanding data use and preference of data visualization for public health professionals: A qualitative study. Public Health Nurs ing, 8(4), 531-541. https://doi.org/10.1111/phn.12863
  24. Smyrnaiou, Z., Georgakopoulou, E., & Sotiriou, S. (2020). Promoting a mixed- design model of scientific creativity through digital storytelling—the CCQ model for creativity. Inter national Journal of STEM Education, 7(1), 1–22. https://doi.org/10.1186/s40594-020 00223-6.
  25. Williams, R. T. (2024). Paradigm shifts: exploring AI’s influence on qualitative inquiry and analysis. Frontiers in Research Metrics and Analytics, 9, 1331589.
  26. Yan, L., Echeverria, V., Fernandez-Nieto, G. M., Jin, Y., Swiecki, Z., Zhao, L., Gašević, D., & Martinez-Maldonado, R. (2024, May). Human-ai collaboration in thematic analysis using chatgpt: A user study and design recommendations. In Extended Abstracts of the CHI Conference on Human Factors in Computing Systems (pp. 1-7).

Izvor: Proceedings of the 16th International Conference on e-Learning (ELEARNING2025)