Hybrid Estimation for Open-Ended Questions with Early-Age Students’ Block-Based Programming Answers

Hao Wu (Zhejiang University City College); Tianyi Chen (Zhejiang University City College); Xianzhe Luo (Zhejiang University City College); Canghong Jin (Zhejiang university city college); Yun Zhang (Zhejiang University City College); Minghui Wu (浙江大学城市学院)*

---

PMLR Page

Abstract

Block-based programming is of great significance for cultivating children's computational thinking. However, due to the following challenges, it is difficult to evaluate students' programming ability in online learning systems: 1) compared with the traditional Online Judge (OJ) system, there is no standard answer for a given task in block-based programming; 2) in order to promote students' interests, although the programs are not totally correct and unrelated to the task, the teacher will give a comparatively higher score. Therefore, current approaches involving output comparison and code analysis do not work effectively. Furthermore, deep learning methods also suffer from the problem of how to represent block code for classification. We propose a novel hybrid estimation model to address these challenges. We first learn graph embedding from the parsed Abstract Syntax Tree (AST) to present the logicality of the code. Next, we provide some methods to measure the workload and complexity of the code. Then, we extracted some key variables and task-irrelevant properties, introduced teacher bias. Finally, XGBoost was constructed for classification. Based on real-world data collected from an online Scratch platform by early-age students, our model outperforms KimCNN, ResNet-18, and Graph2Vec+XGBoost. Moreover, we provided statistical analyses and intuitive explanations to interpret the characteristics in various groups.