Home | Publications

Publications

Beal, C. R., Qu, L., & Lee, H. (2009). Mathematics motivation and achievement as predictors of high school students' guessing and help-seeking with instructional software. Journal of Computer Assisted Learning. Per Blackwell Publishing copyright guidelines, an electronic version may be posted 12 months after publication.

Abstract: The study was conducted to investigate the relation of mathematics motivation and achievement to appropriate help-seeking and inappropriate guessing behavior while using instructional software. High school students completed assessments of math motivation and then working with software for geometry instruction. Actions with the software were machine-classified to identify instances of appropriate help-seeking from multimedia resources in the software, and instances of inappropriate guessing. Mathematics teachers provided information about students' math achievement. Students with low math self-concept were most likely to engage in guessing. Students with low math achievement were most likely to engage in appropriate help-seeking while working with the software.


Beal, C. R., & Shaw, E. (2008). Working memory and math problem solving by blind middle and high school students: Implications for universal access. Proceedings of the 19th International Conference of the Society for Information Technology and Teacher Education, Las Vegas, 2008, in the Symposium on "Promising technology practices for working with students with disabilities".

Abstract: Math achievement tends to be low for blind students, relative to other academic subjects. The project is investigating how blind students solve math word problems varying in text length and grade-level readability, in the context of the AnimalWatch tutorial for pre-algebra. Text-to-speech software allows blind students to access word problems in audio format. Replay actions are used to track processing of word problems varying in word count and other text characteristics. Results will be used to adapt AnimalWatch to select word problems appropriate for students with visual impairments.


Birch, M., & Beal, C. R. (2008). Problem posing in AnimalWatch: An interactive system for student-authored content. Paper presented at the 21st International FLAIRS conference, Coconut Grove FL, May 15-17, 2008.

Abstract: Bringing users in the process of content development may help to reduce the time and costs involved in tutoring system development and may benefit users by deepening their understanding of the domain. We describe a pilot effort with middle school students who successfully authored word problems for the AnimalWatch tutoring system and the design of a new module for content authoring.


Cohen, P. R., Beal, C. R., & Adams, N. M. (2008). The design, deployment and evaluation of the AnimalWatch intelligent tutoring system. Paper accepted for presentation at the 5th Prestigious Applications of Intelligent Systems (PAIS) conference, July 21-25, Patras Greece.

Abstract: Europe and the United States both face the challenges of urban schools with low achieving learners, many of whom are not proficient in the language of instruction. Prior work demonstrated that the AnimalWatch tutoring system for pre-algebra benefits 12-14 year olds in relatively controlled conditions. The current study indicates that the system can help older, very low achieving learners in challenging secondary schools that serve diverse student populations.


Beal, C. R., Walles, R., Arroyo, I., & Woolf, B. P. (2007). Online tutoring for math achievement: A controlled evaluation. Journal of Interactive Online Learning, 6, 43-55

Abstract: We report the results of a controlled evaluation of an interactive online tutoring system for high school math problem solving. Students (N = 202) completed a math pretest and were then assigned by teachers to receive interactive multimedia tutoring or regular classroom instruction. The tutoring group students improved at post test but the effect was limited to problems involving skills tutored in the online system (within group control). Control group students showed no improvement. Use of interactive multimedia hints predicted pre to post test improvement. Benefits of interactive tutoring were greatest for students with weakest initial math skills.


Beal, C. R., Mitra, S., & Cohen, P. R. (2007). Modeling learning patterns of students with a tutoring system using Hidden Markov Models. In R. Luckin, K. R. Koedinger, & J. Greer (Eds.), Artificial intelligence in education: Building technology rich learning contexts that work (pp. 238-245). Amsterdam: IOS Press.

Abstract: The paper focuses on modeling actions of high school students with the Wayang Outpost tutoring system with Hidden Markov Models. The results indicated that including a hidden state estimate of learner engagement increased the accuracy and predictive power of the models, both within and across tutoring sessions. Groups of students with distinct engagement trajectories were identified, and findings were replicated in two independent samples. The results suggest that modeling learner engagement may help to increase the effectiveness of tutoring systems, because engagement trajectories were not predicted by prior math achievement of students.


Beal, C. R., & Qu, L. (2007). Relating machine estimates of students' learning goals to learning outcomes: A DBN approach. In R. Luckin, K. R. Koedinger, & J. Greer (Eds.), Artificial intelligence in education: Building technology rich learning contexts that work (pp. 111-118). Amsterdam: IOS Press.

Abstract: Students' actions while working with the Wayang Outpost math tutoring system were used to generate estimates of learning goals, specifically, the goal of learning by using multimedia help resources, and the goal of learning through independent problem solving. A Dynamic Bayesian Network model was trained with interface action and inter-action interval latency data from 115 high school students, and then test with action data from an independent sample of 135 students. Estimates of learning goals generated by the model predicted student performance on a post-test of math achievement, whereas pre-test performance did not.


Beal, C. R., Shaw, E., & Birch, M. (2007). Intelligent tutoring and human tutoring in small groups: An empirical comparison. In R. Luckin, K. R. Koedinger & J. Greer (Eds.), Artificial intelligence in education: Building technology rich learning contexts that work (pp. 536-538).

Abstract: The efficacy of the AnimalWatch pre-algebra tutoring system plus human tutoring was compared to instruction provided to small groups of middle school students by experienced math tutors, with instructional time held constant. Students completed pre- and post-tests of computation, fractions and rational numbers skills. Results indicated that students showed significant improvement from pre- to post-test but that there were no differences due to type of tutoring (50% AnimalWatch + 50% small group tutoring, vs. 100% small group tutoring). The findings help to establish the efficacy of ITS instruction relative to skilled human tutoring of students in small groups.


Beal, C. R., & Stevens, R. H. (2007). Student motivation and performance in scientific problem solving simulations. In R. Luckin, K. R. Koedinger & J. Greer (Eds)., Artificial intelligence in education: Building technology rich learning contexts that work (pp. 539-541). Amsterdam: IOS Press.

Abstract: Students do not always use efficient strategies to solve science problems. Motivation beliefs were assessed through an online survey instrument, along with performance on easy and more challenging multimedia science simulation problems. Female students reported lower self-efficacy and higher concerns about performance than male students. Initial strategies predicted overall performance on the more challenging problems, and motivational beliefs predicted overall performance. Results suggest that both cognitive and motivational factors influence strategic efficacy.


Woolf, B. P., Arroyo, I., Beal, C. R., & Murray, T. (2006). Gender and cognitive differences in help effectiveness during problem solving. International Journal of Technology, Instruction, Cognition and Learning, 3, 89-95.

Abstract: The article describes analyses of data from the original versions of the AnimalWatch and Wayang Outpost tutoring software, suggesting that scaffolding was differentially effective for students with varying cognitive characteristics, and by gender. Girls were more likely to spend time with multimedia hints, and their perceptions of the tutoring systems were more positive than boys'. Animations were associated with stronger learning outcomes.


Beal, C. R., Qu, L., & Lee, H. (2006). Classifying learner engagement through integration of multiple data sources. Proceedings of the 21st National Conference on Artificial Intelligence, July 16-20, 2006, Boston MA.

Abstract: Intelligent tutoring systems (ITS) can provide effective instruction, but students do not always use such systems effectively. In the present study, student action sequences with a mathematics ITS were machine classified into patterns indicating guessing strategies, appropriate help use, and independent problem solving, with over 90% of problem events successfully categorized. Students were grouped via cluster analyses based on self reports of motivation. Motivation grouping predicted ITS strategic approach better than prior math achievement. Learners who reported being disengaged in math were most likely to exhibit appropriate help use while working with the ITS, relative to average and high motivation learners. The results indicate that learners can readily report their motivation and that these data predict how learners interact with the ITS.


Arroyo, I., Woolf, B. P., & Beal, C. R. (2006). Addressing cognitive differences and gender during problem solving. Technology, Instruction, Cognition, and Learning, 4, 31-63.

Abstract: We evaluated the impact of supplementing user models with additional data about cognitive features of students, including developmental stage, spatial ability, math-fact-retrieval proficiency and gender. These differences were applied along with multimedia and customization in two tutoring systems (pre-algebra and geometry). Results supported the general conclusion that enhancing user models with additional information about student cognition led to improved response to instruction.


Beal, C. R., Shaw, E., Chiu, Y., Lee, H., Vilhjalmsson, H., & Qu, L. (2005). Enhancing ITS instruction with integrated assessments of learner mood, motivation and gender. Poster presentation at the 12th International Conference on Artificial Intelligence in Education, July 18-22, 2005, Amsterdam.

Abstract: ITS instruction may be enhanced by models of student motivation and mood, in addition to cognitive skills and domain knowledge. In an initial study, self-assessments by high school students of their mathematics motivation and mood showed gender differences in response to ITS instruction, and predicted students' intention to learn from the ITS and use of multimedia help features.


Beal, C. R., & Lee, H. (2005). Creating a pedagogical model that uses student self reports of motivation and mood to adapt ITS instruction. Workshop on Motivation and Affect in Educational Software, July 18-22, 2005, Amsterdam. 12th International Conference on Artificial Intelligence and Education.

Abstract: Our project focuses on the design, implementation and evaluation of a ITS pedagogical model that considers student motivation, mood and cognitive processes in making instructional decisions in the domain of secondary school mathematics. Students complete integrated self report assessments of motivation and mood. Cognitive skillls such as math fact knowledge, spatial cognition, and prior math achievement are also assessed. The pedagogical model adapts instruction (problem selection, problem difficulty, topic area, choice of activity, choice of help type, and availability of help) following a model of human tutoring expertise that balances motivational and cognitive goals.


Beal, C. R., & Cohen, P. (2005). Computational methods for evaluating student and group learning histories in intelligent tutoring systems. 12th International Conference on Artificial Intelligence in Education, July 18-22, 2005, Amsterdam.

Abstract: Intelligent tutoring systems customize the learning experiences of students. Be cause no two students have precisely the same learning history, traditional analytic techniques are not appropriate. This paper shows how to compare the learning his tories of students and how to compare groups of students in different experimental conditions. A class of randomization tests is introduced and illustrated with data from the AnimalWatch ITS project for elementary school arithmetic.


Beal, C. R., Chiu, Y., Shaw, E., & Vilhjamsson, H. (2005). Metacognitive reflection in ITS math problem solving.

Abstract: High school students used a self-reflection feature integrated into a mathematics ITS to indicate what they found difficult about a math problem, and what insights were required to solve it. Results indicated that students could often identify what specific information would help them solve a problem, and could evaluate what was helpful and not helpful about the multimedia explanations. Future work will focus on the impact of self-reflection on students' problem solving and transfer.
Copyright © 2006 The University of Southern California. All RIGHTS RESERVED.
All trademarks mentioned herein belong to their respective owners. This site is W3C compliant.