Cristina Conati

Professor

Relevant Degree Programs

 

Postdoctoral Fellows

Graduate Student Supervision

Doctoral Student Supervision (Jan 2008 - May 2019)
A data mining approach for adding adaptive interventions to exploratory learning environments (2017)

Due to the open-ended nature of the interaction with Exploratory Learning Environments (ELEs), it is not trivial to add mechanisms for providing adaptive support to users. Our goal is to devise and evaluate a data mining approach for providing adaptive interventions that help users to achieve better task performance during interaction with ELEs. The general idea of this thesis is as follows:In an exploratory and open-ended environment, we collect interaction data of users while they are working with the system, and then find representative patterns of behavior for different user groups that achieved various levels of task performance. We use these patterns to provide adaptive real-time interventions designed to suggest or enforce the effective interaction behaviors while discouraging or preventing the ineffective ones. We test and confirm the hypothesis that as a result of these interventions, the average learning performance of the new users who work with the adaptive version of this ELE is significantly higher than the non-adaptive version.We use an interactive simulation for learning Constraint Satisfaction Problems (CSP), the AIspace CSP applet, as the test-bed for our research and propose a framework which covers the entire process described above, called the User Modeling and Adaptation (UMA) framework. The contributions of this thesis are two-fold:i) It contributes to the Educational Data Mining (EDM) research, by devising, modifying, and testing different techniques and mechanisms for a complete data mining based approach to delivering adaptive interventions in ELEs summarized in the UMA framework. The UMA framework consists of 3 phases: Behavior Discovery, User Classification, and Adaptive Support. We assessed each of the above phases in a series of user studies. This work is the first to fully evaluate and provide positive evidence for the use of a data mining approach for deriving and delivering adaptive interventions in ELEs with the goal of improving the user’s performance.ii) It also contributes to the user modeling and user-adapted interaction community by providing new evidence for the usefulness of the eye-gaze data for the purpose of predicting learning performance of users while interacting with an ELE.

View record

Master's Student Supervision (2010 - 2018)
Toward user-adaptive visualizations : further results on real-time prediction of user cognitive abilities from action and eye-tracking data (2017)

Previous work has shown that some user cognitive abilities relevant for processing information visualizations can be predicted from eye tracking data. Performing this type of user modeling is important for devising user-adaptive visualizations that can adapt to a user’s abilities as needed during the interaction. In this thesis, we contribute to previous work by extending the type of visualizations considered and the set of cognitive abilities that can be predicted from gaze data, thus providing evidence on the generality of these findings. We also evaluate how quality of gaze data impacts prediction. Finally, we further extend previous work by investigating interaction data as an alternative source to predict our target user characteristics. We present a formal comparison of predictors based solely on gaze data, on interaction data, or on a combination of the two.

View record

Trade-offs in data representations for learner models in interactive simulations (2016)

Interactive simulations can foster student driven, exploratory learning. However, students may not always learn effectively in these unstructured environments. Due to this, it would be advantageous to provide adaptive support to those that are not effectively using the learning environment. To achieve this, it is helpful to build a user-model that can estimate the learner’s trajectories and need for help during interaction. However, this is challenging because it is hard to know a priori which behaviors are conducive to learning. It is particularly challenging in complex Exploratory Learning Environments (like in PhET’s DC Circuit Construction Kit which is used in this work) because of the large variety of ways to interact. To address this problem, we evaluate multiple representations of student interactions with the simulation that capture different amounts of granularity and feature engineering. We then apply the student modeling framework proposed in [1] to mine the student behaviors and classify learners. Our results indicate that the proposed framework is able to extend to a more complex environment in that we are able to successfully classify students and identify behaviors intuitively associated with high and low learning. We also discuss the trade-offs between the differing levels of granularity and feature engineering in the tested interaction representations in terms of their ability to evaluate learning and inform feedback.[1] Samad Kardan and Cristina Conati. 2011. A Framework for Capturing Distinguishing User Interaction Behaviours in Novel Interfaces. Proceedings of the 4th International Conference on Educational Data Mining, 159–168.

View record

Constructing user models from eye gaze data in the domain of information visualization (2015)

A user-adaptive information visualization system capable of learning models of users and the visualization tasks they perform could provide interventions optimized for helping specific users in specific task contexts. This thesis investigates the accuracy of predicting visualization tasks, user performance on tasks, and user traits from gaze data. It is shown that predictions made with a logistic regression model are significantly better than a baseline classifier, with particularly strong results for predicting task type and user performance. Furthermore, classifiers built with interface-independent are compared with classifiers built with interface-dependent features. Interface-independent features are shown to be comparable or superior to interface-dependent ones. Adding highlighting interventions to trials is shown to have an effect on the accuracy of predictive models trained on the data from those trials and these effects are discussed. The applicability of all results to real-time classification is tested using datasets that limit the amount of observations that are processed into classification features. Finally, trends in features selected by classifiers and classifier accuracies over time are explored as a means to interpret the performance of the tested classification models.

View record

Inferring user cognitive abilities from eye-tracking data (2015)

User-adaptive visualization can provide intelligent personalization to aid the user in the information processing. The adaptations, in the form as simple as helpful highlighting, are applied based on user’s characteristics and preferences inferred by the system. Previous work has shown that binary labels of user’s cognitive abilities relevant for processing information visualizations could be predicted in real time, by leveraging user gaze patterns collected via a non-intrusive eye-tracking device. The classification accuracies reported were in the 59–65% range, which is statistically more accurate than a majority-class classifier, but not of great practical significance. In this thesis, we expand on previous work by showing that significantly higher accuracies can be achieved by leveraging summative statistics on a user’s pupil size and head distance to the screen measurements, also collected by an eye tracker. Our experiments show that these results hold for two datasets, providing evidence of the generality of our findings. We also explore the sequential nature of gaze movement by extracting common substring patterns and using the frequency of these patterns as features for classifying user’s cognitive abilities. Our sequence features are able to classify more accurately than the majority-class baseline, but unable to outperform our best classification model with the summative eye-tracking features.

View record

Eye-tracking as a source of information for automatically predicting user learning with MetaTutor, an intelligent tutoring system to support self-regulated learning (2014)

Student modeling has been gaining interest among researchers recently. A lot of work has been done on exploring value of interface actions on predicting learning. The focus of this thesis is on using eye-tracking data and action logs for building classifies to infer a student’s learning performance during interaction with MetaTutor, an Intelligent Tutoring System( ITS) that scaffolds self-regulated learning (SRL). Research has shown that eye tracking can be a valuable source for predicting learning for certain learning environments. In this thesis we extend these results by showing that modeling based on eye-tracking data is a valuable approach to predicting learning for another type of ITS, a hypermedia learning environment. We use data from 50 students (collected by a research team at McGill University, which also designed MetaTutor) to compare the performance of actions and eye-tracking data (1) after a complete interaction, and (2) during interaction when different amounts of gaze and action data are available. We built several classifiers using common machine learning algorithms and techniques, with feature sets that are based on (1) eye-tracking data only, (2) actions data only and (3) eye-tracking and actions data combined. The results we found show that eye-tracking data brings important information in predicting student’s performance for an ITS supporting SRL in both overall and over time analysis. The features used for training classifiers suggest that usage of SRL tools available in MetaTutor can be a good predictor of learning.

View record

Predicting affect in an Intelligent Tutoring System (2014)

In this thesis we investigate the usefulness of various data sources for predicting emotions relevant to learning, specifically boredom and curiosity. The data was collected during a study with MetaTutor, an intelligent tutoring system (ITS) designed to promote the use of self-regulated learning strategies. We used a variety of machine learning and feature selection techniques to predict students‘ self-reported emotions from eye tracking data, distance from the screen, electrodermal activity, and an ensemble of all three sources. We also examine the optimal amount of interaction time needed to make predictions using each source, as well as which gaze features are most predictive of each emotion. The findings provide insight into how to detect when students disengage from MetaTutor.

View record

The impact of individual differences on visualization effectiveness and gaze behaviour : informing the design of user adaptive interventions (2013)

Research has shown that individual differences can play a role in information visualization effectiveness. Unfortunately, results are limited given that there are so many individual differences that exist, and information visualizations are commonly designed without taking into account these user differences. The aim of this thesis is to investigate the impact of a specific set of individual differences (i.e., user characteristics) in order to identify which of these user differences have an impact on various aspects of information visualization performance. Eye tracking is also employed, in order to see if there is an impact of individual differences on user gaze behavior, both in general and for specific information visualization elements (i.e., legend, labels). In order to gather the necessary data, a user study is conducted, where users are required to complete a series of tasks on two common information visualizations: bar graphs and radar graphs. For each user, the following set of user characteristics are measured: perceptual speed, visual working memory, verbal working memory, and visualization expertise.Using various statistical models, results indicate that user characteristics do have a significant impact on visualization performance in terms of task completion time, visualization preference, and visualization ease-of-use. Furthermore, it is also found that user characteristics have a significant impact on user gaze behavior, and these individual differences can also influence how a user processes specific elements within a given information visualization.

View record

User modeling and data mining in intelligent educational games : Prime Climb a case study (2013)

Educational games are designed to leverage students’ motivation and engagement in playing games to deliver pedagogical concepts to the players during game play. Adaptive educational games, in addition, utilize students’ models of learning to support personalization of learning experience according to students’ educational needs. A student’s model needs to be capable of making an evaluation of the mastery level of the target skills in the student and providing reliable base for generating tailored interventions to meet the user’s needs. Prime Climb, an adaptive educational game for students in grades 5 or 6 to practice number factorization related skill, provides a test-bed for research on user modeling and personalization in the domain of education games. Prime Climb leverages a student’s model using Dynamic Bayesian Network to implement personalization for assisting the students practice number factorization while playing the game. This thesis presents research conducted to improve the student’s model in Prime Climb by detecting and resolving the issue of degeneracy in the model. The issue of degeneracy is related to a situation in which the model’s accuracy is at its global maximum yet it violates conceptual assumptions about the process being modeled. Several criteria to evaluate the student’s model are introduced. Furthermore, using educational data mining techniques, different patterns of students’ interactions with Prime Climb were investigated to understand how students with higher prior knowledge or higher learning gain behave differently compared to students with lower prior knowledge and lower learning gain.

View record

Prime Climb : an analysis of attention to student-adaptive hints in an educational game (2012)

Prime Climb is an educational game that provides individual support for learning number factorization skills in the form of hints based on a model of student learning. Previous studies with Prime Climb indicated that students may not always be paying attention to the hints, even when they are justified (i.e. based on a student model’s assessment). In this thesis we will discuss the test-bed game, Prime Climb, and our re-implementation of the game which allowed us to modify the game dynamically and will allow for more rapid prototyping in the future. To assist students as they play the game, Prime Climb includes a pedagogical agent which provides individualized support by providing user-adaptive hints. We then move into our work with the eye-tracker to better understand if and how students process the agent’s personalized hints. We will conclude with a user study in which we use eye-tracking data to capture user attention patterns as impacted by factors related to existing user knowledge, hint types, and attitude towards getting help in general. We plan to leverage these results in the future to make hint delivery more effective.

View record

 

Membership Status

Member of G+PS
View explanation of statuses

Program Affiliations

 

If this is your researcher profile you can log in to the Faculty & Staff portal to update your details and provide recruitment preferences.