Baker Learning Analytics Prizes
Problem 1: Transferability: The (learning system) Wall
- Transfer student model from learning system A to learning system B
- Improve an already-good student model in learning system B
- Change behavior of learning system B in runnable fashion
Problem 2: BLAP2. Effectiveness: Differentiating Interventions and Changing Lives
- Publicize criterion for intervention
- Assign students to control or experimental group
- Use analytics, only within experimental group, to assign intervention
- Collect longer-term outcome measure
- Demonstrate that experimental/analytics-intervention group performs better than experimental/analytics-no-intervention group
- But that experimental/analytics-no-intervention group does not perform better than control/analytics-no-intervention group
Problem 3: BLAP3. Interpretability: Instructors Speak Spanish, Algorithms Speak Swahili
- Take a complex model of a learning analytics phenomenon
- Develop a no-human-in-the-loop method of explaining the model
- Present the explanation to five (new) data scientists and users
- Ask the participants to explain what decision the model will make, and why, for five case studies
- Code the explanations of the model’s decisions
- Verify if the data scientists and users interpret the model the same way for the case studies
Problem 4. BLAP4. Applicability: Knowledge Tracing Beyond the Screen
- Take data from at least four students completing learning activity together
- Model at least four distinct skills for each student
- Predict immediate future performance for these skills
Problem 5. BLAP5. Generalizability: The General-Purpose Boredom Detector
- Build an automated detector of affect
- Demonstrate that the detector works for an entirely new learning system
- with different interactions
- with AUC ROC >= 0.65
Problem 6. BLAP6. Generalizability: The New York City and Marfa Problem
- Build an automated detector for a commonly-seen outcome or measure
- Collect a new population distinct from the original population
- Demonstrate that the detector works for the new population
- with degradation of quality under 0.1 (AUC ROC, Pearson/Spearman correlation)
- and remaining better than chance
