2016 Seed Grant for Promoting Pedagogical Innovation

Crowdsourcing for Crowd-learning: A New Online Tool for Advanced Peer Assessment

Abstract

The objective of this project is develop and test the effectiveness of a new online tool to enable crowdsourced peer assessment – the key conceptual piece in the new socio-technical pedagogical paradigm Crowdlearning that the PIs have been pioneering.

The vision of Crowdlearning is that of a self-sustaining problem-posing and problem-solving environment, where the students of a given subject intermittently take on the roles as (A) the creators of subject-focused problems (problem statements/formulations with answer alternatives, hints, correct answers with explanations, etc.), (B) evaluators of problem quality, and (C) problem solvers – all on a gamified online platform. The activities that the students perform in roles (A) and (B) are intended to be crowdsourced, resulting in a consensus-driven learning process, wherein the students create, and then, “vote in” the problems that help them learn, building a “bank” of subject matter problems to use for learning and assessment.

This project is framed as a set of implementation and assessment activities for building an evidential base for the PIs’ external grant applications to explore the Crowdlearning paradigm: as pointed out by the reviewers, an important concern underlying the value of these applications has been the uncertainty about the students’ ability to competently perform advanced peer assessment. Being a cornerstone of the PIs’ idea, the crowdsourced evaluations should be accurate, reliable, and, importantly, should inform the refinement of the problems that peers prepare to become part of the problem bank. Providing the students with a means to meaningfully contribute to each other’s creations – by exchanging constructive (anonymous) feedback and suggestions for problem improvement – is the need that this project aims to meet. The project plans for conceptual and technological developments to organize the process of online crowdsourced peer assessment: it proposes to develop the elements of the Crowdlearning website “crowdlearning.eng.buffalo.edu” and collect data in STEM classrooms to serve as preliminary evidence in support of further, larger research endeavors.

PI

  • Alexander Nikolaev
    Assistant Professor
    Department of Industrial and Systems Engineering

Co-PI

  • Suzanne Miller
    Professor
    Department of Learning and Instruction