Peer evaluation with graph neural networks

Peer assessment systems are rising in various social contexts, such as peer grading in large (online) classrooms, peer review in conferences, peer art evaluation, etc. However, peer assessments might not be as accurate as expert assessments, thus rendering these systems unreliable. Peer assessment systems’ reliability is influenced by factors such as peers’ assessment ability, manipulation and strategic assessment behaviors, and the peer assessment setup (e.g., peer assessing group work or individual work of others). In this work, we first model peer assessment as multi-relational weighted networks that can represent a variety of peer assessment setups, and capture conflicts of interest and strategic behaviors. Leveraging our peer assessment network model, we introduce a graph convolutional network which can learn assessment patterns and user behaviors to more accurately predict expert evaluations. Our extensive experiments on real and synthetic datasets demonstrate the efficacy of our proposed approach, which outperforms existing peer assessment methods.

Name thesis:degree_name

Master of Science (MSc)

Discipline thesis:degree_discipline

Computer Science

Grantor

University of Ontario Institute of Technology

Year dc:date.issued

2021

Author dc:creator

• Namanloo, Alireza A.

Advisors dc:contributor.advisor

• Salehi-Abari, Amirali
• Thorpe, Julie

• Peer assessment
• Graph neural network

Language dc:language.iso

en

Handle dc:identifier.uri

https://hdl.handle.net/10155/1406

OAI identifier oai:identifier

oai:ontariotechu.scholaris.ca:10155/1406