Buddycheck Feature Update June 2022

Posted on: 20 June 2022 by Will Moindrot in General

Buddycheck is a peer evaluation tool used to help students to feedback and reflect upon the contribution of themselves and their peers towards groupwork tasks. This can be used to make assessment of groupwork fairer through increased accountability and peer-moderation of grades. It can also be used formatively as a prompt for reflection and source of feedback leading to development of teamwork skills. Buddycheck is intergrated into Canvas, and available for all staff at the University of Liverpool to use.

As we enter into the second year of using Buddycheck, two new features have been made available. ‘Team Questions’ can be a useful way for tutors to sample student opinion about their group make-up and performance: this could be used as a milestone towards an assessed task, maybe as a pre-cursor to group formation activities. The second feature ‘Motivations’ can be used to improve the quality of feedback by requiring students to provide textual reasoning for their feedback, encouraging deeper critical reflection. Both of these features aim to extend and deepen how Buddycheck can be used in groupwork. 

Team Questions

In response to user demand the 'Team Questions' feature of Buddycheck has now been enabled by IT Services. Alongside the existing peer questions, Team Questions can now be used to prompt students to feedback about their team performance and working as a whole. Team Questions do not feed into the adjustment factor.

This feature has been very popular at other universities, adding an extra dimension to the evaluation process, and providing a useful visualisation for staff and students. It has been suggested that an evaluation featuring Team Questions could be used early on in a group task to help stage appropriate support interventions (see Newcastle University’s resource on Buddycheck).

There are two pre-written Team Questions that can be added and used out-of-the-box, or you can create your own.

(The Buddycheck evaluation screen for students, highlighting one of the new Team Questions.)

Currently available pre-written Team Questions are:

Team Satisfaction: 5-point Likert (Strongly Disagree - Strongly Agree):

• I am satisfied with my present teammates
• I am pleased with the way my teammates and I work together
• I am very satisfied with working in this team

Team Conflict: 5-point Likert (None or Not at all; Little or Rarely; Some; Much or Often; Very Much or Very Often)

• How much conflict of ideas is there in your work group?
• How much emotional conflict is there in your work group?
• How much relationship tension is there in your work group?
• How often do people get angry while working in your group?
• How much conflict is there in your group about task responsibilities?
• How often are there disagreements about who should do what in your work group?
• How often do you disagree about resource allocation in your work group?
• How frequently do you have disagreements within your work group about the task of the project you are working on?
• How often do people in your work group have conflicting opinion about the project you are working on?

To add one of the pre-written Team Questions to an evaluation:

1. At the evaluation setup stage in Buddycheck click on the Select from library button
2. Add chosen questions from the list of Team questions 

To create a new Team Question (for example converting previous activity to Buddycheck):

1. At the evaluation setup stage in Buddycheck click on the Create question button
2. At the Create new question screen, select the Team question, question type and fill in the necessary details for the question
3. Once created the question can be reused in new evaluations, or even added to institutional templates (please contact IT Services servicedesk@liverpool.ac.uk)
   
   

(The creation of a new Buddycheck Team Question. With questions under the category "How well was your team able to respond to change?". Individual questions within this category include "How well the team was able to respond to feedback", "How ready the team was to change plan", and "How much it had anticipated change in its original planning". The setup box shows that the instructor requires students to respond via a 5-stage Likert scale from Strongly Disagree to Strongly Agree.)

Data from responses to Team Questions is available immediately to the instructor within the Buddycheck Evaluation overview screen against each student, showing what the student gave against the group average. Data is also presented back to students in their Personal Report (if used). The raw data can also be downloaded as Excel/CSV to produce stronger reporting to aid planning or research.

(The output data from a Buddycheck evaluation, focusing on Team Questions data. A small table of data, with team questions for each row, corresponding to two columns for the score that the student gave for that question, and also the group average of the same question.)

Motivations: An existing feature you may not have experimented with yet

When setting Peer Questions in Buddycheck it is possible to require the student to provide a short textual reason for their rating for the student they are evaluating, i.e. to explain why the rating is appropriate. This can be a required field (they must leave an explanation) or optional for the student to complete. Evidence in the research literature suggest that increased accountability through prompts such as for motivations can increase the quality of peer feedback (for example see Patchan et al. 2017).

With the setting enabled, when students click on a rating for themselves or a fellow student a pop-up window appears asking them to provide a motivation for the score. A box also appears at the bottom of the screen (presumably to aid accessibility) and text entered will appear in both places.

(Buddycheck evaluation screen - filling out an evaluation. It highlights the new Motivations text fields that the student is required to complete in this evaluation.)

Data collected from the motivations are instantly available to the instructor through the evaluation overview screen by clicking on the speech bubble icons next to a rating.

(The Buddycheck data that instructors can access to view the motivations left by students for other students.)

For technical help in getting started with Buddycheck please see the Knowledge Base support resources produced by IT Services: Buddycheck Knowledge Base resources. 

For advice and consultation about your overall approach of peer evaluation or use of Buddycheck please contact CIE’s Digital Education team digitaled@liverpool.ac.uk.

Reference

Melissa M. Patchan, Christian D. Schunn & Russell J. Clark (2017): Accountability in peer assessment: examining the effects of reviewing grades on peer ratings and peer feedback, Studies in Higher Education, DOI: 10.1080/03075079.2017.1320374