Turnitin – Technology for Learning and Teaching in Engineering Blog

Prefer video over text? Watch this presentation instead (11 min)

Although rubrics can be useful in enhancing marking consistency (Jönsson & Svingby, 2007), as argued by Brookhart (2018, p.10): ‘The value of a rubric lies in its formative potential (Panadero and Jonsson, 2013), where the same tool that students can use to learn and monitor their learning is then used for grading and final evaluation by instructors.’

A recent meta-analysis conducted by Panadero, Jonsson, Pinendo and Fernandez-Castilla (2023) revealed a positive and moderate effect of rubrics on student academic performance, self-regulation and self-efficacy. Rubrics have been shown to support student ability to establish more accurate goals, monitor progress and reduce cognitive load thus contributing to self-regulated learning (Brookhart & Chen, 2015; Panadero & Jonsson, 2013; Krebs et al., 2022; Reddy & Andrade, 2010). As argued by the meta-analysis’ authors: ‘By making criteria, performance levels, and (when relevant) scoring strategies explicit, these may become objects of action and reflection themselves (i.e., students can use them to regulate their learning) helping students to improve their learning via self and peer assessment (Nicol, 2021; Panadero et al., 2019). This interpretation is supported by students, who are generally positive about being provided with rubrics and claim to use the rubrics to better understand (and meet) expectations (e.g., Andrade & Du, 2005; Jonsson, 2014; Reynolds-Keefer, 2010).’ (Panadero, Jonsson, Pinendo, & Fernandez-Castilla, 2023, p.113).

This ‘formative potential’ of rubrics differentiates it from quality assurance tools such as the University Generic Marking Criteria. The effectiveness of rubrics as a learning tool lies in clear, meaningful and specific criteria (Brookhart, 2013; Nitko & Brookhart, 2007; Popham, 2000; Suskie, 2009) and performance level descriptors referring to observable and measurable qualities which ‘help students envision where they are in their learning and where they should go next’ (Bookhart, 2018, p.2). As defined by Brookhard (2018, p.1):

‘A rubric articulates expectations for student work by listing criteria for the work and performance level descriptions across a continuum of quality.’

The following steps, designed by van Leusen (2013) and adapted by Arcuria and Chaaban (2019) offer a useful framework for designing rubrics:

What knowledge and skills is the assignment designed to assess? (Learning Objective)
What observable criteria represent that knowledge and skills?(Performance Criteria)
How can you best divide those criteria to represent distinct and meaningful levels of student performance? (Performance Levels)
What observable characteristics of students’ work differentiate among the performance levels for each criterion? (Performance Level Descriptors)

The University of Tasmania provides three excellent examples of rubrics worth looking at – Writing Standards Descriptors (for rubrics)

A screenshot of the 'Complete Rubrics - Downloadable' tab

Using Rubrics in Blackboard

Rubrics are available both in Turnitin as well as Blackboard Assignments. They can also be associated with other gradable items in Blackboard such as essay, short answer, and file response test questions, blogs and journals, wikis, discussion forums and threads.

Turnitin Rubrics

Rubrics types

Blackboard Rubrics

Rubrics types

In Blackboard Ultra, you will have the option to generate a rubric based on the course content using the AI Design Assistant.

References:

Arcuria, P., & Chaaban, M. (2019). Best Practices for Designing Effective Rubrics. ASU TeachOnline. Best Practices for Designing Effective Rubrics – Teach Online (asu.edu)

Banta, T. W., & Palomba, C. A. (2015). Assessment essentials: planning, implementing, and improving assessment in higher education. Jossey-Bass.

Brookhart, S. M. (2013). How to create and use rubrics for formative assessment and grading. Association for Supervision & Curriculum Development.

Brookhart, S. M. (2018). Appropriate criteria: Key to effective rubrics. Frontiers in Education 3, 22. https://doi.org/10.3389/feduc.2018.00022

Brookhart, S. M., & Chen, F. (2015). The quality and effectiveness of descriptive rubrics. Educational Review, 67(3), 343-368. https://doi.org/10.1080/00131911.2014.929565

Campbell, A. (2005). Application of ICT and rubrics to the assessment process where professional judgement is involved: the features of an e‐marking tool. Assessment & Evaluation in Higher Education, 30(5), 529-537. https://doi.org/10.1080/02602930500187055

Jönsson, A., & Svingby, G. (2007). The use of scoring rubrics: Reliability, validity and educational consequences. Educational Research Review, 2(2), 130–144. https://doi.org/10.1016/j.edurev.2007.05.002

Krebs, R., Rothstein, B., & Roelle, J. (2022). Rubrics enhance accuracy and reduce cognitive load in self-assessment. Metacognition and Learning, 17(2), 627-650. https://doi.org/10.1007/s11409-022-09302-1

Nicol, D. (2021). The power of internal feedback: Exploiting natural comparison processes. Assessment & Evaluation in higher education, 46(5), 756-778. https://doi.org/10.1080/02602938.2020.1823314

Nitko, A. J., & Brookhart, S. M. (2007). Educational Assessment of Students (5th ed.). Pearson Education.

Panadero, E., Broadbent, J., Boud, D., & Lodge, J. M. (2019). Using formative assessment to influence self-and co-regulated learning: the role of evaluative judgement. European Journal of Psychology of Education, 34, 535-557. https://doi.org/10.1007/s10212-018-0407-8

Panadero, E., & Jonsson, A. (2013). The use of scoring rubrics for formative assessment purposes revisited: A review. Educational research review, 9, 129-144. https://doi.org/10.1016/j.edurev.2013.01.002

Panadero, E., Jonsson, A., Pinedo, L., & Fernández-Castilla, B. (2023). Effects of rubrics on academic performance, self-regulated learning, and self-efficacy: a meta-analytic review. Educational Psychology Review, 35(4), 113. https://doi.org/10.1007/s10648-023-09823-4

Popham, W. J. (2000). Modern educational measurement: Practical guidelines for educational leaders (3rd ed.). Allyn and Bacon.

Reddy, Y. M., & Andrade, H. (2010). A review of rubric use in higher education. Assessment & evaluation in higher education, 35(4), 435-448. https://doi.org/10.1080/02602930902862859

Suskie, L. (2009). Assessing student learning: A common sense guide (2nd ed.). Jossey-Bass.

Wolf, K., & Stevens, E. (2007). The role of rubrics in advancing and assessing student learning. Journal of Effective Teaching, 7(1), 3-14.

van Leusen, P. (2013). Assessments with rubrics. ASU TeachOnline. https://teachonline.asu.edu/2013/08/assessments-with-rubrics/

Prefer video over text? Watch this presentation instead: Using Turnitin Similarity Reports as Feedback (10 min)

How do you use Turnitin Similarity Reports in your teaching? Commonly the Turnitin similarity checker is seen as the ‘plagiarism detector’ and although it can be a useful indicator of whether or not unacceptable academic practices occurred, reducing its role to a purely punitive one is a huge missed opportunity.

If we look at the resources from the tool’s provider, their recommendations advocate for a more pedagogical approach: ‘One of the most effective ways to utilise the Turnitin Similarity Report is to guide students to make improvements in their writing and research practices.’ (Understanding the Similarity Report: An educator guide | Turnitin). They even provide a student guide to understanding the Similarity Report which starts with: ‘Continue reading to better understand what this information means and how you can use it to improve your writing.’ (Understanding the Similarity Report: A student guide | Turnitin).

As suggested by Mphahlele & McKenna (2019, p. 3): ‘‘Much of the literature in favour of such software argues that its most effective use is as a pedagogical tool, rather than a policing one. It is shown to add an enormously positive aspect to the development of student writing when used in such a way. ’ Interestingly, some experimental studies showed a positive impact of using Similarity Reports along with clear and specific guidelines and paraphrasing exercises on reducing plagiarism, suggesting that Similarity Reports could be used in its prevention rather than detection. The research indicates that students are more likely to internalise guidelines on acceptable academic practices when they refer explicitly to their own work (Barrett & Malcoln, 2006). As discussed by David and Carroll (2009, p.6): ‘Many students seemed to have a kind of ‘eureka’ moment, when faced with the onscreen evidence of how they had used sources, where they understood more fully about issues related to academic integrity, as they connected to their own work.’ Using Similarity Reports as the reference point, students can engage in comparison processes, underlying the generation of valuable internal feedback (Nicol, 2021).

The most effective ways of utilising Turnitin Similarity Reports seem to involve an opportunity for students to discuss their reports with staff. Where this is not possible on an individual basis, David and Carroll (2009, p.67) suggested small group tutorials with examples highlighting typical mistakes. If a fully self-directed design is applied, clear guidelines are key (Rolfe, 2011). It has been also suggested that limiting the similarity report generation to one attempt may minimise the risk of system evasion (McKeever, 2006). A potential implementation may look like this:

Clearly labelled folder on Blackboard included in the Assessment and Feedback area:

A folder on Blackboard

Step-by-step instructions with links to further resources:

A crucial factor in the effectiveness of this approach is clear signposting from the tutor. Many other institutions in the sector seem to follow similar practices:

If you are interested in adopting this approach or you like to share your way of using Similarity Reports, please get in touch: ania.udalowska@bristol.ac.uk. To conclude, I would like to second this quote from Kaktiņš, (2019, p. 432):

‘Overall, it appears that the use of text-matching software in its most authentic iteration is most effectively implemented as part of a holistic approach (Lee & Edwards, 2013) focused on formative learning and the development of students’ responsibility for their work, plus disciplinary measures where required.’

References:

Barrett, R., & Malcolm, J. (2006). Embedding plagiarism education in the assessment process. International Journal for Educational Integrity, 2(1), 38-45. https://doi.org/10.21913/IJEI.v2i1.23

Davis, M. and Carroll, J. (2009). Formative feedback within plagiarism education: Is there a role for text-matching software? International Journal for Educational Integrity, 5(2), 58-70. https://doi.org/10.21913/IJEI.v5i2.614

McKeever, L. (2006). Online plagiarism detection services—saviour or scourge? Assessment & Evaluation in Higher Education, 31(2), 155–165. https://doi.org/10.1080/02602930500262460

Mphahlele, A., & McKenna, S. (2019). The use of Turnitin in the higher education sector: Decoding the myth. Assessment & Evaluation in Higher Education, 44(7), 1079–1089. https://doi.org/10.1080/02602938.2019.1573971

Nicol, D. (2021). The power of internal feedback: exploiting natural comparison processes. Assessment & Evaluation in Higher Education, 46(5), 756–778. https://doi.org/10.1080/02602938.2020.1823314

Rolfe, V. (2011). Can Turnitin be used to provide instant formative feedback? British Journal of Educational Technology, 42(4), 701–710. https://doi.org/10.1111/j.1467-8535.2010.01091.x

Turnitin. (n.d.). Understanding the Similarity Report: A student guide. Turnitin. https://www.turnitin.com/papers/understanding-the-turnitin-similarity-report-student-guide

Turnitin. (n.d.). Understanding the Similarity Report: An educator guide. Turnitin. https://www.turnitin.com/papers/understanding-the-turnitin-similarity-report-instructor-guide

Category: Turnitin

Using Rubrics to Support Self-Regulated Learning

Using Turnitin Similarity Reports as Formative Feedback