摘 要

写作项目是高校英语考试重要组成部分，而写作评分的质量直接关系到写作测试的效度与信度。本研究以 9 名英语专业大四学生和 5 名英语专业教师为研究对象，借助SPSS 软件并基于多层面 Rasch 模型，对比分析大学生英语作文教师评分与同伴互评的可行性与有效性。研究结果表明：（1）同伴互评条件下，学生自身英语水平的高低对评分结果有显著影响；（2）从总体宽严度、自身一致性等指标上看，两类评分员的评分质量存在差异，教师评分员的评分质量高于学生评分员的评分质量；（3）从总层面上看，学生评分员与教师评分员的评分结果具有一致性，对于未来基于课堂的大学英语写作评估而言，同伴互评可以作为教师评价的辅助性手段参与写作教学，使写作评分更加合理、科学。

关键词：英语写作；教师评分；同伴互评；多层面Rasch 模型

Contents

1. Introduction 1
2. Literature Review 2
   1. External research 2
   2. Domestic research 4
3. Methodology 5
   1. Research questions 5
   2. Participants 6
   3. Data collection 7
   4. Data analysis 9
4. Results 10
   1. Analysis of difference in proficiency level of peer raters 10
   2. Analysis of rating quality of peer and teacher raters 12
      1. Analysis of rating quality of peer raters 12
         1. Overall analysis 12
         2. Analysis from the perspective of raters 14
      2. Analysis of rating quality of teacher raters 15
         1. Overall analysis 15
         2. Analysis from the perspective of raters 17
   3. Further comparison of peer amp; teacher raters’ rating results 18
      1. Comparison of scores of language use 18
      2. Comparison of scores of ideas and arguments 18
      3. Comparison of total scores 19
5. Discussions 19
   1. Influence of proficiency level of peer raters on rating marks 19
   2. Difference in rating quality of peer raters and teacher raters 21
   3. Consistency in rating results of peer raters and teacher raters 23
6. Conclusion 24

Bibliography 26

A Classroom-Based Comparative Study on Teacher Rating and Peer Rating in College English Writing

Introduction

College English writing is an important component of the teaching system of college English, which has been in a relatively lagging state of high input but low output due to the traditional mode of teaching and learning, where students have been regarded as objects to passively acquire knowledge. Consequently, they have a low degree of involvement and little enthusiasm in class. As for evaluation methods of English writing, traditional evaluation system of college English writing is fairly one-fold. Over the years, many colleges take teacher assessment as the sole evaluation method. In this mode, the teaching of college English writing is faced with great challenge, in other words, teachers spend much time and energy in evaluating students’ compositions, while students have little interest in making detailed revision after teacher feedback, therefore, the quality and ability of students’ English writing are still far from satisfaction.

When it comes to the rating process of college students’ English writing, it is necessary to apply multiple and flexible teaching methods so as to help students fully release their potential abilities in English writing. As the learner-centered teaching theory is put forward, more and more people have realized the importance of students’ participation in writing evaluation. Evaluation is an important part in learning process. If students can take part in evaluation process, they will not only change their passive role in traditional assessment process, but can be inspired by rating others’ compositions and reflect on their shortcomings in English writing, thus improving their proficiency level in English writing. On account of this, peer rating is regarded as a major way of students participating in the evaluation and applied to daily writing teaching. Thereby, it is necessary and feasible to introduce peer rating into the process of writing rating. The reason is that peer rating is beneficial for teachers and students

to improve mutual positivity, thus achieving better teaching efficiency in the college English writing class.

As for classroom-based writing teaching, reliable rating can provide diagnostic feedback for teaching and offer foundation to daily instruction decisions and practices. With regard to teaching English writing, peer rating can be used to promote students’ autonomy (Dheram, 1995). However, rating writing is a kind of subjective evaluation in essence. It is impossible for every rater to give the same marks for the same composition. Hence, it’s unclear whether all raters could well grasp the basic principles of rating scales and give relative marks that can truthfully reflect writers’ actual writing abilities. Therefore, this study aims to investigate the feasibility and validity of rating English writing by comparing teacher rating and peer rating based on quantitative analyses. The results of the study, hopefully, can offer some insights into the quality of the two kinds of ratings above and investigates the feasibility of the participation of peer rating in teaching English writing in Chinese colleges.

Literature Review

• 1. External research

A bulk of researches on the second language writing of rating have been conducted by foreign researchers. Teacher rating is considered as the most essential part of writing process, which has been occupying a central position in the research area of L2 writing evaluation. The research of Ferris et al (1997) show that, “Teachers can always keep the consistency in the process of rating all students’ compositions, in the meanwhile, writing teachers are able to carefully read and revise students’ compositions all the same.” Tsui amp; Ng (2000) conducted a study in a secondary school in Hong Kong, which finds that students prefer to choose teacher rating because they deeply respect teachers’ authority, meanwhile, teachers can clearly point out practical problems in their compositions. However, not all research results hold a positive attitude toward teacher rating, and some scholars are still skeptical to the validity of it. Purves (1984) thinks that the behavior of teacher rating is too mechanical, and the comments of teachers are beyond students’ understanding with

less help.

With the further development of higher education, alternatives in rating have received much attention in the last decade and several forms of rating have been introduced into higher education (Birenbaum amp; Dochy, 1996). Peer rating has become a part of the college English writing classes. In view of the wide use of peer rating, especially in higher education, the relative accuracy of peer ratings compared to teacher ratings has become a major concern for both educators and researchers. Previous studies on peer assessment showed consistent levels of agreement between peer and teacher ratings. The results revealed no significant difference between the learners’ peer assessment and teacher assessment (Azarnoosh, 2013). In addition to students benefiting from being graded by their peers, graders also benefit because it gives them experience in evaluating others, which is an increasingly important skill (Mainkar, 2008). Meanwhile, the results of peer rating often have high correlation with teacher rating (Pope, 2005). Falchikov and Goldfinch (2000) also found the weighted correlation coefficient between peer ratings and teacher ratings to be .69, which is moderately strong. However, despite there is great potential in peer rating,, researchers and practitioners remain skeptical about whether students have the ability to assign reliable and valid ratings to their peers’ work (Liu amp; Carless, 2006). Empirical research on consistency demonstrated that peer evaluation lack reliability and validity (W. Dancer amp; J. Dancer, 1992). Freeman (1995) holds that L2 students have limited linguistic knowledge, and they could not fully find out and revise the mistakes in target language, thus exerting negative influence on the rating validity. Partridge (1981) and Nelson amp; Murphy (1993) also doubt the reliability and accuracy of peer rating. For instance, some studies found low correlations between peer and teacher ratings, such as 0.29 (Kovach, 2009). The empirical study of Mangelsdorf (1992) finds that many students do not trust peer ratings, and they think that peer raters have no ability to give reasonable evaluation to their compositions. While students reported a better understanding of quality in student writing as a result of their experience, many complained that peer assessors’ marks were not “fair” ( McConlogue, 2012). Sengupta’s (1998) research shows that under the exam-oriented