The Effect of Assessment Type on EFL Learners’ Goal-Orientation
Abbas Ali Zarei1, *, Zahra Usefli2
1English Department, Imam Khomeini International University, Qazvin, Iran
2English Department, Islamic Azad University, Qazvin, Iran
Abstract
The present study was conducted to investigate the effects of self, peer, and teacher-assessment on EFL learners’ goal-orientation. The participants were 94 male and female Iranian EFL learners at IT English language institute in Qazvin. The instruments included a 55-item Preliminary English Test (PET), and the Persian translation of an 18-item goal-orientation. The participants were divided into three groups, each group receiving one of the treatment conditions (self, peer, and teacher-assessment). They were also given the questionnaire twice, once before the treatment and once after it. The collected data were analyzed using ANCOVA procedure. The results of the study showed significant differences among the effects of types of assessment on goal-orientation. Self-assessment turned out to be more effective than both peer and teacher-assessment on goal-orientation. However, the difference between peer and teacher-assessment in goal-orientation was not statistically significant. The findings of this study may have theoretical and pedagogical implications for learners, teachers and syllabus designers.
Keywords
Self-Assessment, Peer-Assessment, Teacher-Assessment, Goal-Orientation
Received: June9, 2015
Accepted: June 17, 2015
Published online: July 15, 2015
@ 2015 The Authors. Published by American Institute of Science. This Open Access article is under the CC BY-NC license. http://creativecommons.org/licenses/by-nc/4.0/
1. Introduction
Several researchers have claimed that learner-centered learning strategy use in EFL educational systems has received considerable attention in recent decades (Reiss, 1985; Tamada, 1996). In addition, among learners' personality traits, self-efficacy beliefs such as self-regulation and goal-orientation have been the subject of a considerable body of research (Bandura, 1986, 1989; Pintrich & Schunk, 1995; Zimmerman, 2000). According to Zimmerman (2000), Albert Bandura, a cognitive psychologist, introduced self-efficacy for the first time in 1977 and referred to it as personal beliefs about one's capabilities to perform the required actions to achieve assigned quality in educational accomplishment. Recently, many researchers have attempted to figure out the relationship between self-efficacy beliefs and learners’ performance. One concept in this area is goal-orientation. According to Ames (1992), goal-orientation is a way of reaching mastery and responding to learning tasks through proper thinking in mind to achieve an assigned goal. Actually, learners with this strategy will be more engaged in the activity and will concentrate on their progressive stages to achieve previously assigned outcomes.
Goal orientation may be influenced by several factors. One of the factors that may influence goal orientation, and which is of interest in the present study is the type of assessment. There are several types of assessment, including self-assessment, peer-assessment, and teacher-assessment.
Several studies have investigated the effects of different types of self-efficacy such as self-regulation, goal-orientation, and achievement motivation on learners' performance (Bandura, Barbaranelli, Caprara, & Pastorelli, 1996, 2001; McCarthy, Meier, & Rinderer, 1985; Zimmerman, Bandura, & Martinez- Pons, 1992), as well as the effects of self (Brown, 2005; Butler & Lee, 2010; Chen, 2008, Falchikov & Boud, 1989; Leach, 2012), peer (Cheng & Warren, 2005; Falchikov, 1995; Freeman, 1995), and teacher-assessment (Chacon, 2005; Hoy & Davis, 2006) on learners' performance. However, few studies appear to have been conducted on the relationship between goal-orientation as a psychological factor and different types of assessment (self, peer, and teacher-assessment) (Alfallay, 2004). Therefore, the purpose of the present study is to investigate the effects of self, peer, and teacher-assessment on Iranian EFL Learners’ goal-Orientation. More specifically, this study aims to address the following research question:
Are there any significant differences among the effects of self, peer, and teacher-assessment on Iranian EFL learners' goal-orientation?
2. Literature Review
2.1. Self, Peer, and Teacher-Assessment
Assessment is an essential component of, and a necessary complement to, almost all educational programs without which the successful completion of the program is hardly imaginable. Assessment may take place in the form of self, peer, or teacher assessment. Boud (1992) and Boud and Falchikov (1989) refer to self-assessment as students' reflection and engagement in their own work in classroom activities to evaluate their own performance. Falchikov (1995) defines peer-assessment as the process by which learners assess their classmates' activities against established criteria. Brown and Hudson (1998) see teacher-assessment as the traditional system in which teachers are responsible for students' performance assessment.
Assessment causes learners to gain information about their capabilities and weaknesses in learning process, especially by actively engaging them in their own or peer-assessment. Learners will be informed about the assigned learning goals, and by taking feedback, they will benefit from self-evaluating and self-assessing (Black & William, 1998; Davies, 2002; Shepard, 2000). Butler and Lee (2010) claim that that self-assessment has a positive effect on learners' English learning and self-confidence.
Several studies have investigated various aspects of assessment. In one such study, Sluijsmans, Dochy, and Moerkerke (1999) investigated the effects of self, peer and co-assessment on learners’ performance. The results showed that the salient dimension of self-, peer-, and co-assessment is to assign criteria to evaluate learners' performance. They claimed that using criterion-referenced testing instead of the norm-referenced testing in learner-centered educational contexts is more effective.
Tsui and Ng (2000) compared the effects of peer and teacher assessment on learners’ development of the writing skill, and concluded that peer assessment is more effective than teacher assessment. However, Heywood (2000) reported peer assessment as being less effective than teacher assessment on L2 learning.
In another study, Keig (2000) studied faculty members’ attitudes towards colleague assessment. The study aimed at determining which method of peer-assessment (classroom observation, videotaping of classes, evaluation of course materials, and evaluation of their academic work) would faculty choose to improve their teaching. The results showed that the faculty expressed willingness to participate in all these types of assessment. However, they had more positive attitudes towards evaluation of the course book than to classroom observation.
Other studies have looked into students' feeling about the application of peer-assessment approach and teacher trainee’s opinion about the effectiveness of peer-assessment. The results have generally showed that teacher trainees consider peer-assessment as a useful method that persuades learners to take part in assessment and critically evaluate their peers’ assigned work, and that this is significantly related to their gender (McLaughlin & Simpson, 2004; Karaca, 2009).
Ross (2005) investigated how assessment can affect learners’ foreign language proficiency. The results showed that formative assessment has more effect on learners' language learning, especially on their listening comprehension improvement.
Sluijsmans and Prins (2006) investigated a conceptual framework for integrated peer-assessment in teacher education through two studies. The results showed a positive relationship between the participants' peer-assessment tasks and their general improvement in their learning performance.
Andrade and Du (2007) studied student responses to criterion-referenced self-assessment and concluded that students showed positive attitudes toward criterion-referenced self-assessment with no evidence of gender differences. In other study, Leach (2012) studied optional self-assessment. The study used a form of criterion-referenced assessment to assess the participants' assignments. The results of the study showed no significant difference between students' self-assessed grades and teacher-assessed grades.
Xiao and Lucking (2008) investigated the effect of two types of peer-assessment on students' performance and satisfaction within an online environment. The results showed a positive relationship between the participants, in the experimental group, and a high level of improvement in writing as well as a high level of satisfaction with the peer-evaluation method compared with the comparison group. In another study, Pare and Joordens (2008) investigated peer-assessment and expert mark agreement through peer Scholar system (an online peer-assessment tool). The results indicated a significant positive agreement among expert markers and between expert and peer markers.
White (2009) viewed peer-assessment among learners as the process by which they take the responsibility of evaluating their peers’ performance through their own capabilities with regard to assigned goal and its effect on their relationship with their peers. The study confirmed the participants’ positive attitude and feeling of satisfaction towards peer-assessment. In another study, Karaca (2009) studied teacher trainee’s opinion about the effectiveness of peer-assessment and the effect of variables such as gender, previous experience in peer- assessment. The results showed a significant positive relationship between the teacher trainees’ attitude toward peer-assessment and giving feedback on their peers’ performance. A significant gender difference was also reported with regard to the learners’ attitude toward peer assessment.
Shamir, Mevarech, and Gida (2009) investigated the effectiveness of assessing kindergarten children's metacognition in different contexts such as individual learning, peer-assisted learning, and self-report. The results showed significant differences between children's self-report and their procedural metacognitive performance in individual learning and peer-assisted learning.
Chen (2010) studied the implementation and evaluation of a mobile self and peer-assessment system. The results showed that the participants' attitude affects both their mobile assessment participation system and its implementation. However, a positive relationship between learners' involvement in peer-assessment and improvement in their quality of assessment has been reported by several researchers (Kao, 2012; Li, Liu & Zhou, 2011; Tillema, Leenknecht, & Segers, 2011).
Wever, Keer, Schellens, and valcke (2011) investigated the reliability of peer-assessment in an online environment. They concluded that peer-assessment has a high level of reliability. Esfandiari and Myford (2013) sought to investigate the differences among self, peer, and teacher-assessors on their scoring of EFL essays. The results showed self-assessment was the most lenient one while teacher-assessment was the most severe ones.
Zarei and Sayar Mahdavi (2014) investigated the effect of peer and teacher-assessment on learners' grammatical and lexical writing accuracy. The results suggested that peer-assessment was significantly more effective than teacher-assessment.
2.2. Goal-Orientation
According to Pintrich (2000), goal-oriented learners are mostly concerned with the outcomes of the learning process, and their focus is on the aims of learning through continuing their assigned task. Tercanlioglu (2004) studied 135 EFL learners' goal-oriented beliefs. Results of the study showed that the participants mostly focused on the goals of the task, rather than other factors, which showed their beliefs about their language achievement. Moreover, he concluded found a negative relationship between task goals and performance-avoid orientation. In addition, there was a positive relationship between performance-approach goal-orientation and performance-avoid orientation.
Shih (2005) believed that students who adopted goal-oriented beliefs benefited from their cognitive and metacognitive strategies. Radosevich, Vandanu, Yeo, and Deirdre (2004) conducted a longitudinal study to investigate the relationship between self-regulated learning and goal-orientation of 132 students. The results showed a positive relationship between learning goal-orientation and participants' assigned goals and their cognitively engaged self-regulation. In addition, a negative relationship was found between performance-avoid goal-orientation and cognitive self-regulation.
Mirhassani, Akbari and Dehghan (2007) studied the relationship between Iranian EFL learners' goal-orientated and self-regulated learning and their language proficiency. They hold that there are three kinds of achievement goal-orientation including mastery goal-orientation, referring to the focus on improving learners’ own task competence as an end, ability-approach goal-orientation, referring to learners’ own ability and their desire to be at a higher level than others do, and ability-avoid goal-orientation, which refers to learners' tendency to avoid describing their own weak points in that ability. The results showed that goal-orientation played a significant role in learners' language proficiency, and that there was a significant relationship between task goal-orientation and language proficiency. However, there was no significant relationship between ability-approach and ability-avoid goal-orientation and language proficiency. They also reported a significant relationship between self-regulated learning and language proficiency.
Sadeghy and Mansuri (2014) investigated the relationship between learners' goal-orientation, self-regulated learning and the use of different language learning strategies. The participants of the study were 125 male and female college students. The results showed a significant relationship between goal-orientation and language learning strategies, and between mastery, performance goal-orientation and language learning strategies. In addition, they concluded that there was a significant relationship between self-regulation and language learning strategies.
In another study, Zarei and Gilanian (2014) investigated language learning strategies as predictors of goal-orientation. The participants were 148 homogenized English translation and English teaching university students. In order to obtain data, they utilized the strategy inventory for language learning and motivated strategies for learning questionnaires. The collected data were analyzed through multiple regression analyses. The results showed significant relationships between language learning strategies and cognitive self-regulated learning components. In addition, they concluded that the only predictor of extrinsic goal-orientation was affective strategies. Meanwhile, they reported that affective, metacognitive, and compensation strategies and task goal-orientation had statistically significant relationships with each other.
To sum up, although several studies have been done on various aspects of self, peer, and teacher-assessment as well as the effect of assessment type on different aspects of language learning, there appears to be a paucity of research on the effect of the type of assessment on students’ feelings of goal-orientation. Therefore, the aim of this study is to investigate the effects of self, peer, and teacher-assessment on the goal-orientation.
3. Method
3.1. Participants
The participants of the present study were a sample of 94 male and female intermediate level Iranian EFL students between the ages of 19 to 28 at IT English language institute in Qazvin. All the participants had been studying English for more than four years. After these years of study, they were expected to be familiar with types of assessment. Still, to ensure full understanding, the key elements were explained in their native language.
3.2. Instruments
The present study made use of two instruments to collect data including the following:
1. Preliminary English Test (PET)
2. Goal-oriented questionnaire
3.2.1. Preliminary English Test (PET)
A PET was used to determine the participants’ level of proficiency and to homogenize them before starting the new semester at the institute. PET is a standard test to determine intermediate students’ level of proficiency. The version of the PET used in the present study included 55 multiple-choice items, 30 grammatical items, and 25 vocabulary items. The test also consisted of four types of reading comprehension questions including matching, true-false, comprehension questions and a text with gaps.
3.2.2. Goal-Orientation Questionnaire
The goal-orientation questionnaire used for the purpose of this study was Midgley, Kaplan, Middleton and Maehr’s (1998) questionnaire consisting of 18 items, every 6 items measuring a different goal-orientation component including: task goal orientation, ability-avoid goal-orientation, and ability-approach goal-orientation. The students were asked to report on a scale of 1 to 5 (strongly disagree to strongly agree).The questionnaire was taken from Zarei and Gilanian (2014). To facilitate the participants’ performance, the translated version was utilized. To estimate the reliability of the instrument in the context of the present study, Cronbach’s Alpha was checked; it turned out to be 0.79 (α = 0.79).
3.3. Procedure
To collect the required data, the following steps were followed:
In the first stage, the participants who were studying English in an institute in Qazvin were selected. To prevent the participants' confusion and to remove any possible source of anxiety, all of the participants were informed of the aims of the study.
In the second stage, to check the participants' homogeneity, the PET test was administered at the outset of the study to make sure there were no significant differences among them in terms of the level of English language proficiency. The time allocated to this test was 60 minutes. The results revealed that the participants were more or less at the same level of proficiency.
Then the goal-orientation questionnaire was administered and the participants were required to complete the above questionnaires by choosing from among 5 alternatives ranging from (1) strongly disagree to (5) strongly agree. The time allocated to the questionnaires was 30 minutes.
Then, the 16-session treatment began during which the participants were divided into three groups and each group received a different type of treatment (self, peer, and teacher-assessment).
At the end of the treatment period, the same questionnaire was administered again to measure the participants’ gain after the implementation of the self, peer, and teacher-assessment techniques. The obtained data were summarized and submitted to statistical analysis.
3.4. Data Analysis
To analyze the collected data and to answer the research question about the effects of self, peer, and teacher-assessment on learners’ goal-orientation, the Analysis of Covariance (ANCOVA) procedure was utilized.
4. Results and Discussion
4.1. Results
The research question aimed to find out the differences among the effects of self, peer, and teacher-assessment on Iranian EFL learners' goal-orientation. To this end, the ANCOVA procedure was used. Table 1 shows the results of the descriptive statistics.
group | Mean | Std. eviation | N |
self | 80.03333 | 8.640256 | 30 |
peer | 73.76471 | 7.885771 | 34 |
teacher | 74.36667 | 9.862794 | 30 |
Total | 75.95745 | 9.143909 | 94 |
It can be seen in Table 1 that the first group, which received self-assessment, has the highest mean (x̅ = 80.03), followed by the third group, which received teacher-assessment (x̅ =74.36). The second group, which received peer-assessment, has the lowest mean (x̅ = 73.76). To see whether or not the differences among the groups are significant the ANCOVA procedure was used, the results of which are presented in Table 2.
Source | Type II Sum of Squares | df | Mean Square | F | Sig | Partial Eta Squared | Observed Powerb |
Corrected Model | 1786.54a | 3 | 595.51 | 8.94 | .00 | .23 | .99 |
Intercept | 4150.85 | 1 | 4150.85 | 62.37 | .00 | .40 | 1.00 |
goalorientationpre | 1048.76 | 1 | 1048.76 | 15.76 | .00 | .14 | .97 |
group | 615.69 | 2 | 307.84 | 4.62 | .01 | .09 | .76 |
Error | 5989.28 | 90 | 66.54 | ||||
Total | 550112.00 | 94 | |||||
Corrected Total | 7775.83 | 93 |
a. R Squared = .230 (Adjusted R Squared = .204)
b. Computed using alpha = .05
(I) group | (J) group | Mean Difference (I-J) | Std. Error | Sig.b | 95% Confidence Interval for Differenceb | |
Lower Bound | Upper Bound | |||||
self | peer | 4.34* | 2.09 | .04 | .19 | 8.49 |
Self | teacher | 6.06* | 2.10 | .00 | 1.89 | 10.24 |
peer | teacher | 1.72 | 2.12 | .42 | -2.50 | 5.94 |
Based on estimated marginal means
*. The mean difference is significant at the .05 level.
b. Adjustment for multiple comparisons: Least Significant Difference (equivalent to no adjustments).
As Table 2 shows, there are significant differences among the effects of self, peer, and teacher-assessment on Iranian EFL learners’ goal-orientation. However, it can also be seen from Table 2 that there were significant differences among the groups in the pre-test. Therefore, care must be exercised in interpreting the results (F(1,93) = 4.62, p < 0.05). Meanwhile, the index of the strength of association (ף2=0.093) indicates that about 9% of the observed differences among the groups is attributable to the independent variable (self, peer, and teacher-assessment). This means that the remaining 91% of the variance is left unaccounted for.
To locate the differences, pair wise comparisons were done. The results of the pair wise comparisons are presented in Table3.
Based on Table 3, there is a statistically significant difference between the first and the second groups, and it can be concluded that the members of the first group have outperformed their counterparts in group 2. Moreover, there is a statistically significant difference between the first and the third groups, and that the first group members have outperformed those in group 3. However, there is no statistically significant difference between the peer and teacher-assessed groups in terms of their goal-orientation. This means that self-assessment is more effective than peer and teacher-assessment on EFL learners’ goal-orientation.
4.2. Discussion
The present study attempted to investigate the effects of self, peer, and teacher-assessment on learners' goal-orientation. The finding of this study was that there were significant differences among the effects of self, peer, and teacher-assessment on goal-orientation. This finding of the present study is in line with those of Huang (2011), who concluded that assessment has a positive relationship with self-regulation. This finding can be related to those of Huang because a significant positive relationship has already been reported between self-regulation and goal-orientation (Radosevich, et al., 2004; Sadeghy & Mansuri, 2014). At the same time, the finding of this study seem to contradict those of Tsui and Ng (2000), who suggest that peer assessment encourages learners to work together, and by so doing improves not only learners’ learning, but also their cooperative working skills. One point that needs to be noted, however, is that in Tsui and Ng’s (2000) study, a comparison was made only between peer and teacher assessment; there was no self-assessment involved. This might imply that the advantage they have given for peer assessment may only apply to the comparison between peer and teacher assessment, not self assessment. In that case, there may be a degree of compatibility between their finding and that of the present study. Actually, compared with teacher assessment, peer assessment led to better result in this study, too, although the difference was not statistically significant. Yet again, there are contradictory reports by those like Heywood (2000), who claims that peer assessment is less effective that teacher assessment.
The findings of this study are not altogether unexpected. There may even be intuitive support for the findings. For instance, the reason why teacher assessment turned out to be the least effective type of assessment in this study could be explained partially by the fact that this type of assessment is normally characterized with a high level of learner anxiety. One could cogently argue that learners feel more comfortable if they receive feedback from their classmates, and even more comfortable if they are supposed to assess themselves. Naturally, therefore, learners may have more positive attitudes towards the more comfortable types of assessment, namely, self and peer assessment. The more positive attitudes and lower levels of debilitative anxiety may, in turn, explain the differential achievement of the participants in the three groups.
A number of other factors could also possibly account for these findings. These factors include, but are by no means limited to, variables such as the Iranian socio-cultural educational context, which may have a significant effect on their motivation (Williams & Burden, 1997), the learners’ knowledge of the differences among the types of assessment, the learners’ proficiency level, age, and their personality traits.
5. Conclusion
The finding of the present study showed that types of assessment affect goal-orientation. Self-assessment turned out to be more effective than both peer and teacher-assessment. Based on the findings, it is concluded that it is better to opt for and encourage self-assessment to improve learners’ goal-orientation. However, considering the factors that might have influenced the results of the present study, and considering the findings of this study in light of the previous studies, one may conclude that a sort of mixed method of assessment may be most effective on the development of Iranian EFL learners’ personal traits and academic achievement. By this, it is meant that each type of assessment seems to have its merits and demerits. On the one hand, the kind of feedback provided to learners as a result of teacher assessment may be naturally supposed to be the most useful and reliable, given the superior knowledge of the teacher. On the other hand, assessment carried out by peers may help learners to share the knowledge and experience in a less anxiety inducing atmosphere, which can be conducive to more efficient learning. At the same time, peer feedback and assessment may encourage a spirit of cooperation among learners, thus paving the way for more cooperative types of learning. Self assessment may also have the advantage of freeing learners from the discomfort of having to be assessed by others (be it teacher or other learners) and saving their face. It may also help foster the much praised traits like learner autonomy. However, it will also deprive learners of the valuable comments others could provide and the experience they could share. There could also be the undesirable feeling of alienation from the class.
Based on the above arguments, it can be concluded that different types of assessment may be beneficial for different purposes. This means that teachers and syllabus designers need to be flexible in their approach toward assessment and to try to adapt their assessment mode to learner characteristics and learning purpose. More in line with the findings of this study, if the purpose is to develop goal-orientation, teachers need to take care to resist the temptation to assess every aspect of learning themselves and to provide learners with more opportunities to self-assess their own learning. This may also require giving learners greater levels of autonomy in other aspects of learning. Syllabus designers may also need to design activities that lend themselves more readily to self assessment or do not encourage teacher assessment.
References