Variations in student perceptions of assessment

‘Experiences of assessment: using phenomenography for evaluation’ (Jones and Asensio, 2001)

This paper explores the use of assessment as a tool for structuring students' experiences in a networked learning environment. The phenomenographical approach aims to explore people's qualitatively different ways of experiencing the world. In this case, this was studied through the use of interviews with tutors and students on the Open University 'Information Technology and Society (THD204)' course which used conferencing for communication and in two assignments.

Key points

• "The problems of interpretation and the understanding of others' intentions are fundamental to collaboration" (schwartz, 1999) - in an online, distance learning environment, documentation and the interpretations given to documentation become critical. 
• It is "extrememely difficult to design an online course [or] activities in ways where you are not surprised and/or disappointed by the output".
• Student participation is a common concern - and a common response to to redesign a course with more and tighter control over the learner's choices and pace of interaction/contribution; this was recognised that "the tighter the schedule the more structured the exercise is and there is a danger that you're damping down ont he potential for creativity".
• "it is possible to use assignments as a vehicle for encouraging students to adopt new patterns of learning, whilst at the same time covering course content" (Macdonald et al, 1999) - but for this to be achieve, students must have a clear understanding of the course designers' intentions.
• Students' experiences vary in what may be unpredictable ways for the course designers' intentions. 
• Students can interpret the aims of assessment differently - even if they are in a co-operative group tasked with a joint project! In this case, this was despite 12 pages of detailed instruction.
• Course designers cannot control the background and context in which students interpret instructions or assessment criteria. Even course documentation aimed at students who have many external factors influencing their interpretation may be misinterpreted.
• Distance/networked learning may suffer more from differences in student interpretation because the interpretation of context by students is more vulnerable to variations in setting than in face-to-face settings.
• The approach adopted by the teacher is a key variant in helping to determine a student's approach to learning (Prosser and Trigwell).
• The variation in student interpretation might imply that "teaching interventions were necessary to negotiate understanding 'on the fly' and that a cautious attitude needed to be adopted to reliance on the use of course documents in a networked environment."

Reflections
There were a lot of clarification questions asked in our tutor group and course forums about each of the assignments on H807. This would suggest that (some) students indeed did not have a clear understanding of the course designers' intentions. As in the course described, course designers almost certainly believed that they had produced comprehensive and complete guidance about assessment. I wonder if the level of student anxiety and confusion increases with the weigting of the assignments - if that is the case, I would expect to see the greatest level of uncertainty yet in the coming weeks as we approach the End of Course Assessment.

In the example given, students were using conferencing for collaboration for their assessed tasks. They had only used it for collaborative work once before, so it was unfamiliar to them. This raised questions in my mind about whether this assessment therefore effectively reflected the ways of working which had been previously taught and experienced on the course, or the learning objectives as known to the students.

I wondered whether the difficulties in designing and preparing sufficiently detailed standard documentation might also be mirrored in the design of scaffolding, particularly of group activities. There will be variation in student skills, knowledge and experiences, and therefore scaffolding at any particular level may or may not suite the individual student. How can the individual 'turn off' scaffolding which is built into group activities? Do they just have to sit through (cruising, with little challenge or development) those activities for which they need less scaffolding, but which they are still supposed to engage in? There needs to be a balance between scaffolding and challenge (see Dearnley), but there also needs to be a balance in the way scaffolding is designed bearing in mind the range of experience of a mixed group of learners.

Reference:
Jones, C. and Asensio, M. (2001) ‘Experiences of assessment: using phenomenography for evaluation’, Journal of Computer Assisted Learning, vol.17, no.3, pp.314–21.