A reflection on the efficacy of pre-defined outcomes for lesson preparation, delivery and monitoring student progress on university pre-sessional English coursesPublished in May 2019 DOI: https://doi.org/10.24384/19g9-am74
AbstractUniversity Pre-sessional English (PSE) courses are designed to help students improve their English language and academic skills prior to joining their degree course. These courses are often intense and pressured working environments, where teachers have a limited time to complete all the necessary course administration, preparation and assessment. In order to assist teachers with these areas of the Pre-sessional courses offered by CU Services at Coventry University, we felt that specifying pre-defined outcomes for each lesson would help teachers to (1) narrow the focus of their lessons; (2) stage their lessons effectively in order to allow time for practice and production activities; (3) monitor student engagement and attainment in class time; (4) record student attainment more effectively and meaningfully. The idea of specifying pre-defined outcomes emerges from constructive alignment, a principle of course design in which “all components – intended learning outcomes, teaching/learning activities, assessment tasks and their grading – support each other, so the learner is enveloped within a supportive learning system” (Biggs and Tang 2011: 109). The outcomes-based approach is used to support course design in Higher Education but is not often explicitly adopted within PSE courses. Rather than specifying outcomes for the whole course only, we devised outcomes for each lesson using Bloom’s revised taxonomy (Anderson and Krathwohl 2001) to help phrase them. After teachers had the opportunity to use the explicit, pre-defined outcomes in their lessons, we held focus groups and found that the pre-defined outcomes helped teachers to narrow the focus their lesson, plan their lessons more efficiently and stage their lessons more effectively. However, the efficacy of outcomes to monitor and record student engagement and attainment was less clear, with some teachers highlighting the fact that the complex nature of in-class monitoring involves more than simply noticing student attainment. Though we recognise the criticisms of the outcome-based approach (see Richards 1990:4 and Thornbury 2011), for intensive courses, such as PSE courses, this approach was a welcome and generally effective adaptation.
Pre-sessional English (PSE) courses are designed to help students improve their English for academic purposes. These courses are intensive and high pressured, with many institutions recruiting a high number of teachers at peak periods during the year. The short-term nature of these teaching roles means that many teachers could work at a different institution for each PSE session. Therefore, teachers often have the added challenge of getting to know new people, systems and processes in addition to their other duties. As might be expected, there are institutional differences, with some courses being highly structured (i.e. all materials provided and aligned with pre-defined assessments) and others which are structured more loosely (i.e. teachers need to design and deliver lesson content themselves), but most institutions require teachers to complete numerous tasks including course administration, lesson preparation and delivery, assessment marking and feedback.
The PSE courses offered by CU Services Ltd (CUSL) at Coventry University are highly structured. We produce our own in-house text books which we encourage teachers to adapt to meet their particular group’s needs. Teachers are required to continuously assess their students’ progress through in-class activities, coursework tasks and other forms of assessment delivered during the course. Our teachers need a means of monitoring and recording day-to-day student progress within lessons as this information feeds into student reports. When we reviewed such reports, rather than providing a holistic overview of student attainment, some teachers focussed on student attitude and motivation, or had a natural bias towards a particular skill area. It emerged that some teachers felt overwhelmed by the number of aspects they could monitor within a lesson, which may have arisen because the lesson materials provided could be interpreted in different ways and it was not always easy to narrow the focus of the lesson.
To establish an effective means of monitoring and recording student progress, we needed to help teachers focus appropriately on the content of the lessons so that they could identify the key features and determine individual student ability in relation to these features. To do this we needed an approach to translating the overarching course aims and outcomes into specific aims and outcomes for individual lessons, and drew on the principles of constructive alignment to help. Constructive alignment is a principle of course design in which “all components – intended learning outcomes, teaching/learning activities, assessment tasks and their grading – support each other, so the learner is enveloped within a supportive learning system” (Biggs and Tang 2011: 109). We believed that explicitly stating the intended learning outcomes for each lesson would help teachers to (1) narrow the focus of their lessons; (2) stage the lessons more effectively in order to allow time for practice and production activities; (3) monitor student engagement and attainment in class time; (4) record student attainment more effectively and meaningfully. The first step in this approach was to clarify exactly what we mean by an aim and outcome.
Within language teaching, a number of different terms are used to describe the focus of courses and lessons, and these terms are often used interchangeably. The most common terms are the following: goals, aims, objectives and outcomes. Graves (1996:17) describes goals as ‘general statements of the overall, long term purposes of the course’. Richards (1990: 3) describes goals as ‘general statements of the intended outcomes of a language programme, and represent what the curriculum planners believe to be desirable and attainable programme aims […]’. In this explanation of ‘goals’ Richards has also referred to aims and outcomes, demonstrating the confusion around the terminology.
The terms ‘aims’ and ‘goals’ are often used interchangeably. However, goals are often used to talk about whole courses whilst aims can be used to talk about individual lessons as well as courses. Outcomes focus solely on what the student can do. They are always written with a measurable verb which describes the competencies students should be able to demonstrate as a result of learning (Biggs and Tang 2011). Importantly, for our purposes, outcomes are distinct from objectives, aims or goals, in the broad sense, because they focus on what the student does, rather than what the teacher does. Outcomes are often used in Higher Education to describe what students should be able to demonstrate on completion of a module or course of study. However, we have used them not only to describe what students should be able to achieve at the end of the PSE course, but what students should be able to demonstrate within a lesson.
For our purposes, it is not the terminology that matters but the intention behind the terms. Our intention is to shift the teacher’s perception to focus on what the learner is learning rather than what the teacher is teaching. We have chosen the terms aim and outcome to describe this shift. An aim is what the teacher intends to teach and an outcome is what the student will be able to demonstrate they can do.
The value of outcomes
The following example shows how aims and outcomes might apply to a specific lesson on a PSE course. A typical PSE course outcome might be for students to demonstrate an ability to participate effectively in a seminar discussion and this might be translated into a lesson aim, to give seminar discussion practice. This is, however, a broad aim that needs to be more narrowly focused for a lesson. ‘Seminar discussion practice’ can be unpacked by analysing some of the key constituents in a seminar discussion. One such constituent could be building arguments, so the aim of the lesson now becomes to help students build arguments within seminar discussions.
The next step is to define the outcomes of the lesson by breaking down what it means to build arguments in a seminar, and to do so in such a way that will allow the teacher to observe the student doing this. The following outcomes are an example of this:
- Recognise the structure of an argument in a seminar discussion.
- Analyse the language used to structure arguments.
- Summarise another student’s argument in a discussion.
- Develop someone else’s argument by adding their own ideas.
The measurable nature of the outcome is achieved by using a behavioural, or measurable, verb. This defines the behaviour the students should demonstrate as well as the level of cognitive challenge the task presents. There are a number of taxonomies that categorise thinking and understanding, such as SOLO (Biggs and Collis 1982) and Bloom’s Taxonomy (Bloom 1956). We referred to Bloom’s revised taxonomy (Anderson and Krathwohl 2001) which organises learning on a cline of lower order thinking skills to higher order thinking skills (remembering → understanding → applying → analysing → evaluating → creating), and each level has associated verbs. Choosing teaching and learning activities which require students to engage with higher order thinking skills is said to promote a deeper approach to learning (Biggs and Tang 2001: 24 – 27). However, in terms of language teaching, it must be said that cognitively challenging tasks are not always linguistically challenging and vice versa.
In the class that aims to build argument skills in seminars, the teacher may observe that all of the students are able to recognise the structure of an argument, most are able to analyse the language of an argument, but fewer could summarise other’s arguments etc. This approach discourages teachers from norm-referencing (e.g. X is the best student in the class), instead favouring criterion-referencing assessment within the classroom (e.g. X can summarise arguments) (Biggs and Tang 2011: 98). Information about individual student progress could then be recorded in each student’s continuous assessment profile to help build a fuller and more meaningful picture of their performance. The teacher also now knows which features of this particular skill area need to be revisited in subsequent lessons. Of course, outcomes are intended and other unintended outcomes may present themselves within the context of the lesson and these should not be ignored.
Implementing the outcomes-based approach
Graves (2001: 183) points out that aims and outcomes can be difficult to articulate, especially for teachers who are new to a course (as many are within PSE teaching), stating that: “one cannot map a route until one has travelled it”. Recognising, therefore, that teachers come from different teaching backgrounds and have different levels of experience, we added pre-defined outcomes to each lesson so that teachers could see how they could narrow the lesson focus.
In line with the principles of constructive alignment, the intended learning outcomes we suggested for each lesson aligned with the suggested teaching and learning activities and the final assessments as well as the skills students would need to achieve success on their destination course. We also developed ‘suggested procedures’ which ran alongside lesson materials, showing approaches to teaching the lesson materials and how these materials would help students engage with the target language and/or skill(s). These suggested procedures show how the tasks in the lessons link to the outcomes and where teachers should monitor student attainment of outcomes.
Outcomes-based teaching has come under criticism in the past for trivialising language learning (Richards 1990:4), or by being too rigid or for treating learning as a linear process (Thornbury 2011). However, as PSE courses are intensive and high-pressured environments in which students have a pre-defined destination, we felt specifying pre-defined outcomes would help to provide a supportive learning environment for teachers and students alike.
The efficacy of outcomes for lesson preparation and delivery
To determine the success of our approach, in December 2017 we held a focus group with a number of teachers from a short (5-week) and a longer (10-week) PSE course to find out how they felt about the addition of outcomes. Overwhelmingly, the teachers reacted positively to the introduction of pre-defined outcomes. All teachers across both courses believed the outcomes helped them narrow the focus of their lessons and felt that having the outcomes helped them plan lessons more efficiently, with comments such as: “[the outcomes] saved time for the teacher” and “I could see where the lesson was going just from looking at the outcomes.”
It was felt that the outcomes had a positive effect in terms of the lesson staging, and some teachers felt it allowed them to make more time for practical activities. One teacher commented:
I thought the outcomes helped me focus the staging of the lesson better. I can think more critically about each task or each activity. […] When you’re looking at the setup of the lesson and you know what the end goal is, you’re thinking ‘are the steps helping the students to get towards that goal in a quicker way?’ […] So, when you see what you could potentially cut out or make more efficient, you end up with a lot more time left which then you think well ‘it’s not a bad thing if that’s devoted to production, refection and feedback’.
Providing outcomes in this way also had an unexpected benefit in the classroom: the majority of teachers displayed the outcomes at the beginning of the lesson, which was not common practice previously, therefore making the focus of the lesson transparent to the students. The teachers felt that this allowed the students to self-monitor and assess their own progress, with one teacher stating:
I think they are quite useful for the students, because it’s all about breaking things down into manageable chunks for everyone involved. It’s not just ‘oh today we’re going to do reading for a purpose’, we’re going to do four very specific things, and you can tick them off.
Whilst the reaction to outcomes for lesson planning and staging was mostly positive, there were some reservations. One such reservation was that pre-defining outcomes may take away the teacher’s “thought process” and encourage teachers to be “lazy”. However, this was refuted by another teacher who said that the outcomes “don’t really contradict my instinct so I feel that we are naturally looking to achieve them. I don’t think by having the outcomes stated for us this is taking our decision-making away from us.” Another teacher also liked the fact that the outcomes provided consistency across all courses and led to “an element of standardisation”.
Another reservation was that pre-defined outcomes may prevent teachers from adapting or changing materials, with the teacher stating: “if you want to change things around on the course, the students might think ‘well why didn’t you do that one [task]’, so that might limit [the teacher’s choice] a bit.” However, the same teacher observed that, when things were changed, students did not actually respond in that way.
The efficacy of outcomes for monitoring and recording student progress
There was less enthusiasm for the efficacy of outcomes for monitoring and recording student progress. Most lessons had around four pre-determined outcomes and each class contained approximately 14 students. In terms of monitoring, teachers felt that they were “juggling too many balls” and were not able to measure each student’s attainment of each outcome. This sometimes led teachers to monitor for the attainment the outcome they perceived was the most important, or focus their attention on the ‘strongest’ or ‘weakest’ student. One teacher also pointed out that classroom monitoring serves a number of purposes and is not completed just to measure student attainment, stating:
When I’m monitoring, I’m monitoring for completion of the task rather than for the outcomes. […] You’re monitoring for – ‘Are they doing what they’re supposed to be doing? How well are they doing what they’re supposed to be doing?’ You’re answering questions, you’re encouraging, you’re giving feedback, so there are a lot of things going on at that point and quite often in a short space of time, sometimes 10 minutes.
Whilst teachers felt it was difficult to monitor for each outcome, teachers did recognise that the outcomes helped them widen their focus, rather than focusing on attitude, motivation or a particular skill area. One teacher stated that when recording student progress: “it’s very likely that with one student we would be repeating the same sort of comment and overlooking [certain aspects] and if it wasn’t an outcome we would neglect to write about it”. Another teacher also felt that the measurable nature of outcomes was helpful as it was “adding a measurement to something that was difficult to measure”.
A further concern was that teachers felt students could display attainment of the outcome within a highly scaffolded environment (i.e. the lesson), but were not always able to produce the same level of attainment independently. This reflects the truism that language learning is not a linear process as pointed out in Thornbury’s (2011) criticism of outcome-based learning. One teacher emphasised this by saying:
Because they had lots of support and it was controlled, they understood and they achieved those outcomes within the context of that lesson. However, when it came to actually marking coursework, when they have freer rein particularly within their project, it was quite obvious they hadn’t grasped that outcome.
Teachers felt it would be helpful to have increased targeted practice, which students complete independently outside of the lesson and which requires students to display the language and skills taught within the lesson. This would have the added benefit of consolidation for the student, whilst giving the teachers more dedicated time to assess the level of student attainment without the added distractions of managing the classroom.
Without exception, teachers felt that the measurable nature of the outcomes benefitted the students, who could use the outcomes to monitor their own progress within each lesson. One teacher said that she often displayed the outcomes at the end of the lesson and this was “self-affirming for [the students] and they felt they achieved something because it was them saying whether or not the objectives [outcomes] were met”. Another teacher agreed, stating: “at the end [of the lesson], where you actually show the learning outcomes when the students can just go ‘yes I’ve got that’ ‘I need a bit more work on that’ or ‘I don’t get that at all’”.
We sought to determine whether pre-defining outcomes for each lesson would have benefits for teaching and monitoring student progress within PSE classrooms. Teachers generally responded positively to the introduction of outcomes as they found they helped them narrow the focus their lesson. Outcomes also had the added benefit of helping teachers think more critically about their teaching practice and stage their lessons more effectively. The use of outcomes to monitor and record student engagement and attainment was less straightforward, with many teachers highlighting that the complex nature of in-class monitoring involves more than simply noticing student attainment. Due to the number of students in the room, the number of outcomes, and the fact that in-class monitoring is multifaceted, the teachers sometimes found it difficult to measure each student’s progress through the outcomes during lesson time. However, making the outcomes available to students meant that the students themselves were able to monitor and measure their own attainment. The addition of further targeted tasks carried out by students independently outside the classroom and returned to teachers may help students demonstrate their true attainment of the intended learning outcomes, as well as assisting teachers in monitoring student progress and attainment.
Though our approach of pre-defining outcomes for each lesson was not without its challenges and criticisms, drawing on principles from constructive alignment and implementing outcomes for each lesson was mostly viewed as a positive step. We found that outcomes helped teachers to plan more efficiently and stage lessons more critically, often maximising opportunities for practice and production. They also helped teachers to broaden their focus, allowing them to get more complete picture of each student’s skills and abilities. Biggs and Tang (2011: 109) suggest that constructively aligned courses envelop learners within a “supportive learning system”. We go one step further and suggest it extends this support to the teachers as well. Outcomes provide teachers and students with a road map, which is particularly useful when they have not travelled on the road before.
We invite your comments and reflections on outcomes-based approaches in your context.
Anderson, L. W. and Krathwohl, D. R. (2001) A Taxonomy for Learning, Teaching and Assessing: A revision of Bloom’s Taxonomy of Educational Objectives. New York: Addison Wesley Longman
Biggs, J. B. and Collis, K. F. (1982) Evaluating the Quality of Learning: The SOLO Taxonomy. New York: Academic Press
Biggs, J., and Tang, C. (2011) Teaching for Quality Learning at University. Fourth edition. Berkshire: Open University Press
Bloom, B. S. (1956) Taxonomy of Educational Objectives: Handbook 1, Cognitive Domain. New York: Longman
Graves, K. (1996) Teachers as Course Developers. Cambridge: Cambridge University Press
Graves, K. (2001) ‘A Framework of Course Development Processes’ in Innovation in English Language Teaching. ed by Hall, D. R. and Hewings, A. (p. 178 – 196)
Richards, J. C. (1990) The Language Teaching Matrix. Cambridge: Cambridge University Press
Thornbury, S. (2011) A is for Aims. [online] available from <https://scottthornbury.wordpress.com/2011/04/24/a-is-for-aims/> [10 August 2016]