For teachers maximizing impact on learning

The lesson experience from the teacher’s perspective
The good news is that that inspiration and passion are two-way: the aim of deliberate
change agents (the teacher) is to make as many students as possible inspired and passionate
in learning the subject – and this requires a teacher to inspire that passion.Yes, for some,
it is about getting students to move from novice to be at least capable, rather than proficient
and full of passion, but for students to become ‘capable’ still requires a teacher to believe
passionately that these students can become capable. Steele (2009) has shown that this
passion does not necessarily mean bubbly exuberance, but a sense of ‘with-it-ness’,
involvement in each student’s learning, and evaluating your effect on each student.
One powerful, but most unused, method is student evaluations of teachers (SETs).
Students are more than passive observers of teachers. As the theme in this book indicates,
the relations and perceptions that they have of a teacher’s impact on their learning is critical
to being involved in the discipline of learning: for all students, the basic principle is that
there needs to be a reason to be at school. So often, students are omitted in the quest for
new innovations and plans for transforming schools – and motivated and engaged students
are central to lasting school improvement (Pekrul and Levin, 2007).
Wilson and Corbett (2007) interviewed many students who had not performed so well
in school.The students were adamant that teachers not be allowed to find excuses to not
teach them, not leave them alone if they did not participate, and not to allow students to
decide on their work or whether to work or not.There were six major characteristics of
the type of teacher whom students wanted to see in their classroom:
The end of the lesson
141
1
2
3
4
5
6
7
8
9
10
1
2
3
4
5
6
7
8
9
20
1
2
3
4
5
6
7
8
9
30
1
2
3
4
5
6
7
8
9
40
1
2
3
4
5
61
VISIBLE LEARNING – CHECKLIST FOR THE END OF THE LESSON
40. Teachers collect evidence of the student experience in their classes about their success
as change agents, about their levels of inspiration, and about sharing their passion with
students.

1. someone who stayed on with students to complete their assignments;
2. someone who was able to control student behaviour without ignoring the lesson;
3. someone who went out of his or her way to provide help;
4. someone who explained things until the ‘light bulb went on’ for the whole class;
5. someone who provided students with a variety of ways through which to learn; and
6. someone who understood students’ situations and factored that into his or her lessons.
Student evaluations are often a hotchpotch of questions relating to course effectiveness or
improvement, or teacher effectiveness or improvement. Irving (2004) looked at things the
other way around. Using the dimensions from the National Board for Professional Teaching
Standards (see Chapter 2), he identified, wrote, and compiled 470 questions that related
to the 470 statements in the mathematics standards. He ran many focus groups (groups of
teachers who rated the items for agreement with the standards) to whittle the items to a
smaller number that still ensured that the essence of each standard was well represented,
and he also administered the items to large samples of high-school students. From the
Delphi and factor analysis, he created a 51-item SET (see Appendix F for the best subset
of 24 items), the underlying dimensions of which were:
1. commitment to students and their learning;
2. mathematical pedagogy;
3. student engagement with the curriculum;
4. family and community; and
5. relating mathematics to the real world.
To evaluate the validity of this questionnaire and factor model, he administered the SET
to over 1,000 students from the 32 National Board Certified teachers (NBCs) and 26
experienced non-NBC colleagues. He achieved a more than 70 per cent success rate in
correctly classifying the NBC status of the teachers by using the student responses!
The seven items that best distinguished the accomplished teacher from the experienced
teachers were that the teacher:
■
challenged students to think through and solve problems, either by themselves or as a
group;
■
encouraged students to place a high value on maths;
■
helped students to construct an understanding of the language and processes of maths;
■
got students to think about the nature and quality of the work;
■
developed students’ abilities to think and reason mathematically, and to have a mathe-
matical point of view;
■
encouraged students to try different techniques to solve problems; and
■
showed students interesting and useful ways of solving problems.
These items could well form a set of prompts for teachers to evaluate their level of
inspiration and passion. Of course, seeing it through the students’ eyes is a more compelling
The lessons
142
1
2
3
4
5
6
7
8
9
10
1
2
3
4
5
6
7
8
9
20
1
2
3
4
5
6
7
8
9
30
1
2
3
4
5
6
7
8
9
40
1
2
3
4
5
61

The end of the lesson
143
1
2
3
4
5
6
7
8
9
10
1
2
3
4
5
6
7
8
9
20
1
2
3
4
5
6
7
8
9
30
1
2
3
4
5
6
7
8
9
40
1
2
3
4
5
61
way in which to triangulate these evaluations (see Exercise 1 at the end of this chapter).
As has been noted, the hallmarks of the passionate and inspired teacher are a commitment
to challenge, engagement, understanding, quality, reasoning, and developing learning
strategies.
The lesson experience from the curricular perspective
The critical part when evaluating the lesson(s) is a review of the learning intentions and
success criteria.You need to start by asking: ‘Did the students know these?’; ‘Could they
articulate them in a manner that demonstrated that they understood?’; and ‘Did they see
the learning intentions and success criteria as appropriately challenging?’ As importantly,
what changes were made to the learning intentions and success criteria in light of the
class experience? Not all learning can be pre-scripted, and there needs to be an opportunity
for teachers and students to suggest other learning intentions and success criteria – provided
that they are related to the mission of the lessons. As Hastie (2011) showed (see Chapter
4), it may be worth asking students to keep a work diary that details what they think they
are learning, indicators of their progress, how confident they are that they will achieve
these learning intentions in the time available, and their perception of their degree of
success. The students could also be asked whether they consider the learning intentions
involved to be attainable and worthwhile challenges – that is, does achieving the success
criteria for the learning intentions lead them to progress beyond what they already knew?
This is only knowable for students at the end of the lesson.
Another method is to ask colleagues to critique your learning intentions and success
criteria – preferably before you implement them, although it is also worth reviewing them
at the end of the lesson(s). This can be done alongside examples of students’ work to
evaluate the level of attainment of the success criteria and to assist in addressing the
question:‘Where to next?’ Or, you could provide the examples of planning from the lesson
including the learning intentions, and ask colleagues to comment on what they see as the
success criteria (and maybe also the quality of the learning intention in light of the
examples of student work): do they match your success criteria?
Do your colleagues need to know the learning intention or success criteria to do this
task? Sometimes ‘yes’; sometimes ‘no’. Michael Scriven (1991) has long talked about ‘goal-
free evaluation’. By not knowing the teacher’s intentions and success criteria, a colleague
VISIBLE LEARNING – CHECKLIST FOR THE END OF THE LESSON
41. Together, teachers critique the learning intentions and success criteria, and have
evidence that:
a.
students can articulate the learning intentions and success criteria in a way that
shows that they understand them;
b.
students attain the success criteria;
c.
students see the success criteria as appropriately challenging; and
d.
teachers use this information when planning their next set of lessons/learning.

can evaluate the students’ reactions and claims about these (through interviews), can ask
what was actually learnt rather than what was intended to be learnt (through examples
of student work), and not have the tunnel vision that can come from looking for evidence
for the intended goals and thus overlook many positive or negative unintended side effects.
Scriven noted that merit is determined by relating program effects to the relevant needs
of the affected population – in this case, the students.The teacher may then see what the
students experienced, and reflect on how close this was to his or her own intentions and
notions of success.
The lesson experience from a formative and summative
perspective
One major mistake is to consider that the notions of ‘formative’ and ‘summative’ have
something to do with tests; in fact, there is no such thing called summative or normative
tests.‘Formative’ and ‘summative’ refer  to  the time at which a test is administered and, more
importantly, to the nature of the interpretations made from the tests. If the interpretations
from the test are used to modify the instruction while it is ongoing, it is formative; if the
interpretations from the test are used to sum up the learning at the end of the teaching,
it is summative – as illustrated by Bob Stake’s maxim: ‘When the cook tastes the soup, it
is formative; when the guests taste the soup, it is summative.’
In the same way that the goal of the cook is to make the best soup possible for the
guests, it is imperative that teachers have excellence summative evaluation in place in their
classes, because that can be among the most powerful evidence that there is likely to be
excellent formative evaluation in place. If a school has poor summative assessment in place,
then it is unlikely that teachers will have the ability, purpose, or wherewithal to be
concerned with formative interpretations. Serving poor soup to the guests is probably the
best indicator that the cook was lousy at tasting it during the preparations. Too much
reliance on tasting the soup, as well, may lead to inattention to the goals – with the result,
for example, that the soup is cold when the guests arrive.
Many systems are emerging that help teachers in their assessments, although most tend
to be summative. Even the so-called ‘predictive’ tests tend to be more about what the
student is supposed to know at the end of the lessons, and thus can be less effective in
providing information that can lead to change during the instruction.Tests that are most
powerful for formative interpretations tend to be those created to measure what is to be
taught in a series of lessons (not a whole term or year), drawn from a large item pool that
references the learning intentions from the curricula, and that aim for each student to get
50 per cent correct and 50 per cent incorrect – because, in that way, the students and
The lessons
144
1
2
3
4
5
6
7
8
9
10
1
2
3
4
5
6
7
8
9
20
1
2
3
4
5
6
7
8
9
30
1
2
3
4
5
6
7
8
9
40
1
2
3
4
5
61
VISIBLE LEARNING – CHECKLIST FOR THE END OF THE LESSON
42. Teachers create opportunities for both formative and summative interpretations of
student learning, and use these interpretations to inform future decisions about their
teaching.

teacher can know what has been accomplished and what still needs to be accomplished.
This may mean adaptive assessment (the computer choosing the optimal set of items to
administer to each student), but the emphasis needs to be on the quality of interpretations
made from the assessments for this to have an effect on what the teacher and student do
next.
Our own system, as one example, was developed less as a repository of ‘tests’ and more
as a reporting engine – which made us concentrate on providing worthwhile and
dependable interpretations to teachers about who they taught well, what they taught well,
their strengths and weaknesses, their effects and progress, and what they could do next to
enhance levels of performance and progress (Hattie, 2009).
While these kinds of reporting engine are not inexpensive, schools need to make a
decision about the best reporting engine to use or whether to devise their own school
report about how successful teachers are teaching all students, both in terms of students’
progress and their levels of performance – with the proviso that the system needs to be
available during, and not only at the end of, instruction.
The notion suggested here is for a report for teachers (and students) to monitor a
teacher’s effect, progress, and success with each student – for example, by using data teams
to share interpretations across the school to ensure maximum effect. Unlike many more
public reports, the essence of the suggested reports relates to informing teachers’ overall
judgements in a collaborative manner: if we cannot inform and enhance these judgements,
we are missing the components that have a major effect on students – the teacher’s
expectations and notions of challenge and progress.
Conclusions
The lesson does not end when the bell goes! It ends when teachers interpret the evidence
of their impact on students during the lesson(s) relative to their intended learning intentions
and initial criteria of success – that is, when teachers review learning through the eyes of
their students.What was the impact, with whom, about what, and how efficiently? Often,
answering these questions requires help from others observing and thus providing extra
‘eyes’ into student learning, video analyses to provide extra ‘eyes’, and various forms of
informal and formal assessment to provide extra ‘eyes’. Did the lesson ‘invite’ students to
participate, engage, and progress? Were there sufficient starting points, given the various
phases of prior achievement and learning of the students? Were there any unintended
consequences of your teaching? How many students gained the criteria of success – and
for those that did not, what is now needed to assist them to meet the criteria? Underlying
these questions is whether the students became active partners in evaluating their progress.
As evaluators of the teaching impacts on their learning, students are at least as effective as
teachers – and often well ahead of most administrators and parents.
A key question when reviewing the effect of the teacher and lessons is not only
effectiveness, but also efficiency. Could there have been more efficient methods for having
an effect on the learning and achievement of all students? ‘Efficiency’, in this context, does
not necessarily mean ‘speed’, but rather, more cognitive efficiency. Such efficiency comes
from many sources – especially the use of diverse learning strategies. Such versatility in
the use of learning strategies can lead to less time taken, greater effort invested, reduced
error rates, and opportunities for the further development of a multiplicity of strategies.
The end of the lesson
145
1
2
3
4
5
6
7
8
9
10
1
2
3
4
5
6
7
8
9
20
1
2
3
4
5
6
7
8
9
30
1
2
3
4
5
6
7
8
9
40
1
2
3
4
5
61

Exercises
1. Use the Invitational Teaching Survey (http://www.invitationaleducation.net/ie/
ITS.pdf) to see how inviting your students see you.
2. Check each department, or across school years, for the degree of co-planning and
critique. Do teachers know what other teachers are teaching, recognize the difficulty
of what is taught, and appreciate the concept of ‘challenge’? Do they share in deter-
mining the quality and nature of success criteria and learning intentions, and regularly
review together their effect on students?
3. Invite groups of teachers to share their marking of assignments – which aims to help
teachers to see how their concepts of challenge and standards are being realized. In some
cases, share the learning intentions and success criteria, and ask for comment on how
examples of student work is demonstrating these; in other cases, do not share the
intentions and criteria, but ask colleagues to comment on what they consider the
learning intentions and success criteria to be based on the evidence in the students’
work.
4. Develop a bank of lesson plans to be shared across teachers, identifying the learning
intentions and success criteria, including evaluation comments from many sources as
to the impact of the lesson(s) on students for consideration in future teaching, and
making suggestions for modification in light of these evaluations.
5. Revisit the class and ask students what they now see as the learning intentions and
success criteria from these lessons.What did and did they not understand in the lessons?
What did they do when they did not understand? Did they seek help (or not)? What
were the reactions from their peers to their learning? Did they believe that they were
listened to by this teacher? What was the nature of discussions among students during
the lessons? What questions did they ask and what would they now still wish to ask?
Were there multiple opportunities to learn, re-learn, and re-learn again? What do they
now understand by success in these lessons? Finally, how close do the students think
they are to the success criteria?
6. Revisit the staffroom and ask teachers if they know what other teachers are teaching
at the moment, what other teachers’ concepts of challenge are in relation to this
teaching, and whether there are high levels of relational trust (for example, respect for
each person’s role in learning, respect for expertise, personal regard for others, and high
levels of integrity) when making policy and teaching decisions. Ask about the degree
to which this school carries out collaborative evaluation of the effect of its teachers.
7. Revisit leadership in the school and ask about the common understanding of the
school’s self-review of programs, the quality of this review program, and, most
importantly, the degree to which the interpretations from this review process are having
on how all in the school are increasing their impact on students.
8. Administer the Irving Student Evaluation of Teaching to your class (see Appendix F).
Share your results with fellow teachers and work out strategies to increase the students’
perceptions of your commitment to the students’ learning, your teaching effectiveness,
the level of engaging students with the curriculum, and how you relate the learning
to the real world.
The lessons
146
1
2
3
4
5
6
7
8
9
10
1
2
3
4
5
6
7
8
9
20
1
2
3
4
5
6
7
8
9
30
1
2
3
4
5
6
7
8
9
40
1
2
3
4
5
61

1
2
3
4
5
6
7
8
9
10
1
2
3
4
5
6
7
8
9
20
1
2
3
4
5
6
7
8
9
30
1
2
3
4
5
6
7
8
9
40
1
2
3
4
5
61
3
Mind frames
PART

1
2
3
4
5
6
7
8
9
10
1
2
3
4
5
6
7
8
9
20
1
2
3
4
5
6
7
8
9
30
1
2
3
4
5
6
7
8
9
40
1
2
3
4
5
61

A major theme of this book and of Visible Learning is that the quality of teaching makes
all the difference.Yes, it would be nice to have eager, well-groomed, invested students with
financially gifted parents, but our neighbourhood schools must take all who walk through
the gates.We could ask that students need to be ‘ready’ and motivated, and come to school
well fed, having been supported at home to do their homework, and are attentive and
calm.This would be wonderful, but a major role of schooling is to help students to acquire
these habits; we should not discriminate against students whose parents may not know
how to help them to do so.We could remonstrate about the quality of teacher selection,
preparation, promotion and so on – but the chances of making differences in these matters
has thwarted so many for so long with little evidence of change.These issues are important,
but history has shown that resolving them has not made much difference to student
learning to the degree that is required. For example, there is not a lot of evidence that
improving teacher education colleges has improved the overall quality of teaching (but,
of course, this is not to say that we should stop trying to find better ways in which to
educate teachers to have these impacts).We have used tests to measure the surface know-
ledge and used these data to name, shame, and blame – and teachers have learnt to play
this game – but playing the testing game even more smartly will not make the difference.
We have spent billions on buildings, restructured curricula to align with tests and vice
versa, and engaged in wonderful debates on the peripheries of what really makes the
difference.We love to talk about the things that do not really matter. Perhaps the greatest
resistance to change of the current system is that we have asked millions of teachers to
improve this system – and they have applied their creative thoughts, and thus improved
and sustained the current model far beyond its use-by date.
We know that the major source of controllable variance in our system relates to the
teacher, and that even the best teacher has variability in the effect that he or she has on
his or her students. The message in this book is that teachers, schools, and systems need
to be consistently aware, and have dependable evidence of the effects that all are having
on their students – and from this evidence make the decisions about how they teach and
what they teach.The message is that the evidence is about student learning – particularly
progress – provided that the learning intentions and success criteria are worthwhile,
149
1
2
3
4
5
6
7
8
9
10
1
2
3
4
5
6
7
8
9
20
1
2
3
4
5
6
7
8
9
30
1
2
3
4
5
6
7
8
9
40
1
2
3
4
5
61