Improving the quality of school teaching through the professional development of teachers is a global concern echoed with growing urgency in a vast array of political and educational circles.
Professor Jennifer Gore, Director, Teachers and Teaching Research Program, The University of Newcastle, will outline her research on quality teaching and quality teaching rounds, emphasising the importance of a strong pedagogical framework and adherence to effective professional development principles in systematically improving the quality of teaching.
Jennifer Gore is a Professor in the School of Education at the University of Newcastle, Australia, where she recently completed a six year term as Dean of Education and Head of School. Professor Gore is Director of the Teachers and Teaching Research Program and Co-Editor of Teaching and Teacher Education.
Good evening, everyone.
I love the way that you queue me in by just this little lowering of volume, and that's
the clue.
But welcome to this.
This is our eighth Dean's Lecture for 2015, which is the busiest year we've had so far
in Dean's Lectures that have been going for about 10 or 11 years, I think, but I always
start by acknowledging the traditional custodians of the land on which we hold this event, and
I pay respect to elders both past and present of the Kulin Nation, and extend that respect
to any Indigenous Australians present.
Tonight's lecturer, Professor Jenny Gore from the University of Newcastle.
It's not often we fly people from interstate to come down and give a lecture, but Jenny
has got some very good research to share with us.
She's Professor in the School of Education, where she recently completed a six-year term
as Dean of Education.
I might ask her later, what's that like, that is, completing the term?
She began a career in education as a secondary physical education teacher in South Australia,
before going on to complete a Master's Degree at the University of British Columbia in Canada
in 1983, and then a PhD at the University of Wisconsin-Madison, one of the close universities
to us in terms of partnership.
She did that in 1990.
She is genuinely regarded as one of Australia's leading teaching and teacher education academics.
Her educational and research interests have consistently centred on quality and on equity.
She recently was commissioned to write a position paper on that subject by AITSL, as part of
their push to improve teacher quality as well.
She is Director of the Teachers and Teacher Research Program and co-editor of the prestigious
international journal, Teaching and Teacher Education.
She's won nearly $5 million of research grants and research funding, including nine grants
awarded by the Australian Research Council.
That's a lofty figure indeed.
She's widely published and cited five books, I think, 25 chapters, 35 referee journal publications.
Her current major research projects focus on teacher development through quality teaching
rounds and understanding the complex factors that shape students' educational and career
aspirations.
At the conclusion of the lecture, I've asked my colleague, Associate Professor Larissa
McLean Davies, Deputy Director of Learning and Teaching, to thank Jenny at the conclusion.
I'm delighted, Larissa, that you've agreed to do that.
Jenny, I'm particularly delighted that you agreed to come all the way down from Newcastle
to contribute to the Dean's Lecture Series.
Please join me in welcoming Jenny to the front to speak on the topic of improving the quality
of school teaching through professional development, with impact as the main emphasis.
So, Jenny, thank you.
Thanks very much, Bill, for the warm welcome and thank you all for coming.
I'd also like to pay my respects to the traditional custodians of the land and pay respects to
Elders past, present and all Aboriginal people here today.
I've got, as is my habit, I think, lots of slides and not a lot of time to speak with
you but I'm really going to try and cover the decade or so of research I've been doing
on these topics to do with teacher development and teaching quality.
The aspirations side of my work, which is another massive project I'll be reporting
on quite a bit at AARE this year if anyone's particularly interested.
So I'll get straight into the topic.
What I'm really talking about is this complex field of teacher professional learning and
it's meant to work something like this.
We design professional development, we set up some processes to ensure that it's going
to be successful and this applies really at the pre-service or in-service level and it's
meant to have an impact on teacher learning, which really should impact on teaching practice,
which is supposed to impact on student outcomes and it would be great if it worked so simply
but it doesn't, of course, and those of you in schools would be very aware of some of
the challenges with professional development experiences that really have very little impact
on any three of those things.
That's why I've put the question mark there.
There's a lot of research that shows that the vast majority of professional development
that teachers experience actually has very little impact on their practice and my question
really is how can we have more impact?
What can we do to really support teachers?
We know from so many sources that the quality of teaching really is the most significant
in-school factor over which we have some control.
I've cited that out of the Australian Teacher and Performance and Development Framework.
There's many other sources where people have made this kind of a claim.
There are so many external factors, of course, that impact on outcomes for students but it's
the quality of teaching really that we can address.
We know from projects like the Gates Foundation's Measures of Effective Teaching project that
teachers actually receive little feedback on their work when they do.
It's pretty perfunctory often and that most teachers receive satisfactory ratings.
It's a bit vague often and in this context of enormous evaluation, particularly in the
United States, evaluation and rating of teachers, most of them in fact receive the highest rating.
These are different school districts in the US.
The biggest bubble is excellent or outstanding or whatever.
You see the bubbles as they go down in size illustrating how many teachers are rated at
each of those levels.
Part of this signals that we actually don't have a very strong system for thinking about
how we assess or evaluate teaching.
Now, there's questions about whether we ought to, of course, and how we ought to and for
what purposes but I think it's fair enough to say that we don't have a very good system.
Elizabeth City and Richard Elmore and their colleagues, after a lot of work, have suggested
that one of the big problems is we can have 10 people watching a segment of teaching practice
and we have everything from high praise to tearing it to shreds, excoriation, because
we all have such different views about what constitutes good teaching and as they say,
gaining an explicit and widely held view of what constitutes good teaching and learning
in your setting is a first step towards scaling up quality.
Now, quality is a problematic term for some people and I think one of the challenges,
major challenges we've faced in education, whether it's education policy, educational
research, educational practice, is that we really haven't come to terms with this term
quality and we've been grappling over it.
In part, there have been paradigm and ideological kinds of positions that have got in the way
of us thinking more seriously about quality in a way that might support teachers and Oakley,
whose name always reminds me of my favourite dress-up outfit when I was a child, in her
important work, talks about how these divisions, which are old hat now but they're still very
strong traces in educational research communities, the divisions between quantitative and qualitative
methods are unhelpful, she says, in the pursuit of knowledge.
The rejection of quantitative and experimental methods in particular prevents us from understanding
both the parameters of social inequality and the effects of professional interventions,
like the interventions teachers make, in people's lives and obstructs the development of a critical
and emancipatory social science.
There's all kinds of academic debates we can have around these issues, but it's very interesting
for me as one who was writing from a Foucauldian perspective on critical and feminist pedagogy
discourses as regimes of truth, one of my most widely cited publications, and something
I'm still very passionate about and that still informs a lot of the work I do, but I've now
moved, in my most recent work, to doing a randomised control trial, and people might
say, Jenny, you've sold out to the dark side or something, but in fact, I've come to see
just how important numbers are in our lives, in the name of equity, and I mean, just to
use a few examples, when we look at Australia's record in taking refugees from around the
world, numbers matter, when we look at equitable pay for women, numbers matter, when we look
at domestic violence, we're very quick to talk about the statistics, it's important
to acknowledge them, the life expectancy for indigenous children is another example, one
that people may not even know about, there are only 25 public hospital beds allocated
for patients with eating disorders, and yet so many Australians, many of them women, suffering
from an eating disorder, the level of homelessness in this country, the level of mental illness,
these are all really important statistics, and one, just to lighten the mood that we,
you know, people care about numbers, is my point, this is an old statistic from earlier
this year, and the British really like this one.
So, the focus on teaching rather than teachers is part of what I talk about a lot, I think
that in the name of pedagogical reform, so much of it at the moment is focusing on teachers,
selecting the right teachers, rating teachers, evaluating teachers, assessing teachers,
in the end, to me, it's the teaching that matters, not the teacher, and to teach well,
teachers actually have to know what constitutes quality, and so we're back to this question of
the grappling with what we mean by quality. If we don't know what quality is, how do we get there?
And then, of course, how do we measure it, which is a challenge for educational researchers,
and how do we improve it? How do we improve teaching if we're unsure about what good
teaching really is? One of the things that also challenges us in teaching, as Philip Jackson said
decades ago, teaching is this incredibly complex, multi-dimensional, unpredictable enterprise with
so much going on simultaneously, so many decisions to be made at every minute in the classroom,
and so how do we come to terms with that enormous complexity in ways that actually help teachers?
My challenge, I guess the challenge I've set myself in my recent work,
is not to try and measure teachers, not to try and rate teacher education programs,
although those things may be important, but to ask a different kind of question.
The question is how can we actually support teachers to refine their practice in ways that
are not punitive or seeking simple compliance, but ways that actually help them, respect them,
and produce identifiable outcomes? There's a lot in there. I come from a very long line of teachers,
and so respecting teachers is also at the core of everything I do, and I think as a society,
we've been quick to blame teachers for so many of society's challenges, and yet
what we fail to do, I think, is to support teachers enough. We've had reform after reform,
Bill Loudon's well-known, well-cited paper, 101 Damn Nations, to do with the review of teaching.
Report after report, review after review, and yet we're still finding the level of improvement that
we'd like in Australian schooling to be quite elusive, and I think it's partly because we may
be needing to come at those questions in a different way. Now, the work I've been doing for
the last, well, since 2003, gosh, I was going to say it was less than a decade, but it's more.
Where did the years go? The quality teaching model that was developed for New South Wales
public education system, with its antecedents in authentic pedagogy, Fred Newman's work from
the United States, and the productive pedagogy work that was done in Queensland, in which I was involved.
This quality teaching model is what I'm going to be talking about today, and the research that
we've done using the quality teaching model to try and achieve some of those kinds of outcomes
I've talked about, of supporting teachers in really useful ways, but also thinking about,
I guess, the obligation to be accountable for the investment that is made in education,
in schooling, and in teacher education, and trying to hold those two, if you like, quite
discourses that are in tension, hold them together in this enterprise. The quality teaching model,
for those who are new to it, has these three dimensions and 18 elements that are about
good practice in teaching. They're derived from a review of the educational research literature
on aspects of practice that make a difference for students, so things like learning styles are not
in there, and are derived from hours and hours of observations in classrooms.
They're not about teaching skills. It's not about pacing and positioning in the classroom.
It's not about the quality of questioning. All of those things matter. It's not actually about
the quality of feedback, which absolutely matters. It's about approaches to teaching.
It's about the mindset the teacher has, and it's about the goals of teaching. If the goals
are about achieving high levels of intellectual quality, an environment in the classroom that
absolutely supports student learning, and high levels of significance, so students can see the
value of what they're learning beyond just learning it because you're in school and I said so,
we're setting students up for more powerful learning experiences.
I don't have time to explain the details underneath all of these words,
but I would just say that because it's trying to, in a sense, provide a coherent overview
of quality teaching in a manageable number of elements, it actually makes it accessible to
teachers because otherwise we're bombarding teachers with so much information that they
really find it hard sometimes to frame it all, to make sense of it all, in exactly the same way we
expect senior students to understand the key concepts of their particular subjects and disciplines
in order to produce high quality work. It's about really providing that level of framing, not as a
kind of a recipe or a formula or a gospel of quality teaching, but as a basis for professional
discussion, debate and conversation. The other well-known systems that are out there in the
public sphere, in particular, are the class system. I believe Bob Pianta actually gave one of the
Dean's Lectures recently, maybe in the last year or two, and also Charlotte Danielson's Framework
for Teaching. Now both of these are well known in the literature because they've done a fair
bit of measurement work, and so that issue of measuring quality is also an issue for us.
There are hundreds of other frameworks for effective teaching that we could be drawing upon,
and my argument is that the choice of framework matters. They're not all alike. We can't just
pick any one and think they're going to do the same kind of work. The quality teaching model,
if we were to plot it on a number of continua, it's generic. It's designed to apply to teaching
from kindergarten to year 12 and across all subject areas. Actually, it can go before
kindergarten and after year 12 as well. It can certainly apply to university education.
Other frameworks are very specific. They deal with Aboriginal pedagogy, or they deal with
pedagogy in a particular subject area, like the Plato language arts system that Pam Grossman and
colleagues have developed. Some are very narrowly focused on a small part of practice, for instance,
on questioning or on thinking or on engaging students. Others are much more comprehensive,
and the quality teaching framework is a comprehensive framework. One of my favourite
ways of illustrating its comprehensiveness and the impact of that is a deputy principal who I
interviewed as part of one of our research studies who'd been teaching for 20 years,
and he'd been using this work for a couple. He said,
this is the first time in my career I feel I'm actually teaching students. Until now,
I've just been giving them work to do. And it's about how this idea of a comprehensive
understanding of teaching can re-conceptualise how people think about their work, the purposes
they're trying to achieve. Some frameworks are designed to assess teachers. There's stacks of
them in the US. Others are designed to support teacher professional development. The quality
teaching model is much more about professional development. Some are oriented at capturing all
of teachers' work. The Danielson framework is more about teachers' work. Others are focused
more on instructional practice or classroom practice in teaching. Some focus on teaching.
Some say they're more about learning. I find that a really tiresome debate. If you're thinking about
teaching, surely it has to be to produce learning, and if you're focusing on learning, surely you
have to think about the teaching that's going to produce it. The quality teaching model is
absolutely about both. I know some people might want to say it's called quality teaching. Why
isn't it called quality learning? Or we're actually focusing on what teachers need to do
in order to produce quality learning. Some frameworks are really open. The clinical
supervision model that was developed in the 1960s, we still use in teacher education all the time.
Tell me what you'd like me to focus on today. And I might say I'd like you to focus on my
questioning technique, and someone might give me feedback on that. Questioning technique might
be fabulous, but the way I'm lowering my expectations of these students, or the way
that I'm handling knowledge, oversimplifying things, or whatever, may be the problem that day.
And so having that openness can sometimes mean it's not very helpful to teachers in the end.
At the other end, we have frameworks that are incredibly closed. There's one they use in the
US that says you must give praise five times in a lesson, or you cannot be rated a high-quality
teacher. I mean, just silliness. I'm glad you laughed before I even finished the sentence.
And some are negotiated. And the quality teaching model is closed. In a way, there's 18 elements,
but the relevance of particular elements on a particular day in a particular lesson
is open to debate. The example I often use as a former PE teacher is when I teach javelin,
the element called student direction, where students have some control over their learning,
rightly is a one. You will not move until I blow my whistle for you to go and collect the spear.
Self-reflection is one function of some of these frameworks. Others are for external evaluation,
and our work very much now is about collaborative analysis of teaching practice, and I'll come back
to that shortly. So just to summarise some of that, I would say that of all the frameworks
that exist, some focus on teaching, some are much more about teachers, and more broadly about
teachers. Rightly so. I'm not suggesting that that's inappropriate. Some are about assessment,
some are about development of teachers. Each quadrant does different kinds of work.
I'd say the Australian Professional Standards for Teachers sit somewhere down here. It's about
teachers' work. It can be used for assessment, can be used to support teacher development.
Quality teaching, on the other hand, is about teaching. It's about classroom practice,
so it sits up there somewhere. The framework for teaching, Daniel Wilson's work, has been
very much about assessing teachers. The class framework is more about assessing teaching
practice. The quality teaching rounds work that I'm going to be talking about shifts it very
firmly into this quadrant of how do we go about supporting teachers in improving their practice.
I'd argue that when we're operating at the top end here and really focusing on classroom practice,
we're very much also impacting back on how teachers think about their work more broadly.
I'm not sure that the same happens at the other end. If we just focus, for instance,
on the standards, important as they are, they don't have the kind of specificity that feeds back
into what does it mean to be an effective manager of classrooms, for instance. We need to all figure
that out for ourselves. The problem with frameworks that focus on assessment is they motivate teachers
through fear. When we talk about merit pay for teachers and some of the kinds of processes that
are going on in the U.S. at the moment with value-added measures, check how your students
went a year later, and if they didn't achieve decent gain scores, then you're obviously not
suitable to teach. I mean, painting a kind of a caricature there. That's not really a very
helpful way to build a profession that attracts the kinds of people that we want into teaching.
One of the challenges, I think, with some of the work at the moment that's going on
is that it's actually likely to have more negative impacts on who even wants to come
into teaching because they see the over-regulation of teaching and the kinds of punitive systems
that are in place much more in the U.S. than here. I actually don't think Australia would allow some
of the systems that are being used in the U.S. to be transported. I think we have a different
culture of educational practice and I'm glad for it. At the other end of the scale, I think we can
motivate teachers through greater hope, giving them more confidence in themselves and their
colleagues. Because we have this challenge about defining quality, most teachers actually don't
know how good they are or bad, but many teachers are really good and they don't know. It's one of
the things we're really finding with this work we're doing is that teachers are affirmed by it.
They get to see, they get feedback on their practice. Many of them never get feedback on
their practice except the occasional thank you, miss, note from a parent or the principal's glad
that there haven't been any issues in the classroom, that kind of issue. I'm going to talk
about some of the empirical studies we've been doing over this period of time that I've been
working in this space. The first one, as you can see, was conducted with New South Wales
Department of Education, a linkage project, and really looking at what are the implications for
the system of this relationship between pedagogy and achievement. It was a descriptive study.
The next one was an intervention study, but in a kind of a let's trial something way.
The third one, and that was conducted with the Parramatta Catholic Education Office,
who approached us to work with them. The ACT was really interested in the work we were doing.
They wanted to know if they could do it on a shoestring budget instead of on an ARC linkage
grant kind of project. I'll talk briefly about that. Then at the end, the last study that we're
doing at the moment is the randomised control trial, which is where I will finish this talk.
When we're talking about this framework, we use a coding system, and people go, numbers.
Just to give you an example of how the numbers function in this system,
for each of the 18 elements, there's a coding question. In this case, for deep knowledge,
to what extent is the knowledge addressed in a lesson focused on a small number of key concepts
and the relationships between them? Then there's a coding scale where people make judgments
about whether what they observed in a lesson is closer to a one, two, three, four, or five.
There's a quasi-quantitative feel to all of these, but it's a basis for making judgment,
because without it, what do we say to teachers in giving them feedback? We say that the lesson
seemed to go well. The students seem engaged. We get this kind of vague feedback that really,
in a lot of cases, doesn't help teachers. They don't know how to do better as a result of it.
When I start talking about numbers on a one-to-five scale, five is high in general,
one is low in general, but as I said, it's not a simple numeric thing either, because a one
is the right score for my Javelin lesson on the element called student direction.
I just wanted to make sure you had that sense of how the numbers work.
The first study we called SIPA for short, just the acronyms. It was a big study in terms of the
amount of data that was gathered. I'm not going to go into a lot of detail with a lot of these
slides, but you can see there with questionnaires, interviews, classroom observations, assessment
tasks, every piece of student work associated with that task, not just a random sample,
teachers involved in some of the coding, lots of inter-rater reliability checks, and so on.
I do want to just draw attention to the classroom observations. We had two observers in most of
those 665 lessons, and we always stayed for a whole lesson. I know there's a lot of emphasis
at the moment in walk-throughs as a productive way of gathering some information about what's
going on in schools and classrooms. Our view is that it's much more respectful of teachers
and teaching practice to watch a whole lesson, and teachers, much as we don't like to be driven by
the bell, lessons are divided into periods of 40 or 55 or 80 minutes length, and to watch the whole
lesson where teachers think about what they're doing at the beginning, middle, and end, and how
they bring it together, even if it's carrying on to the next day or whatever. It's important to
watch a whole lesson, we believe. I once watched the triple period of year 11 trigonometry and
was taken way back to my high school math days. I usually put about 12 slides in here, and I've got
one to illustrate some of the major findings, trying to be restrained and just touch on some
of the key points. Overall, we found the quality of teaching to be below the midpoint of those
scales, now the 1 to 5 scale, so below a 3 on average. The scales were designed to maximise
variability. It means we don't expect a lot of 1s, it's not happening at all for any of the students.
We don't expect to see a lot of 5s, it's happening all the time for everyone. We expect a relatively
normal distribution. We found the scores skewed towards the lower end, and the question is
qualitatively, what difference would it make if we could get the scores skewed to the higher end?
We found that the more Indigenous students in a classroom, the more students from low SES
backgrounds, and the more students with low prior attainment, the poorer the quality of teaching.
When we talk about OECD comparisons of countries, and we wonder why Australia is not doing so well
on equity, this is one of the reasons why. The students who are struggling most at school
receive, on average, poorer quality teaching. Schools don't just reproduce societal inequalities,
schools contribute to the production of inequality. This is one of the ways in which that happens.
We found that when teaching was better, there was a narrowing of achievement gaps for Indigenous
and low SES students, which is exactly the kind of finding that Newman, with his work on authentic
pedagogy, and others – sorry, I keep touching buttons accidentally – that Newman and others
have found, and so to see that narrowing of achievement gap, in some ways, the real winners
of better quality teaching are the students who traditionally haven't done so well at school,
because they have the most to gain. And we also found this really interesting and somewhat
confronting finding that there were no significant differences in the quality of teaching between
beginning and more experienced teachers. And people say, how can that be on these measures?
Now, I think there's two likely explanations. One is that teacher ed programmes are doing a
ground running, and you laugh. I mean, I am passionate about the reform of teacher education.
I think we can be doing better. So I don't know that that's the most viable explanation.
The other explanation is most teachers have not received professional development that's impacted
on their practice. They've received professional development, a lot of it in curriculum,
welfare, IT, workplace health and safety, all kinds of other things. But in terms of impacts
on the way they fundamentally approach the job of teaching and interacting with young people,
maybe that's why we find no difference in the quality of teaching between beginning
and experienced teachers. So that was a very descriptive study. It wasn't designed
to make a difference, per se. It was looking at what was happening in New South Wales public schools.
The Paramount and Catholic Education Office was really interested in the work and said,
we've got a small jurisdiction. We don't have 2,200 schools. We've got 77.
Can we work with you to see if we can really make a difference here? And they said, let's try and
work with you to come up with a way of really effectively implementing the quality teaching
model in our schools. We drew on the literature on effective professional development, and we're
particularly interested in notions of professional learning communities and instructional rounds as
two of the kind of contemporary approaches that really emphasise collaboration among teachers.
But we were very mindful of the fact that with both of those approaches,
there's no analytical lens provided for teachers to do that work. In instructional rounds,
what do you see? What makes you think that? Are the kinds of questions asked. In professional
learning communities, let's get together and identify an area of practice that we should
focus on. And teachers are coming with their very different experience, very different backgrounds,
very different beliefs, and taking a long time to even start speaking the same language.
Now, one of the things when I ask teachers, what does a good classroom look like? They almost
invariably say students are engaged, which is great, but what does engagement mean? And when
you start to drill down into that, people have very different ideas even about what engagement
means, which goes to the fact that in teaching, we don't have a very highly developed language for
talking about our practice, or a very full set of concepts for talking about practice.
So we came up with this idea of what we called quality teaching rounds. So it's sort of taking
something from the idea of rounds, taking something from professional learning communities,
and adding to it the quality teaching model to provide teachers with some kind of framing
for doing that important analytical work that they might do together.
Quality teaching rounds have three sessions. Basically, you have a group of teachers who
commit to each other, and in this project there were eight teachers. Often the principal was
actually included as a member of the professional learning community. We actually didn't want that
at first because we thought it would do harm to the democratic relations among the teachers,
but in fact, it turned out to be very powerful because the principal went first,
and whether he or she had to borrow a class or was used to teaching, but if they put themselves
out there first, it seemed to make a real difference to the power relations, and you'll
hear strands of my earlier work coming through here. So the first session involves that group
of teachers having read something or watched something, a TED Talk, whatever. They come
together and they discuss it, and we've found this to be a really important time to give teachers
space to just talk to each other and think together. It's interesting in schools how busy
teachers are that it's a real luxury for them to actually talk to each other about something
they've read in common, and it's also a space for them to talk about, well, what does this mean in
relation to the quality teaching model? We're not, as I said, trying to create disciples to some kind
of evangelical way of thinking about teaching. We're trying to get teachers to interrogate,
to question, to criticise, to raise issues, to really think about their own practice.
The second session involves a classroom observation. Every member of the professional
learning community goes and watches one of the group teach a whole lesson. In this case, as I
said, that meant seven teachers going into a classroom watching one of their colleagues teach,
which they said was pretty daunting at first, and not surprisingly because we're not used to that
in teaching. After the lesson, everyone codes the lesson on the 18 elements, and we allow time for
that. It typically takes people maybe 20 minutes the first time they do it, and subsequently maybe
10 minutes, but we ask them to prepare notes looking at the evidence they gathered during
the lesson and thinking about how they're going to talk back to their colleague who taught the
lesson. It's about supporting and nurturing their colleagues, giving honest, constructive,
critical feedback, but in a way that is respectful, supportive, and realises that
this is emotional work teachers are doing as well as work on practice. After they've coded,
they come together and they discuss the lesson, but they do it in a systematic way. Each person
in the group takes their turn to say why they coded each element in a particular way, and we've
found that that's really critical to breaking down hierarchies. The teacher who speaks over
everybody takes their turn. The teacher who hardly ever speaks, the first year out teacher,
takes their turn to contribute to the conversation, and it has a really powerful effect
on breaking down those kinds of barriers to productive professional work that often exist
in schools. We had four schools participating in the quality teaching rounds process and 12
comparative schools in the diocese, and I won't go into too much detail. We had 105 rounds conducted
in the first three years of the study. We did some interviews the year after. We gathered NAPLAN data
from the year before they began, as well as through the first three years of the study while
they were doing quality teaching rounds. And just to try and summarise what we found in that study,
certainly there was a positive correlation between participation in rounds and NAPLAN scores.
In some of the primary schools we worked with, there may have been eight of the 13 teachers
in the school participating, and so even though we can't in any way suggest a causal link here,
we can talk about this correlation between the quality of teaching that was happening and
the results for students. We found some significant differences between the teachers participating
and those who weren't in terms of professional satisfaction and the coherence of their
professional learning, a whole lot of issues that we explored through survey and interview.
And importantly, we found significantly higher quality teaching than in the first study.
Now, as I said, the ACT thought, this is good, we'd like to have a piece of this, but we can't
release eight teachers for eight days a year for three years. They said, if that's the Rolls-Royce
version, can we please have the mini-minor version? Apologies to anyone who drives a fabulous
mini-minor. And so they said, can't we just do half-day rounds? Can't we have fewer people?
Can't we just get rid of the readings and so on? And so in the name of design experiments,
we said that there were some parameters that could not be messed with, but there were other
things that they were free to work out how it could work best in their school. We've given their
budgets, their conditions, and so on. We had 18 schools from special schools right through to
senior colleges, year 11 and 12 campuses that they have in the ACT, and as you can see, teachers from
various career stages. And basically what we found, we said there have to be at least three people
in a professional learning community. Otherwise, it's coaching or mentoring or some other kind of
relationship. So we wanted at least three. They could have as many as they liked, and we said at
least three hours has to be set aside for teachers to do this important work. And it's about, again,
respecting teachers' need to have time to talk with each other and to talk about practice.
Basically, however it was done, teachers found it to be really valuable, and we found a higher
quality of teaching produced, even when all they did was three rounds. So a round is one day,
observing one teacher. Three teachers each took their turn to teach a lesson, and we found higher
quality. But just to summarize, with the three dimensions of the quality teaching model, the
first column in each group is from the SIPA study, the descriptive study. The second one is from the
study we did with the Parramatta study. We call it IPA for short. And the third one they like to call
the ACT study, where we did these much lighter touch, if you like, intervention, but still found
that it had an effect on practice and a significant effect on practice. Still lots of room for
improvement, we would say. Now in terms of some of the methodological issues with gathering this data,
who was collecting, who was doing the observations, and so on, we came to the view that we really need
to attempt a randomised controlled trial. Now for me to be uttering those words, as I said,
seems to have, it means I've come to a very different place that I didn't expect I would
ever be in. And I was, I had the pleasure actually of hearing Professor Paul Connolly at BIRA this
year talk about randomised controlled trials. His keynote lecture is available, I believe,
on their website. And it was a really fabulous argument about when randomised controlled trials
are useful and when they're useless. And he talked very passionately, and I think very elegantly,
about the value of this kind of research for answering certain kinds of questions.
So the New South Wales Department was really interested in this work and happy to fund a
really large project. I think there's $730,000 that's gone into this work. And in the Great
Teaching Inspired Learning Blueprint, which is the New South Wales version of something you have
very similar here in Victoria, quality teaching rounds is already written into the department's
commitment to how they want to support teachers and help build collaborative teaching cultures.
We observed two teachers, two observations for 192 teachers in 24 schools before they began,
six months later when they'd completed the quality teaching rounds, and then 12 months later,
as in they may or may not have done any more work on quality teaching rounds, but we were
really interested if there was any, if there was an effect, first of all, and then if there
was any sustained effect. We also supplemented that, though, with a whole lot of survey interview
and case study data because we were interested in the impact on teachers' identities, on the
culture in schools, on teachers' career plans. You know, were teachers who were unsure about
teaching as a career, early career teachers, thinking that they might stay in teaching after
all? Or did it scare them off completely? Were teachers thinking of retiring and suddenly
re-energised? These were the kinds of questions we wanted to explore. We adhered very strictly
to the consort protocols for conducting these kinds of trials. Our observation team was blinded
to group allocation. At first I thought that meant blindfolds, but what it actually means,
of course, is that they didn't know whether the teachers they were observing had been involved
in the rounds or not, and we have a paper on the protocol that's about to be published.
We needed 24 schools. We had 243 that wanted to participate in this, knowing that they may be
allocated to a control group, which means they may not get to do this until a year later. So it
was what we call a wait list of control group, meaning that they get to do it, but they just had
to wait for it. We divided our schools that expressed interest into primary and secondary,
urban and rural, low, mid and high SES for the urban schools, and then after the baseline
observations, they were allocated to the set choice or control groups.
The control group, as I said, waited until just doing their rounds at the moment. The choice group
could choose to do one set of rounds. They could do two if they wanted to, and they could decide
how many people in a professional learning community. They could have eight. They could
have three if they wanted, but it was eight teachers basically in each school. The set group,
we said groups of four teachers and two sets of rounds. I guess what I want to do is just share
with you some of the results of this work. First of all, these are the overall scores in terms of
the quality of teaching from the first big study that we did with the New South Wales Department
12 years ago, and this randomised controlled trial. What was fascinating is the same spread
of scores, but what's important from a measurement perspective is that there is that spread of
scores. If everyone scores in the big bubble that I showed at the beginning, or if the scores are in
a narrow band, then we don't have a very powerful tool for thinking about differences in quality.
What you see there is that there's a good spread from the one to five scale.
We've got our three groups at baseline. The set intervention group actually was producing
slightly better teaching, but it was not statistically significant. Basically,
the three groups in measurement terms started the same. Here's what the picture looked like
after six months. We saw gains in the quality of teaching for both of our intervention groups,
and basically no difference for the control group. Here's our 12-month data, which we're literally
just finishing analysing, and all of this will be double-checked shortly, but my statisticians
tell me that this is certainly an accurate representation of the findings. A really
important issue for us there is not only did teachers sustain the same quality 12 months
after the intervention, but for the choice group, they've actually gone up a little bit.
What we had here was an effect size of 0.4 for the choice group, and interesting about
where the choice is a real feature in that, and a 0.2 effect size for the set group. Sorry,
getting my words muddled up. We also did serious implementation fidelity checks, visiting
the professional learning communities and staying with them for their rounds to check how they were
enacting them. We had this kind of eight-point check. Were people actually present the whole
time? Because this is not a thing where you duck in and out of the classroom when your phone rings,
or when you need to consult somebody. It's about really committing to the group. The trust and
respect that is conveyed through the process is a really important part of it. Did everyone code,
and so on. What we found here on the bottom left corner is that there were a couple of
professional learning communities who really didn't meet the implementation fidelity standards.
When we ran what's called a protocol analysis, you see basically the same kind of pattern,
but in fact a bit stronger now when we dropped out those two groups that weren't doing it as
it was intended. The effect size is here around 0.3 and 0.4, which for a short-term intervention,
in many cases, it was teachers having four half days of doing this kind of activity. To get these
kinds of gains is a pretty powerful finding. Interestingly, we also used the Peter Hart
surveys on teacher morale. What we found for the three groups is at the six-month point,
some groups dropping right off. Now that's December, remember, and I don't know about
how you're feeling, but I feel pretty low at the moment myself. The set group actually went up in
morale, even though the rest of the teachers were dropping. There's the 12-month finding,
which is also pretty interesting that the control group dropped further, and both of the groups that
have participated in rounds have come up. Now this is just scratching the surface. There's so much
more technical work that could be done to explore these kinds of issues, but it does give us some
sense that we may be onto something here that not only helps improve the quality of teaching as
defined in the way we're defining it, but actually is really supportive of teachers as well in terms
of morale. It's not just, I guess, the numbers, but it's actually what teachers are saying.
We have hundreds of interviews with teachers, and I have so much wonderful qualitative evidence
I'd like to share with you, but I've obviously just selected a few quotes today, and there's
a whole lot more work to be done here. This is a really interesting one in terms of morale. They
did not like me, and I did not like them. This is Sandra of 21 years experience, and it was only on
hearsay and reputation alone. They didn't know me from a bar of soap, and I did not know them
from a bar of soap, but when I was in the room with them and working with them on quality teaching
rounds, I respected them, and I learned to trust them, and I learned who they really were. It's a
pretty powerful way of talking about how it can change relationships in a staff room. This was
a catalyst for changing the culture in our school. Staff are happy now to embark on new projects and
have their peers observe their teaching, which is a massive change. Rounds makes you question
your practice and continually review and refine your pedagogy. It makes us think about what and
how we teach. I've changed the tense there. It gives us experience of seeing teaching strategies in other
areas. Teachers are actually telling us they like to do this activity with people from different
subject areas rather than the same subject area. They learn more from the PE teacher going
and watching the maths teacher than the maths teachers watching each other. This one from an
experienced teacher, I found renewed enthusiasm for the classroom. It's made me aware of the need
for a whole school approach to teaching. I love the fact that other subject specialists were able
to share their methods, and I learned so much. I really enjoyed the collegial support and sharing,
and it's reinvigorated me because after all these years, you just do what you do, and it was a bit
scary to start with because I kind of thought they know everything. They've done it at uni,
and I've had to learn it. Quality teaching, I've had to take it on. I'm not immersed in it, but now
I'm on an equal footing with them. That's an experienced teacher talking about young teachers,
which is fascinating. Then, of course, we have many beginning teachers. This one's second year
out. I'm more willing now to interact with the others. I don't feel as intimidated. As a new
teacher, you sit back and go, I don't know what they're talking about, but quality teaching has
given me a lot more confidence, so I can speak more. I'm more confident as a teacher. I'm not
afraid to speak in front of other people, even if I'm wrong. We're finding some really powerful
impacts on how teachers relate to each other, how they see themselves,
and certainly how they see their teaching. Here, Peter, a secondary teacher of seven years,
experienced that discussion's the most powerful element, but obviously you can't have a discussion
without the observation, but it's the discussion on the observation that's really powerful. You
have other people seeing what you're doing. It's like a mirror, having them discuss what you're
doing, what you didn't do so well, what you did well, and all the other stuff. It's just eye
opening. I think that's what we're hearing time and time again, that teachers don't really know
how other people see their practice. I have many more, but I think I'll just move to the fact that
we've also done an analysis of 39 beginning teachers in these studies who've been doing
quality teaching rounds. For this analysis, we've used Carol Barchi's notion of the discursive
subjectification and lived effects of any kind of new policy as it's enacted, and we're using
quality teaching rounds in that way. In fact, I thought I'd typed it on this slide just before
I came here, but this work has actually been published in the International Journal of Ed
Research in a paper called Interrupting Attrition, if anyone wants to have a look at that.
I don't have time to go into the detail of it at all. Some of the preliminary findings, I guess,
are to say that we do have this significant variation in quality teaching scores I showed you.
We have significant positive effects for our intervention groups. There was no moderating
effect of school sector or location. We had a few moderating effects that we wanted to test.
We have found a moderating effect of the sex of the teacher. We found that female teachers
actually get greater gains after this activity than male teachers, but who knows? We can offer
all kinds of explanations for that as well. Again, here we are 12 years later. We found no
significant difference in terms of years of teaching experience as to how much teachers
gained from the participation in this process, and that's really important. We can work on who
comes into teacher education. We can work on our pre-service activities, but we have this massive
teacher workforce, 250,000 teachers. We need to be working to support them as well because
otherwise change is going to be, as Tony Braik often says, glacially slow in education.
Sorry, that's the slide I meant. When Vicki Phillips was out here from the Gates Foundation
recently, or 2013, it feels like it was recent, the conclusion of the big multi-million dollar
Gates work, one of their main conclusions was that we have to explore these tactics for improving
teacher effectiveness. We've got to work with new teachers. We've got to support the professional
growth of all teachers. We've got to retain and really leverage our high-impact teachers.
We need to find a way of having a more equitable distribution of good teachers with the students
who are struggling most in schools, and their fifth strategy was to get rid of those who are
less effective. I guess what we'd argue is the work we're doing on quality teaching rounds is
doing the first four of these. It really is working with new teachers, all teachers, the
really good teachers, and it's absolutely about trying to ensure a good quality teaching and
learning experience for all students. Now that's a hypothetical work, what do they call it? Change
the game and work the curve. Our actual data shows a bit of a working of the curve. The blue dotted
line is our SEPA data from our initial study, and what we do have in the green line is the
effect of the randomized control trial work that we've been doing. A working of the curve, it's not
the hypothetical that we'd love to see on the previous slide, but this is real. As one of my
stats guys said to me the other day, Jenny, it's an 11% increase. If you don't think that matters,
can I have an 11% pay rise? I guess that made the point. Not that I have money. Who has control over
money? In short, I guess what we would say is that quality teaching rounds really is, it appears to
be at least, a viable approach for doing some of the really important things that I think we're
struggling to do in education, to enhance teacher learning, refine teaching practice, build teaching
culture in schools, improve student outcomes, and through some of our early work on that plan, and
there's more to do, having an impact on narrowing the equity gaps that we care about. To go
back to the slide I commenced with, with quality teaching rounds in that middle box, I
think we can go from the question mark to at least a small tick, if not a bold tick. I might just
leave you with a bit of a word cloud that one of our RAs put together. When we asked teachers what
they got out of quality teaching rounds, of course the bigger the word, the more times it was
mentioned, and one of the things I found very interesting is way near the bottom on the right,
we see the word coding. It's a tiny word, so while some people I think have been concerned that
this kind of process is all about numbers and coding and judging, what teachers say it's about,
what they talk about when they're talking about quality teaching rounds is teaching and learning
and quality and experience, and there's words like reflecting and in professional and colleagues and
enjoyment and so on. So I might just leave that one up there for you to ponder, and thank you
very much for being with me on this journey.
Thank you Jenny for a fabulous Dean's Lecture with some wonderful challenges I think for us
here at the Graduate School and in teacher education. So in presenting the vote of thanks
and very sincere thanks to Cumming and for being prepared to share decades of your work with us
and your most recent research in quality teaching and the quality teaching rounds,
I guess I wanted to draw out a couple of key themes for us. One is it seems to me, and it's
wonderful to read Jenny's work and to see that respect for teachers and teaching is absolutely
at the core of this work. We're in a policy environment where teacher quality is very much
in the focus, and as Jenny pointed out, sometimes that can make teachers feel very anxious about
their own performance and make them feel that the focus is on assessment, but really what we've
heard tonight is about supporting teachers and about being very focused on how we can improve
teaching and learning for teachers and for their students. So there's just a couple of things I
guess I wanted to draw out of that. From that position of respect and from thinking about
teaching and learning from a position of professional learning, we get a couple of
themes. One is the importance of collaboration, and it seems to me that was a really key theme
that comes out in Jenny's research, but also a good and important reminder for us. Collaboration
not only in research from principals to teachers, but collaboration amongst researchers with
different theoretical paradigms. So I was very encouraged to hear that we have a researcher
looking at Foucaultian regimes of truth on the one hand and randomised controlled trials on the
other, and being able to actually hold those things, those tensions in balance and being able to say
there's something very valuable for us to think about, that it's not about an evangelical allegiance
to one way of thinking or being, but recognising that we need robust frameworks and different
tasks for different roles. So that seemed to be one really key thing, and that in doing that,
and in doing that with a position of respect, we have the opportunity to develop what we see as
an incredibly robust framework for looking at quality teaching. And I guess the other challenge
for us is to be specific and claim the language. So claim the language of quality teaching.
Actually identify what we mean by that, because otherwise those words and those terms and
understandings will be set by others for us. So as teacher educators, we need to do that, and as
teachers, we very much need to do that. And so it's encouraging and motivating for us to think
about the research that's been done that sees teachers working very much in concert with
researchers. And it's certainly at a time in our graduate school, we're looking at the reaccreditation
of our Master of Teaching in ways that we can really focus on improving quality of pre-service
teacher education. These themes of collaboration and breaking down barriers seem to me incredibly
important for us this evening. So thank you very much, Jenny, for a wonderful Dean's Lecture.