Classroom data would expose our orthodoxy problem

There is a way to get to the bottom of our education system’s failures, but are we ready to face the uncomfortable truth?

The next decade of budget repair will be brutal on policies that can’t show impact. It will challenge the dominant orthodoxy of evidence-based policy as defined by the academic research.

You don’t have to be dux of school to know Australian students are falling behind those in comp­arable countries on the best international measures of edu­cation performance.

The OECD’s Program for International Student Assess­ment reveals a long-term decline in Australian school students’ reading, mathematics and science skills, while other countries that once were on par with Australia have raced ahead.

These trends are alarming enough, but changes to the way Australian governments operate and spend money are about to make education policy harder.

Across the next decade the largest budget repair program Australia has had will shine a spotlight on how education policies have been devised and how we know whether they are working. It will expose flaws in the dominant orthodoxy of how education policy is developed.

But education policy still can thrive if we follow the lead of those policymakers who have already embarked on a new path that bases education policy firmly on what occurs in the classroom.

For now, Australian govern­ments are spending more than ever. Even before the budget, the size of the federal government’s Covid response came in at more than $200bn, more than four times the size of the fiscal response to the GFC just over a decade ago.

And tThis figure doesn’t count stimulus spending by state governments or the growing list of infrastructure investments. Stimulus spending in Victoria, the state hit hardest by Covid will seetake state debt blow out to $155bn over the next four years.Budget repair is inev­itable. Across the coun­try, Treasury depart­ments are bringing together teams to work out how they will approach budget repair. For the next decade or so, Treasurers will rule the cabinet table. They will lead expenditure review committees that make expenditure recommendations based on the merits of policies and programs across most areas of government. Other ministers and departments have to start planning their responses now.

The years ahead will furnish many headlines on the effects of budget cuts: which ministers lost, which groups in society are hurt. Understanding this process – how governments decide what is funded and what is cut – shows how education policies need to change to survive the next decade. There are some areas of government ex­pendi­ture that won’t be cut: health and aged-care expenditure probably will increase, defence and border security also will be safe, while there is a considerable amount of unavoidable spending to keep roads maintained, schools open, trains running and police funded.

That brings the burden of budget repair down to a few key policy areas in which the real winners and losers will emerge. To understand how this battle plays out, think of government not as a single entity but as a bunch of competing departments and policy areas. Fighting among them can be savage, especially over a shrinking pie.Strong ministers are better able to protect their budget in cabinet discussions; some public servants are better than others at fighting Treasury officials looking to cut their budget.All departments will invest huge resources in trying to retain as much of their budget as they can. In most states, school education has had growing budgets for many years, in part as a result of the Gonski reforms. Policymakers in health, welfare, infrastructure and other areas of spen­ding will be quick to point this out and argue it’s their turn now.

How will each minister and department argue the merits of their policies? In essence, it comes down to impact – how they can demonstrate the impact of their policies on what happens on the ground, how they can prove that their programs shouldn’t be cut.

Government spending is wildly popular today, as last month’s federal budget showed. But in coming years many policy areas will be hit hard as governments wind back massive spending programs that were introduced to counter the impact of Covid-19.

Voters are at first very appreciative when bags of money are handed out, but as they come to see already huge deficits getting larger, and tax increases and interest rate raises entering political discussion, their mood can turn fast. They will demand to know, “what did all this spending achieve, how did it make Australia better?”.

This political pressure will shape governments’ budget repair processes for the next decade, affecting what is cut from the budget and why. History shows that in times of budget repair, education policyhas been particularly vulnerable to treasury knifecuts. I fear it will happen again.

Since the 1990s, the mantra of evidence-based policy has come to dominate debate. A huge amount of resources is invested to ensure that policy development is based on solid evidence. There remains, however, a serious failing in the way we define and develop evidence-based policy and nowhere is this problem more stark than in school education.

Evidence-based policy in education is rarely evidence of impact or evidence of what happens in classrooms but, rather, it is defined through academic research. In school education, across the past 20 years, academic research has highlighted the importance of quality teaching and, to a lesser degree, quality school leadership.

In response, education policy has created a raft of standards, rubrics and frameworks that reflect the research on what is good teaching and good leadership. Teachers and school leaders can measure their practice against what the research says is important and try to move up the teacher standards – from a proficient level to highly accomplished, for example – by improving their practice. Similar frameworks have been developed for school improvement, teacher professional development and training. All reflect the orthodoxy of evidence-based policy making over the past 20 years, basing everything on academic research.

Datasets track performance against the research, while systems track things such as how many teachers have been rated as highly accomplished against the research-based ­standards.

In previous budget repair discussions, these policies would have looked good when ministers were arguing against having their budgets cut. Intuitively there is a lot to like: we want policy to be based on evidence, and who wouldn’t want more highly accomplished teachers in their system? But the uncomfortable truth is that evidence-based policy has a poor record of improving student learning. The growth of evidence-based policy assumed that focusing on academic research would translate into outcomes; with a few exceptions – especially in yearly years reading – but it simply hasn’t. No one likes to talk about it but it is true.

Selective research

The problems with a focus on academic research are complex but it boils down to what research we focus on and how we translate that research into policy.

The research we have used is largely based on general teaching practice and general leadership practice. But the problems teachers and school leaders face are specific learning issues.

Policies based on general research are simply not that helpful in addressing specific learning issues of the curriculum. For example, each year we have large numbers of students who don’t learn the foundational concepts of arithmetic, geometry and algebra on which mathematical learning in the later years of schooling is based. Rubrics and standards of general teaching practice don’t real­ly help solve this critical issue.

Unless we can identify the reasons this is occurring and take specific steps to address them in classrooms, students will not progress. In fact, the policy focus for teachers on standards, frameworks and rubrics regularly pushes them away from understanding and improving the impact they are having on students’ learning of the curriculum.

If we focus less on academic research and more on the impact of what happens in classrooms, then we would focus on the three factors that most affect student learning in classrooms:

● What we teach. The decisions made about the curriculum taught in classrooms: the instructional materials used, the texts students read, the learning tasks they undertake, and so on.

● How we teach. The teaching methods and practices we use to teach the curriculum.

● How we assess. Assessing students’ learning of the curriculum through tests and tasks that students undertake.

Data from schools and classrooms can be gathered to enable policy to target these areas and then monitor and evaluate pro­gress. But in Australia we have little data on any of them.

Our education systems have followed the dominant orthodoxy of evidence-based policy and focused on academic research. The modern policymaker has access to some of the best academic research but has little data on what happens in classrooms.

In practice this means that in mathematics, for example, virtually all education systems have objectives and programs to improve performance. If we focused on what is happening in classrooms and the impact on student learning, we would collect, analyse, monitor and improve data on the quality of the mathematics curriculum and instructional resources teachers use, the teaching practice they employ with these resources, and the quality and outcomes of their assessments of student learning of mathematics.

The modern policymaker has access to some of the best academic research but has little data on what happens in classrooms.

We would be able identify specific issues of why, for example, grade 4 students in several schools are struggling with learning measurement and the fundamentals in geometry.

We also could see what curriculum and instructional materials were being used, how they were being taught and how student learning of measurement and geometry was being assessed.

Teachers and school leaders also would be asked to share the difficulties they are having teaching measurement and geometry and any blockages in their school – or across the system – that are preventing improvement.

Whenever we work with systems to collect this information we have a rich set of information that highlights tangible and practical ways forward. A minister who can argue for funding to support practical and tangible improvements in classrooms will be well-placed to fight for their budget.

Good policy still should refer to academic research but the focus should always be on the classroom: on the quality of the maths curriculum, on improving mathematics teaching and on mathematics assessment practice. Policy would monitor these changes across time to understand what is and isn’t working.

Across the country, a small but growing group of policymakers and system leaders has started to collect and analyse data on what is happening in their classrooms and the blockages to improvement teachers face. They are developing policies accordingly, then using school and classroom data to track progress against them. They are not ignoring the academic research; they are using it when it is appropriate for what is happening in their schools and classrooms. They are changing the face of education policy.

Learning First, the org­anisation I lead, recently worked with a smaller education system that collected data on what was being taught in its mathematics classrooms, the problems teachers faced and barriers to improvement. Teachers highlighted the support they needed to identify and access the highest quality curricu­lum resources and to improve their teaching.

Policy has been developed to fund and support the system’s best maths teachers to work together to identify the highest quality math curriculum and instructional resources for their classrooms. This approach is education best practice.

More than that, in the coming period of budget repair, its widespread adoption would give education ministers the data they needed to be able to tell their cabinet colleagues that cuts to the schools budget would have these impacts on schools, on teacher practice and student learning. They could tangibly lay out what our students stood to lose.

Showing results

For many education ministers it is hard to point to improvements in classrooms, to identify problems that are being addressed, and to identify the specific teaching and learning issues at stake. In the past, it might have been good enough to argue against cuts to the education budget because it would mean teachers doing less of what academic research said was important. During the next decade, however, if you can’t show what it means for results on the ground then your policy will lose out to those who can.

The next decade of budget repair will be brutal on policies that can’t show impact. It will challenge the dominant orthodoxy of evidence-based policy as defined by the academic research.

Many policymakers will find the shift confronting. The policies they developed in the past would have had big ticks for being based on academic research, but these same policymakers will be criticised for not focusing on what happens on the ground and their programs will be cut, even though they are shining examples of the policy orthodoxy of our time.

On the other hand, a few system leaders are already making profound shifts in how education policy is developed. Budget repair will elevate their systematic collection, interpretation and use of data of what is happening in classrooms, and their work will come to dominate our policy landscape.

Ben Jensen is chief executive of Learning First. A full version of his essay is at theaustralian.com.au.