STEM not rooted in groundwork

A multimillion-dollar strategy to boost science and maths capabilities is being ‘undermined by a lack of ­robust data on effectiveness’.

Just one in five of the 69 STEM programs analysed have undergone any external evaluation to ascertain their effectiveness.

A multimillion-dollar national strategy to boost science, technology and maths capabilities among Australian students is being undermined by a lack of ­data on the effectiveness of the STEM programs targeted at schools.

Dozens of programs ranging from science fairs and coding camps through to professional mentoring and new curriculum resources for teachers have ­received funding in line with the Nat­ional Science, Technology, Engineering and Mathematics (STEM) School Education Strategy 2016-26, which aims to ­improve students’ achievement and encourage more senior students into advanced science and maths subjects.

Some have attracted tens of millions of dollars in government funding.

But according to a new report ­released by the Council of Australian Governments Education Council, just one in five of the 69 STEM programs analysed have undergone any external evaluation to ascertain their effectiveness. Most relied on internal evaluation — largely surveys or feedback from participants — with 15 per cent having no evaluation processes in place.

“There is very little robust data on which to assess the outcomes of the initiatives,” a statement on the Education Council’s website said.

The report, which comes amid ongoing concern about the declining performance of Australian students in international STEM assessments and dwindling student participation in senior advanced maths and science subjects, raises serious questions about the 10-year strategy, particularly in regards to its stated aim to establish a stronger evidence base around what works in Australian schools.

It calls for STEM initiatives to be evaluated wherever possible, and the results made public, so that “money that is being spent on an underperforming initiative can be redirected to another program that will achieve better results”.

“There are many STEM initiatives that have not yet been evaluated, and some that may not be evaluated at all,” it says. “An even greater number will not be externally evaluated, generally the most reliable form of evaluation.”

Australian Mathematical Sciences Institute director Tim Brown said he supported better reporting of the impacts of initiatives that received public funding.

Professor Brown said the institute’s own schools program, Choose Maths, which is funded privately by the BHP Foundation, had undergone multiple evaluations pointing to its effectiveness in increased awareness among students of the importance of maths for a range of careers.

“We are absolutely on board in regards to the importance of evaluation,” he said.

“Also … starting a program and running it for a few years and then stopping is likely to limit its effectiveness.

“If we want to change students’ attitudes to studying more ­advanced STEM subjects, we’re going to have to continue this hard work in the longer term.”

Greg Ashman, a senior secondary maths teacher who has published a book on evidence-based teaching practices, questioned the focus of STEM intervention outcomes on survey results, such as whether teachers liked the program or changed their practice rather than improvements in students’ achievement.

“This is hardly the point,” he said. “More broadly, the report notes that many of these initi­atives seek to inspire or engage students in order to improve the STEM achievement or uptake of STEM electives. This is based upon the theory that motivation comes first — first you motivate the student, then she or he will achieve.

“However, there is plenty of evidence that … first you help the student to achieve, then they will become motivated. We like being good at things, after all.”