Schools paid for flawed program

THE NSW Education Department charged its own primary schools at least $42,000 for a program that was found  to have poor results.

THE NSW Education Department charged its own primary schools at least $42,000 for a program it designed to boost numeracy that was found by an independent evaluation to have improved teachers' skills and only "in some cases, students".

The Taking Off With Numeracy program was one of eight programs evaluated that were implemented by NSW under the National Partnership in Literacy and Numeracy for which NSW received about $140 million from the federal government.

The evaluation reports, released to The Australian, failed to find significant improvements in students' literacy or numeracy skills but flaws in the analyses meant that only student results for the whole school were analysed, not the students targeted.

The reports also found that schools failed to properly implement some programs, picking bits and pieces, or introduced a number of different programs, making it difficult to identify the impact of a single approach.

Half the evaluations were conducted by consultancy group Urbis and the other half were conducted by the department.

The federal government provided about $2.5 billion to the states and territories in three national partnerships targeting literacy and numeracy, teacher quality and disadvantaged schools. One of the aims of the partnerships was to build up an evidence base of successful practices to help schools share their experience.

The TOWN program was the most expensive program offered in NSW, costing schools a minimum of $42,000 to train up to 11 teachers and $53,000 to train more than 11 teachers.

The TOWN report says it "delivered numeracy outcomes for teachers and, in some cases, students" but it questions how much of the effect was due to the process of implementation rather than the program itself, which teachers amended extensively.

"It may be . . . more accurate to view TOWN as a 'process' or a 'catalyst' for a changed approach to numeracy teaching, rather than a 'program' as such," it says.

"There was some legitimacy to the strong view about TOWN by a number of school staff that there 'wasn't enough to it', that it . . . did not provide enough value for money."

Departmental staff perceived these views as "a fundamental misunderstanding . . . about the nature of the program".Also evaluated was the MULTILIT (Making Up Lost Time in Literacy) program, which was developed at the Macquarie University Special Education Centre based on research of the effective teaching of reading.

The report found that only half the schools implemented MULTILIT for the recommended minimum four sessions a week, with only 16 per cent of schools providing the full 40-minute sessions and one-quarter delivering sessions of 20 minutes or less. In addition, the three components of the program were not delivered equally.

NSW Education Minister Adrian Piccoli said the evaluations had not been very well done but they highlighted the issue that funding alone would not improve schools and how the money was spent was crucial.

"When you see evaluations done like this, not even done with any mind to rigour . . . it highlights what the problem is in education policy -- it's driven by politics not policy," he said. "It's driven by wanting to be seen to be doing something in education as opposed to doing things."

The director of the Centre for Education Statistics and Evaluation in the department, Jenny Donovan, who did not conduct the evaluations, said they were restricted by the short timeframe.

Dr Donovan said the department was building a website to provide schools with information about programs, their context and their outcomes.