New ways to target quality – comment by CEO Craig Robertson

If a plane left Canberra for Perth by the most direct route but was off direction by just one degree, it would miss the airport by 51 kilometres. 

A rocket to the moon is more exacting. Just one degree off target has it passing by the moon by 6,406 kilometres – or over two moon-widths.

The exactitude of the destination demands good steering. This was the message from a report from NCVER this past week – Begin with the end: RTO practices and views on independent validation of assessment – a number of elements of the VET system need to be aligned to get to good assessment.

The Productivity Commission in its review of the National Agreement for Skills and Workforce Development Agreement recommended independent validation of assessment be explored, although qualified its use for areas where there is public benefit like aged care. The VET Reform Roadmap has senior officials considering a new industry endorsed system for independent, moderated and/or graded assessment. Trials underway in Victoria may inform a new approach.

Consider the flight path to quality assessment, given VET curriculum logic is based on an exact landing position for all graduates. I have written previously that the combination of elements, performance criteria, foundation skills, performance and knowledge, evidence and assessment conditions prescribed for a single unit of competency can easily add to 100 separate functions which must be assessed, documented and stored for audit. If a qualification usually comprises about 14 units, you can do the maths. The 58 units in plumbing literally must be sending trainers around the bend.

As of yesterday, there were 15,660 units in the national register, which means there are around 1,566,000 assessable functions in the national VET system.

If there is this degree of specification, it’s hard to see the need for independent validation. The clash in expectations is one of the key messages of the report:

Independent validation of assessment is driven by regulatory requirements, which can generate a compliance mentality, leading to over-assessment, but not necessarily better assessment practices or improved training.

The journey to quality assessment has logistic and logic barriers, especially with engaging with industry and employers.

Industry representatives say that assessment is the responsibility of providers, nor do they have the time or skills. The report concludes, “…..it does seem difficult to get direct engagement of industry in the development of assessment tools, usually devised by a subject matter expert and experts in assessment and training packages, or as independent validators.” As one provider commented – “They (employers) don’t want to be involved in the examinations − it’s hard to get them even to come along to a breakfast.”

As to the logic, the report says, “industry engagement can reveal shortcomings in the training package requirements.” Therefore, what extent is there for employers to vary the assessment elements given the specification of functions which much be assessed?  Where two or more employers are involved it is hard to see them agreeing to all the functions, whether they are in the unit or not.

Before the author of the report shoots me, it’s important to point out the purpose of the report was to examine the practices of RTOs in validation of assessment. I’m simply taking the analysis further. Nor am I debunking independent validation. If it is part of internal quality processes of a provider intent on continuous improvement, it is fantastic and would feature strongly in their self-assurance strategies.

The danger is regulatory over-reach. After all, senior officials have on their agenda – a new industry endorsed system for independent, moderated and/or graded assessment.

If senior officials decide that the answer to quality lies in the quality and assurance of assessment, which is a reasonable position to reach, there needs to be legitimate trade-offs. The obvious response is to de-clutter training specification and free-up delivery modes and options. This would take the system back to the true application of an outcomes-based philosophy embedded in the standards approach to qualification specification.

If, on the contrary, end design has industry in some form engaged in a determinative function in addition to training packages the bodies will need to be regulated. It is implausible to hold an RTO to account for assessment (after all they issue the testamur) if it can be over-ridden by any entity which is not equally regulated. Otherwise, fairness and appropriateness of the actions and decisions, which is the very essence of assessment, is lost. And, of course, who bears the cost needs to be dealt with.

The better answer surely is trust. Professor Valerie Braithwaite in her report All Eyes on Quality laments that quality assessment relies on ‘trust chains’, from occupational specification, to delivery, to assessment, to quality assurance by ASQA, and clearly ASQA can’t be there for every assessment. It is understandable for training package developers to insist on specifying delivery requirements given the rocky road of VET when it comes to quality. Mandated assessment adds new validation costs to training package costs, with little guarantee of improvement and little financial support if the history of VET funding is anything to go by.

How to engender trust? Offer it where it is deserved. There is little incentive in the system for providers to pursue quality in their own way, apart from the false promise that open competition creates room to stand-out on quality grounds. Exhibit 1 – VET FEE-HELP.

Begin with the end provides examples of good processes providers have in place now. And ASQA is offering self-assurance as the strategy to underpin this approach. So, it is feasible.

This  approach has more chances of landing on a quality outcome than more regulation. If the 4.2 million students in VET in 2019 accessed on average five units that’s 2.1 trillion pieces of functional evidence sitting within the system! I suspect that’s enough specification and regulation.