MEASURING EXCELLENCE IN RESEARCH AND RESEARCH TRAINING
Canberra, 22 June 2004

Assessment criteria for excellence in research and research training – Breakout groups
Phil McFadden (Chair)

Today we have listened to some fantastic talks that have just been stunning in terms of informing us and giving us an exploration about the tremendous breadth and complexity of the issue we are facing here. I think that exploration is going to give us the context within which we are going to be able to move to the next phase of today, the breakout groups, to discuss this question of measuring excellence in research and research training.

I would just like to make a couple of comments before we go to that.

Valerie Braithwaite, in her talk, mentioned integrity quite a lot. Now, in terms of the way people behave, there is an old saying: ‘If you don’t have integrity, nothing else matters.’ I think in the game we are talking about here today, if you don’t have excellence, nothing else matters. It is the core, it is the intrinsic need that we have. If you don’t have excellence, you are wasting your time. But although it is an intrinsic thing and it is a necessary component, it is not sufficient to give the extrinsic value that we need elsewhere.

The extrinsic factors will be different in the Victor Chang Institute from those in CSIRO or BHP Billiton et cetera. And we need to be able to sort out this inner core: how do we measure excellence?

We all say, and we have said it several times today, ‘Well, hang on a moment. You know excellence when you see it.’ And there is no question about that. If you listen to Yo Yo Ma you know there is excellence. You know there is not excellence when you watch me trying to run a hundred-metre sprint. It doesn't take you long to figure it out.

We are going to be asked the hard question: how do we actually measure excellence in research in such a way that it will inform funding policy? We want it to inform funding policy, and we want to inform that funding policy in the right way. The issue, then, is: how do we do that?

In many instances it is fairly easy. If you listen to Yo Yo Ma, you say, ‘Well, was that performance wooden, or did it soar? Did it transport me?’ You have got an immediate measure of excellence. ‘Did this guy run the 100 metres faster than anybody else in the world ever has done? Yes? Oh, there you are – excellence.’

In research we don’t know what the true answer is, what the real answer is, so it is very hard to judge: ‘This guy’s response is closer than that guy’s response.’ So we have to come up with a way of actually measuring excellence, measuring that intrinsic capacity in such a way that it will inform our funding models in the right way.

So what I am going to ask you to do is to go away into these breakout groups and in each of those breakout groups we ask you to examine a couple of questions. In terms of developing a quality assessment framework for research training, what must it achieve? Bearing in mind that we want it to inform funding, what must it achieve? We are going to ask you to add to that some concept of how you would go about achieving that, because that is necessarily going to affect what you think it must achieve. Of course, there is also the issue: what must it avoid? I think it is important to examine that aspect as well.

Similarly, for a quality assessment framework in research: what must it achieve? And to some extent please consider how you would go about achieving that. And what must it avoid?

When we come back here the rapporteurs for each group will have five minutes to give a brief overview of what it is that they have done, and we will then go into a discussion to try and suss out those really critical elements of how you go about developing a measurement framework for excellence, for that intrinsic capacity.