MEASURING EXCELLENCE IN RESEARCH AND RESEARCH TRAINING
Canberra, 22 June 2004

Groups report back
Phil McFadden (Chair)

Jim Angus (Group 1)

We started with research. We believe that what we must achieve is public confidence, and that there is value in our R&D because what we do in our research sector will apply, as we said, across to defence, security and many other things.

We need some clarity when we are talking about quality versus applied research, the application. We need to develop, within our own sector, our scorecard – whether it be university, CSIRO or whatever – and within that scorecard we have to indicate what are the priorities for that sector.

We spent quite a bit of time talking about the concentration of minds. What do we believe that means? Is it at the department level? When it comes to an institution, would centres make up the quality or the concentration of that institution? This is particularly important if we think about block funding: would there be a university, for example, that may only be excellent in one centre, and therefore that would be its baseline for funding, rather than thinking about whether there should be a baseline across the sector in the universities for block funding.

Then in considering our quality we were interested in discussing where we are going to get the expert advice. And we must not forget the international competitiveness of our research, and therefore of course not be at all shameful or worried about going for an international panel.

That was what we must achieve in relation to research. What we must avoid, we thought, was the ‘soft’ or the ‘safe’ research that often is developed from a research fellow who may have one technique and continue to slice that salami for the next 10 years. We want researchers to take risks and develop their own research teams in that way.

We need to avoid the international cringe. If we are publishing in international journals we are doing international research, by that peer group that assessed that research. We have to avoid, in any research assessment exercise, the abuses that occurred after the first phase in the UK. We had a quiet chuckle about whether or not, whatever we bring into Australia, there will be abuses and people will try and avoid (or use) the system. So we need to be preparing version 2 as we start version 1. And we need, of course, to avoid any heavy workload – use the light feather touch – on any research assessment exercise.

As far as research training is concerned, we believe that we must, in our new PhDs, recognise the changed culture of research in Australia and therefore develop in our graduates these broad attributes. Mainly, of course, today it is around the teamwork, particularly in the sciences but also within the humanities. We don’t want the PhD students, because of lack of facilities, to do their PhD at home. They have to come into the university, and one way of warming them up is to get them to teach. So some teaching is terribly important, particularly in the humanities but also in our sciences – to get them into the research labs and the practical teaching labs as we do.

We need, of course, to encourage this capacity to write. It is not only research grants but it is the mini-review that leads on to the important review as it leads into the first chapter of their thesis.

Finally, we really need to look at the better resourcing of our research students. It is not good enough to accept research students in a laboratory at the moment unless you have got other grants, because you are needing those other grants to cross-subsidise. So I believe that is a really important area that Australia needs to look at carefully in resourcing our PhD students.

What we must avoid, I think, is the pressure to complete in four years or less. This is coupled with the 3–3½ years of the APAs, and if we are serious about developing a good PhD graduate we certainly need, in most cases, more than four years to do so. We need to develop opportunities to stop a PhD student, I believe, at the end of the second year and make a very tough decision in many cases, where you might say, ‘Let’s complete a Masters degree and leave the system.’ I think for many students we would be doing them a favour.

Lack of resources again we must avoid. It is not just the wet labs but also spaces for PhD students in the humanities. Many universities are struggling there with the increase in numbers.

And, finally, we must avoid poor supervision. It is all about developing that relationship with a very good supervisor in a good laboratory that builds to that excellent centre which is going to develop so much and be so important in developing the quality of our PhD graduates.

Tom Clark (Group 2)

Most of these points are more or less by consensus within the group, but there might have been more disagreement than time allowed us to explore.

We made a couple of preliminary observations, first noting that a pivotal strategy in achieving Finland’s outstanding performance on the graph we saw was targeting ISI journals; that has boosted their numbers hugely. We also began with a questioning of the notion of so-called research training. Leon Mann put it much more dispassionately than I would when he said, ‘It’s much broader than that.’

I am afraid we kind of flowed from one question into another, but I will try and let you know when that happened.

On Question 1, we thought that postgraduate research education should lead to maximal graduate satisfaction. I hope that is pretty clear. The next point in our summary is about ANU but I will come back to that, because actually the ANU has done a major exercise recently.

We have to account for the diversity of higher degrees by research. That is a diversity of modes, a diversity of disciplines, a diversity of ways of interacting with campus and off-campus experience, of industry versus traditional, et cetera.

We must take account of the quality of research that higher-degree research candidates undertake. This goes back to that question of research training. HDR candidates are researchers, and they are trying to prove something about their ability as researchers at the same time as they are learning.

Probably still on Question 1, but starting to fade: we need to accept the use of proxies if we are taking a metrist or measurement approach. I think most of the points here are a bit even-handed about the question of whether we are taking a metrist approach or a peer review dominated approach – the current numbers system or the mooted possible alternative RAE style system – but this is one point specifically about metrism.

Back on research training, so-called: we need to avoid a fixation on completions for their own sake. I can think of an adviser who had carriage of David Kemp’s research policies, Andrew Norton, who was a very proud non-completion in political science.

This is definitely moving more generally into research. We need to develop measures and expectations that are calibrated finely by discipline. We need to be clear about the difference between measuring quality, which is a relative judgment, and detecting excellence, which is an absolute judgment. I don’t think I was alone in the room, in the sense that if you talk about measuring excellence you really need to deal with a c