MEASURING EXCELLENCE IN RESEARCH AND RESEARCH TRAINING
Canberra, 22 June 2004

Excellence in mission-directed research
Dr Michael Barber

There is a bit of a danger this morning that we are going to cut up the research enterprise into various silos, and I guess that is almost inevitable, but then I hope the afternoon will start to pull the various threads together. So I am going to again adopt that view. I am going to try and sketch out some of the dimensions of excellence in the topic of mission-directed research.

I guess I had three objectives in the next 15 or 20 minutes. I wanted to give you some characteristics of what we regard as mission-directed strategic research; I will then use the CSIRO’s National Research Flagship Program as, if you like, an exemplar of that in the Australian context, although I will draw a little bit on international comparisons and other exemplars; and I will then spend perhaps the last part of the talk talking about what does it mean to be excellent in this sort of research, what are the possible metrics and, in particular, the role of peer review.

As I have already indicated, there is a danger in what I am saying that it will take on a sort of a life of its own, and I think one of the issues clearly before us is the complexity of the system and the fact that, in a way, for Australia we need to have excellence in a whole range of different types of research. The balance of that system is part of what we need to think about and it should not be a consequence, albeit unintended, of any assessment process to unproductively destroy the balance – though we may want to come back and debate the balance a bit.

So if I put that as a precursor, I am going to focus very much on what internationally has been called mission-directed research.

What do we mean by mission-directed research? It is really R&D directed, as the title would suggest, towards some specific application or outcome area. It is not directed towards a scientific or technical endpoint; rather, towards enabling the applications or the mission to be achieved. Its phrase probably goes back to programs like NASA’s man on the moon, clearly aimed at an auditable mission for which the success measure was to land a man on the moon by the end, as you might recall the John Kennedy statement, of 1970. But the outcomes are the critical part of it.

It does not imply that you cannot do excellent and, in fact, Nobel Prize-winning science – if you put it within the science context – in mission-directed research. Teasing out those dimensions is, I think, important.

What are some of the characteristics that you might associate with mission-directed research? Again these are not by any means unique to mission-directed research, and in many ways the difference as one goes across the spectrum from the investigator-initiated research that Bob Graham talked us through to this sort of research, to the research that Bob Watts will talk about later, inspired out of industry, is really a matter of degree rather than obvious cut-offs.

The characteristics of what I will call mission-directed strategic research are a large-scale, long-term focus on national or critical problems, usually performed by transdisciplinary teams. The objective as an outcome is to solve some problem, to deliver some outcome. They tend not to be problems which immediately define themselves as the province of chemistry, the province of biology. Generating a cotton industry for Australia, which was a mission-directed program of CSIRO’s Plant Industry, didn’t require just of plant scientists but also agriculturalists, across that. The disciplines that are needed are the critical part of it.

It is driven, as I have said, very much by outcomes and needs, and therefore very critically actively managed towards adoption. And I will talk a little bit more about that ‘critical’ thing.

The measures that you need to consider are clearly the science excellence – or perhaps more broadly today, since mission-directed research tends to involve societal impact and therefore our social scientists are becoming critical elements of the teams which address the problem – to which must be added operational excellence and in particular the outcomes and how we demonstrate outcomes. In the Australian context and again globally, these tend to be strong in the publicly funded research agencies. In fact, I would argue that without strategic mission-directed research there isn’t really a logical role for a CSIRO. It is similar for ANSTO or AIMS or, in the US, the Department of Energy national laboratories, or the office of NASA’s research, as examples.

DARPA’s ‘end-game’ approach
(Click on image for a larger version)

There also, in a sense, are design features which I have tended to take from this model from DARPA, the Defense Advanced Research Projects Agency of the United States. It distinguishes one of the aspects which I think we do need to distinguish.

In the Defense Advanced Research Projects Agency of the US Department of Defense, they really are mission directed. They are initiating part of the future. What are the new products and processes they need? Can they develop a detection mechanism to see through a few hundred metres of the surface of the ocean? Can you detect infrared signals? Can you specify those end products that they would want? Can you specify new materials? So it is to anticipate the needs, come down to the present and put together the research teams that will actually deliver those outcomes and then, in particular in the mission-directed sense, manage that process to the end.

It is the complete flip of Bob Graham’s description of investigator-driven research, where the ideas flow from the researchers and the research teams, are evaluated by a peer review process and lead to research outcomes which may or may not be picked up by industry or commercialisation and may or may not deliver towards those new processes. This is an attempt to take the processes and the end requirements, and drive very hard towards the end.

Peer review plays a role, and we will come back to where peer review plays a critical role. Excellence plays a critical role, but it is the other end of that dichotomy of whether it is initiated at one end under ‘research teams’ on the left of the slide, or whether it is initiated by the outcomes.

CSIRO’s Flagship Programs, I would contend, are in fact a classical example of mission-directed research, and I want to tease out a couple of the general features of them before I turn to the question of excellence, because again these are very much part of the differentiation of the system.

They are clearly focused on major national issues. They have got ‘big hairy audacious goals’, to use Geoff Garrett’s phrase, and they are a key driver of our alignment, in fact our response, to the National Research Priorities which we were as a Commonwealth agency required to take into account in our research planning. They have scale and impact, and will over five years consume something like 30 or 40 per cent of the appropriation of CSIRO. But they are not just about CSIRO. To build that scale we do need to reach out, to bring together ‘Team Australia’, to use Geoff’s phrasing.

They are very much outcome-focused and outcome-directed. Their success will not just be measured by excellence in the science we do, but will be done in fact by achieving outcomes. And, critical to that, there is an active management throughout the program towards adoption.

The Flagships
(Click on image for a larger version)

I am not going to spend too much time on the Flagships; they have been well publicised. I have just put up their missions, if you like, because I want to pull out of them a couple of key features. For example, the Flagship P-Health is to improve the health and wellbeing of Australians and save $2 billion in annual direct health costs by 2020 through prevention and early detection of chronic diseases. You can read the other ones, but for each one of them you will note that there is a quantifiable indicator of global success at some point in the future – Light Metals: to double export income and generate significant new industries, Food Futures: to transform the international competitiveness and add $3 billion annually to the Australian agrifood system.

The Flagships
(Click on image for a larger version)

The other three – Energy Transformed: to halve greenhouse gas emissions and double the efficiency, Water for a Health Country: to achieve a tenfold increase in the social, economic and environmental benefits of water in 2025, and the third is Wealth from Oceans.

There are two remarks I want to draw from that. One I have already alluded to. Firstly, each of those missions has a quantifiable indicator of what you might actually say is success at some time in the future. Secondly, it is not only science and technology which is going to give those outcomes, and it is not only CSIRO that can clearly deliver them. We cannot give a tenfold increase in social, economic and environmental benefits from water by 2025 with science and technology alone. Therefore, it is critical in that case that we think through the processes of adoption, the issues that need to be addressed in achieving those outcomes, because science and technology alone, and science and technology along with our social sciences that are critical in many of these alone, are not going to achieve it. It could be regulatory, it could be tax arrangements, it could be government policies – a whole slew of issues need to be, in fact, addressed.

I am not going to drill down through the Flagships to show how these objectives are translated into detailed plans; that is a talk in its own right. But again the critical aspect is that flowing through the Flagships are these two dimensions of strategic research: a focus on the research that is going to lead to those outcomes, but secondly a critical and equal weight on what are the steps we need to take to achieve adoption so it is not something left for the future. So Tony Filmer, the director of Light Metals, whose ultimate goal is to develop a titanium industry for Australia, is already beginning to think about and talk to industry partners about what a pilot plant might look like, what a pilot plant might cost, long before the R&D that would be going into that pilot plant is in fact anywhere near fruition. The blending together of science, the research and the outcomes is critical.