Critic and conscience of a national science challenge

By legislation, universities have a role to play as critic and conscience of society. Having recently moved to a professorial post at the University of Waikato, I’m taking up that role to initiate this blog at the Our Land and Water (OLW) National Science Challenge’s 2018 Symposium. The event is on yesterday and today in Wellington, hosted by journalist Rod Oram. The Symposium’s purpose is to design the next tranche of work, likely investing $60 million or or more in perhaps the most iconic issue of national importance, following on $18.6 million invested so far.

This challenge is widely considered to deserve criticism, with the firmest evidence being a lack of clear ways forward on the tradeoff between agricultural production and water quality. Other Challenges won’t be the subject here, except to say that some are considered to be performing well, while others are not performing and may be failing. That’s an important point, because if Challenges were really as big and audacious as they were meant to be, the success of some implies others must fail.

The prospect that Challenges have to risk failure in order to succeed raises the obvious question: how would we know if a Challenge, or a major part of it, is failing?

Here, I’ll argue that the available evidence suggests that OLW is largely failing, and more importantly may be hiding the failure. That’s unfortunate and deserves transparency and examination at a time when the Challenge is designing an even bigger investment in one of New Zealand’s biggest economic and environmental issues.

A lack of transparency is, itself, part of the evidence for failure. 

Success shows itself. That’s not what we’re seeing. The Symposium started out with the governance board’s chair, Paul Reynolds stating, “Sadly I am we are not going to tell you much about the excellent science we have done.” The explanation given is that the Symposium needs to focus on the Challenge’s future.  To some degree that explanation is reasonable, and it was clear that the Board is putting a lot of constructive pressure on the Challenge’s Director Ken Taylor to justify this situation.

Yet, one might obviously expect the future of the Challenge to be based on its successful research so far. The symposium contained nothing about this. That’s stunning when you think about the $18.6 million investment, so far. That investment is explained in a glossy “Research Book”.

I tallied up the investment in the book, and found I could quite easily classify the work into 2 categories: 1) innovative and new; and 2) business as usual, or worse, including approaches that have failed in the past.



Each theme contains a mix of projects ranging widely in size. The bottom line is this: the Challenge continues to be dominated by existing thinking rebranded into Innovative and Resilient Land and Water Use Theme, with some of the larger projects trying things I’ve seen proposed or tried multiple times over the past 15-20 years. There’s no evidence they’re on track now. But they are connected to the consultancy activities that help fund Crown Research Institutes. 

Due to the lack of evidence that new science has created ways to guide the Challenge forward, I  worry that these are the same approaches that allow research institutions to keep clipping the ticket on problems, without offering solutions. As a major review of our research institutes concluded some years ago, but only temporarily corrected, “This emphasis encourages CRIs to deliver $1 million in profit to their bottom lines rather than $100 million to New Zealand as a national benefit." National Science Challenges emerged from recommendation 9 of that review, so it seems reasonable to be concerned if its implementation has undone successfully implemented recommendations. 

Equally concerning is the lower level of investment, or level of guidance provided by the Greater Value in Global Markets and Collaborative Capacity Themes. These contain largely new and important work. Yet, they’ve had a slow start. Sadly, the Challenge now proposes to invert its mission statement, so enhancing value of what we export is no longer in the lead. It remains hard to see how else the cost of changing land use and management can be afforded.

Contestable programmes have provided a mechanism to inject new ideas into Challenges, including the large body of work on land-use science, but are criticised for a couple reasons. First, they’re another funding mechanism in an already crowded and disorganised space. Second, and in this Challenge in particular, they seem chronically underfunded. Underfunding can be a mechanism of crippling new ideas, rather than shepherding them to success, when they effectively compete for Challenge funding with 'business as usual' projects. Even worse, some contestable projects appear to be poorly reviewed, with one from GNS Science proposing that a new method was needed to quantify denitrification when a good number already existed. Another technique available within GNS had big wins globally (historically, and recently, for example), but was excluded from the challenge. Observations like this lead to serious concerns about the Challenge's formation, ongoing review, and contestable processes. How can the Challenge do better at fostering rather than frustrating the pathways to new science all the way to delivery of excellence, and the transformational change that is needed.

Sadly, the process for new projects presented as "pitches" at the Symposium's first afternoon raised a wide number of concerns. The process was primarily a roadshow that most researchers found out only 1-2 weeks before meetings across the month of February. Many had other commitments, and few had time to consult with colleagues nationally or stakeholders. I'm particularly concerned that the process disadvantaged teams working nationally, or with pre-existing commitments during the roadshow dates. Although there were some opportunities to inject additional ideas outside the roadshow process, there was still a flurry of last minute activity pulling pitches together for yesterday's session. Some were very good in terms of potential, others frankly disappointing. 

Combined with three useful and highly relevant, yet rather disconnected think pieces from Ronlyn Duncan, Stephen Flood and Guy Salmon, the pitches dominated what the Challenge provided to over 100 researchers and stakeholders to organise its future. This was carried out through a series parallel workshops, sometimes known as "post it note hocus pocus". This mirrored, but was slightly altered from the processes that formed the Challenge's structure. I've commented previously, and will say again that these processes are great for individual $1-3 million research programmes, but don't deal with the complexity of so-called "grand challenge" problems. As I've done in that past, I can point to a TED talk and a web-based example that illustrate more appropriate processes for challenges of this scale.

I could possibly find a few other useful points, but I'll come back to my main point: the Challenge can be more transparent, better organised, and more engaged. Useful comparisons can be made across Challenges, and also across other funding mechanisms. We now know, for example, that getting a Marsden grant makes a researcher more successful, when compared with applicants who just missed out. One thought experiment we can and should conduct, is to ask whether a project, a researcher or a desired outcome be better developed inside or outside this Challenge (if funds were released to other mechanisms). As a critic, I'd suggest that releasing funding to other mechanisms might be more successful.

Doing so could be seen to undermine some of the hope that comes from Challenges and their workshops. As Rod Oram commented at the outset of today's session, yesterday featured "a great sense of excitement in the room.” The level of optimism and engagement continued right through today. The question is, can it be harnessed? And is it realistic?

There are some great reasons for enthusiasm. One great initiative is Next Generation Influencers, a group of leading young minds engaging on the Challenges issues. Other young people also provided excitement and energy today, and represent a view that the literature tells us that successful transformation takes 19.5 years on average. So, as Director Ken Taylor enthused at the beginning of the day, we should invest in the young and in capability. But if the Challenge wants to achieve fast progress on the goal I've heard, of doubling the pace of transformation, should it be seeking tighter and deeper relationships with research-informed teaching at the universities that can shape graduates in 3 years?

Making a Challenge's enthusiasm realistic and enduring also means making a convincing case for delivering, rather and not using enthusiasm and big new words or untested ideas (like co-innovation) to mask non-delivery. Has this Challenge, its lack of transparency, and domination of the funding space led almost all major players to avoid criticism? This was a concern voiced in Silencing Science. I hope that pushing this out will urge others to consider how research challenges at this scale can be monitored, criticised and improved.

As a (hopefully constructive) critic, I encourage the Challenge, its leaders and its governance board to  think hard about the push points and sequencing required to achieve greater focus and clarity. This, and greater transparency can harness the excitement of the Challenge Symposium. Doing so will redouble, and certainly won't undermine the Challenge's success. 

This work is licensed under a Creative Commons Attribution 4.0 International License.