PBRF: What's hiding in the hype?

Results from the PBRF Quality Evaluation (QE) came out last week, and Universities began immediately to spin the result to their favour. Do they (the people claiming to represent the university) get it right? Should everyday academic staff be questioning, now that universities have pitched their spin on the results?

This matters, because $173 million per year is at stake, of which $6 million goes to other tertiary providers and $167 million goes to universities. So I'll focus only on Universities here, but congratulate other tertiary providers who got a piece of the pie. Also, let me note this is just the QE and not the other large part of the PBRF pie linked to ERI (research income) and RDC (degree completions).

The hype: this tends to be universities finding and pushing their strengths. If I listen to this at Waikato, I hear about subjects that held their place, and those that increased.

But let me just say, "Wow!" .. um ... no one is saying we lost almost a million dollars out of our indicative budgets for next year!

	Indicative		Change $
TEO	QE 2019 $M	Movement	2018 - 2019
University of Auckland	47.98	-2.17%	($1,042,617)
University of Waikato	8.50	-10.85%	($920,845)
Massey University	22.36	-8.59%	($1,919,688)
Victoria University of Wellington	19.36	8.42%	$1,634,698
University of Canterbury	14.52	-15.98%	($2,325,400)
Lincoln University	4.59	-0.75%	($35,030)
University of Otago	35.80	-3.22%	($1,154,003)
Auckland University of Technology	12.62	33.34%	$4,216,577
Sum --> University Sector	165.73	0.21%	-$1,546,309

So, where is this hiding and what can we do better? Look at AUT - up over $4 M in their budget for next year.

It seems there are two stories there. Both hide in the points system.

First, a huge amount of work goes into categorising each individual researcher, and then a simple point score is assigned. That's simple: A=5, B=3, C=1 ... but C(NE)=2. So what's that last one? A new and emerging researcher (post-doc) counts for two... nearly as much as a well established national class researcher (B=3). And this is a change from 2012, when C(NE) was separated but still 

So let's look at how these played out. 

2018 Distribution of funded Quality Categories by university
TEO NAME	A (FTE)	B (FTE)	C (FTE)	C(NE) (FTE)	TOTAL
Auckland University of Technology	47.5	251.63	265.29	124.81	689.23
Lincoln University	22	74.7	57.25	22.58	176.53
Massey University	131.07	419.61	375.14	88.77	1,014.59
University of Auckland	390.64	733.63	343.3	276.6	1,744.17
University of Canterbury	105.56	256.38	124.12	110.4	596.46
University of Otago	228.67	555.69	347.66	225.63	1,357.65
University of Waikato	60.26	203.05	116.05	49.11	428.47
Victoria University of Wellington	172.92	399.51	204.46	87.68	864.57
TOTAL	1,158.62	2,894.02	1,833.27	985.58	6,871.67
2012 Distribution of funded Quality Categories by university
TEO name	A (FTE)	B (FTE)	C (FTE)	C(NE) (FTE)	TOTAL
Auckland University of Technology	19.16	132.1	183	95.21	429.47
Lincoln University	18.2	51.7	94.4	9.8	174.1
Massey University	94.65	341.69	358.69	123.59	918.62
University of Auckland	288.61	635.63	405.09	226.72	1,556.05
University of Canterbury	80.84	269.8	162.32	104.3	617.26
University of Otago	179.56	504.11	290.98	193.59	1,168.24
University of Waikato	38.72	200.87	156.53	44.51	440.63
Victoria University of Wellington	111.59	339.52	130.55	59.88	641.54
TOTAL	831.33	2,475.42	1,781.56	857.6	5,945.91

Ok, so AUT increased all their funded categories, with big gains in A, B and to a lesser extent C(NE). I'm a bit surprised at universities who play up their ratio of A:C or (A+B)/C without noting C(NE). Ensuring places for postdocs remains one of NZ's biggest long-term research problems. This matters for environmental sciences because knowledge of biology, earth science and policy system is place specific, and recruitment from overseas provides no easy salvation for an unsustainable population of researchers which doesn't train good replacements.

Universities probably could have done more than they did with C(NE).