Scott of the anarchic

One delights of the EMAC annual is conference combating with Scott Armstrong on the occasional years when this great and controversial scholar has a paper accepted. This year the reviewers got it right, and Scott was there to fire off his challenges.

Scott's joyful challenges must have been a late development; otherwise, his parents would have nailed his head to the floor. This years favourite quip was: "Have you learned anything?" This led to one little exchange about how concern for teaching quality damages research quality and how the best researchers are awkward people. Certainly, some excellent researchers tap into a reservoir of selfishness and arrogance, but others retain the common touch. Other generous souls find reward in helping others and are excellent teachers. Unwilling to accept the idea that there is a positive link between excellent teaching and research, Scott requested a paper. Well Scott, here it is!

Hurray Hooley or wrong about Wong?

As the head of a business school, I have to admit that it would make life simple if teaching quality and research quality were negatively correlated. Unfortunately, they are not. Even worse, the excellent teacher-researchers are often excellent managers and ambassadors for the business schools. Because they both work in my Business School, I can give two examples two such paragons: Graham Hooley, EMAC's new President, and Veronica Wong, EMAC's Vice-President (Conferences). Both achieve excellent rating for their postgraduate teaching, have developed numerous young academics, and are accomplished leaders. Veronica leads Aston's marketing group and Graham is Aston University's Pro-Vice Chancellor. Incidentally, both Veronica and Graham were part of the late Peter Doyle's team at Bradford where he was an outstanding example of an excellent teacher, researcher, academic innovation, entrepreneurship and getting really rich!

Are the two EMAC executives examples of a pattern or aberrations? I use three sets of secondary data to explore the issue central to Scott's concern: the relationship between teaching and research. The first looks at individuals using information from a single business school. The second is published data on the teaching and research quality of UK business schools and the third set compares the overall teaching quality and research quality of UK universities. In all cases, teaching and research quality are independently measured.

Person by person

Like many others, Aston Business School measures the teaching and research performance of all staff annually. The process is taken very seriously since it informs a substantial Performance Related Pay system, sets the funding level of Subject Groups, individual teaching loads and promotion prospects. Teaching quality is measured by the Quality Unit using information from student feedback, Programme Directors, other colleagues across the school and the leaders of Subject Groups. The measure is a four-point scale ranging from zero to three where zero is unsatisfactory and four is outstanding.

A two by two cross tabulation of high scores (>2) for teaching and research cross-tabulated against the low scores shows a pattern (Table 1).

Table 1: Individual teaching and research ratings compared
	Research	Teaching quality
	Quality	Lower	Higher
	Higher	13	29
	Lower	17	17

The relationship is not overpowering (χ2 = 2.85 significant at 0.1 level) but it does suggest that the people with lower research output have an even chance of being a top rated teacher. The ratio for top researchers is 2 to 1 in favour of them also being top teachers.

Although this evidence does support the idea that excellence in teaching is correlated with excellence in research, the data is not in the public domain so is hard to check. That is not the case for the school-by-school and university-by-university analyses that follow since both use published information.

Business school by business school

The research quality and teaching quality of UK business schools is compared using data from two independent exercises conducted by the UK's Higher Education Funding Councils. Although each of the four countries in the United Kingdom have their own funding councils, they distribute university funds differently, until recently they all use the same method to assess the teaching quality and research quality of all university departments.

The Research Assessment Exercise (RAE) periodically reviews the research output of all University departments at the time. Business and Management covers most of all business schools' output although there is a separate panel for Accounting and Finance. The RAE 2001 comprised eleven academics and one practitioner that covered all disciplines who conduct a paper based exercise. The overall grading they gave to departments considered four publications chosen by each academic in a school, research grants obtained, PhD programmes, research strategy and other measures of esteem. The original ratings use a seven-point scale that shows signs of compromises and adjustments: 1, 2, 3a, 3b, 4, 5, 5*. At one extreme 50 percent of the output of the top 5* rated schools is rated of international standing whereas 1 rated schools have minimal output of national standing.

The RAE has real financial teeth. The Government aims to concentrate research funding in a few top Universities so only the minority of schools rated 5 or 5* receive significant research funding.

Until recently, Quality Assurance Agency (QAA) assessed the teaching quality of university departments over a long cycle of years. The QAA's methods changed over time although they consistently used a panel of assessors in a similar fashion to EQUIS. However, here EQUISfs panel is usually very senior academics that rely on their experience to guide them; QAA assessors are often middle ranking academics who QAA train for the exercise. AACSB International accreditations are a hybrid that uses trained senior academics.

The QAA results have never had the direct financial implications of the RAE but the detailed results are widely published effect rankings in the UK. In its original for the QAA in England, Wales and Northern Ireland rated departments as Excellent, Satisfactory or Not Satisfactory, although there were very few of the latter. Scotland added a Most Satisfactory category. More recently, results appeared using six criteria that could award a maximum of 24 points. It is generally accepted that Excellent in the old system corresponds to 22+ points in the new system.

To see if research and teaching quality are related we compare the QAA results of the Internationally RAE rated school (5 or 5*) with those of the schools rated from 1 to 4 (Table 2)

Table 2: The Research Quality and Teaching Quality of UK Business Schools
	Research Quality
Teaching Quality: QAA score	Not International Standing: RAE 1, 2, 3a, 3b or 4	International Standing: RAE 5 or 5*
Excellent or 22+	28	12
Satisfactory or < 22	41	4
ƒÔ2 = 6.2 significant at 0.05 level

Eyeballing the table suggests that excellent teaching quality occurs more frequently in research excellent schools than in those that are not. A more detailed breakdown of RAE and QAA scores indicate an interesting pattern. There are only three top (5*) rated schools but all have excellent QAA scores. In contrast double jeopardy occurs for 4 rated schools who just miss out on generous RAE funding and have a lower proportion (7 out of 21) QAA excellent scores than any other research category.

University by university

Britain's Higher Education Funding Councils do not publish league tables or ranking but do publish a mine of performance indicators for those that want to. One of the most respected institutions that does so is The Times who in one week each year publish their results in the newspaper as well as their sister publications The Times Higher Education Supplement and The Times Good University Guide. Among lots of other manipulations, The Times aggregates the teaching scores and research across universities. The number of items is actually higher than the number of business schools in the UK because a few universities still do not have business schools (Figure 1).

Figure 1

The plot looks conclusive and represents data with a correlation coefficient of 0.675 (significant at 0.01 level). Regressing the RAE scores on those for the QAA gives:

QAA = -22.99 + 1.26 * RAE (r2 = 0.46)
(-7.67) (9.02)
.
with both t-statistics significant at a 0.001 level. These suggest that for every 1 point increase on the research quality score, expect a 1.26 increase in the teaching quality score.

Conclusion

I am tempted to end with a one line conclusion: "Scott is wrong" but I am not one to gloat. All the results suggest that teaching quality and research quality are positively related at an individual, business school and university level. Why should this be so since spending time on research must reduce the time spent on teaching? There are several reasons. Firstly, excellent researchers are likely to be better informed than those who are not since publishing at the top-level forces people to be up to date and be clear thinkers. Secondly, both teaching and research excellence are driven by people with a desire to excel and, thirdly, both teaching and research are helped by overall ability. Certainly the leading research schools and universities can attract excellent staff who, given the motivation can turn themselves to the task in hand.

The UK environment may have particular features that make teaching and research quality related in a way that they are not elsewhere. As mentioned earlier, the RAE rewards excellent research at a school and university level. In the top rated research schools almost half the salary is paid directly for research, meaning that researchers in the top schools need to teach less to cover their salary than do people in teaching dominated business schools. In many markets the top research schools also have the ability to charge higher fees so, again, are better resourced. Finally, the positive correlation between teaching and research might exist in the UK because both are measured and published widely. At the individual level within Aston, the Performance Related Pay scheme rewards teaching and research quality equally and a highly respected Director of Research and a Director of Quality champion the areas. At the business school and university levels, the funding council promotes both ideals and uses independent dedicated mechanisms to do so.

As a marketer, I am relieved teaching quality and research quality can abide side by side since, in the long term, it is hard to champion quality in bits. The relationship also closes a virtues cycle that can be a foundation for a school's strategy. An excellent research reputation helps attract excellent researcher and students; the excellent students rightfully demand excellent teaching; the combination of excellent teaching and researcher enhances an excellent reputation and attracts more students who are willing to pay higher fees for the experience; the higher fees pay the resources to support excellent research...

Scott, have you learned anything?

John Saunders

EMAC Fellow
Head of Aston Business School

1. The process and results of the exercise are at http://www.hero.ac.uk/rae/
2. The QAA process and all its reports are at http://www.qaa.ac.uk/
3. Many of the performance indicators are at http://www.hefce.ac.uk/learning/PerfInd/ although other are at other parts of HEFCE's web site.
4. Andrew Hindmarsh, Bernard Kingston, John O'Leary (Eds), The "Times" Good University Guide 2005, The Times: London, 2004

One of my publishers tells me that it is now necessary to bring out a revised edition of a mainstream marketing textbook every three years. If one considers the incidence of significant changes in theory and practice then , clearly, this cannot be the cause and one is left to conclude that the only compelling reason is “recency”.  The book is less than three years old, so it must be up-to-date!  This phenomenon is particularly marked in academic journals, accompanied by a tendency to recite anything and everything that might be remotely connected to the subject of inquiry to the point that soon the number of pages given to the references will exceed those devoted to the text itself.  Such a proclivity is akin to the manner  in which self-made men of limited education establish extensive libraries in which the primary criteria are quantity - buying books by the yard - and appearance - fine bindings and co-ordinated colours.  “Fine feathers make a fine  birds”.

In English Proverbs Explained (Pan Books, 1967) Ronald Widout and Clifford Whitting observed that: “the chaffinch is much more colourful and attractive than a house sparrow, yet they are of the same family; they are both finches, and without their feathers they would be identical in appearance”.  The message is that while appearances may influence our first impression they should not encourage us to draw erroneous conclusions, e.g that chaffinches are radically different from house sparrows. Or, to quote another proverb, “Beauty is but  skin deep” - a pleasing appearance may hide an ugly nature, or complete ignorance of the content of the library.

So is this polemic (“a verbal attack on a belief or opinion”) prompted by advanced years, a tendency to live in the past, and ignorance of current thinking and ideas?  Possibly, but I hope not.  As an editor, and member of numerous editorial boards, I am invited to read too many modern articles to be much out of date.  My concern is that what I have termed “research myopia” leads younger scholars to overlook or ignore all the seminal contributions of earlier generations on which the foundations of modern marketing thought are built.  In doing so they lose sight of the knowledge and insight accumulated over many years in other social sciences like economics, psychology and sociology which the marketing discipline seeks to integrate into a holistic explanation and interpretation of human consumption and exchange behaviour.

When Newton said “If I can see further it is because I am standing on the shoulders of giants” he was acknowledging his debt to all the scholars who had preceded him.  But  Newton,1642-1727 (not very recent) was propounding  completely new theories that represented a major breakthrough in our understanding.  In addition to discovering the binomial theorem, differential and integrated calculus and that white light is composed of many colours, he also developed the three standard laws of motion and the universal law of gravitation.  Unless and until these propositions/theories are falsiified (the central tenet of positivism) physicists and others will continue to accept their validity and build upon the insight they provide.  Thus science proceeds by a process of accumulation, most of which is incremental, until the next major discontinuity or breakthrough occurs, initiating a new cycle of evolutionary refinement.

But as  Luecke (1993) pointed out in his book, Scuttle your ships before advancing, social scientists seem to be unwilling (unable) to agree a common start point representing what we believe we know so that we may advance from a well-established base to explore what we don’t know.  As a result the social sciences appeared to be doomed, like the Marie Celeste, to drift aimlessly, continually revisiting the same issues and reinventing themselves.

What is needed is a better appreciation of the past and the origins of our discipline.  While it may be asking too much to agree laws and axioms of the kind that underpin the physical sciences, it should not be beyond our ability to develop what in my time at Harvard Business School in the late Sixties we called CUGs - currently useful generalisations.  Generalisation is usually seen as the third step in the creation of knowledge following observation and classification.  While it may lack the precision and authority of an axiom, law or principle it is usually sufficient to be used as a basis for decision-making and action.  All the more so if it is current and useful.  By the same token, if it ceases to be useful, or is overtaken by an improved and more up-to-date version, rejecting a CUG  does not call for the same burden of proof as does the falsification of a law.

Accordingly, we should not automatically discard an argument or piece of research because its foundations are 10, 20, 30 years old or more.  What we need to establish is whether more recent work has improved or  invalidated the original authority and, if so, how.  Further, unless we can come up with a convincing rejection of the earlier work, then we should restate its value and importance if for no other reason than that the original articulation of an insight or idea is often simpler and clearer  than later versions of it.  (If you look at papers in the Journal of Marketing in the 1950s and 60s you’ll find most are short and contained few if any references!) Also, incrementalism, without reference back to the original, can result, over time, in a radical distortion and misrepresentation of what the original authority actually said.  If you really want to know what Wendell Smith said about differentiation and market segmentation as alternative strategies you would be better advised to read his article in the Journal of Marketing, July 1956 (which contains no references at all) than Blomquist’s (2004) version based on Smith (2003) and Jones (2002) not to mention Uncle Tom Cobleigh (2001).

Being aware of the past does not require one to live in it.  But as Santayana observed “He who is unaware of history is bound to repeat it”.  To avoid this, revisit the older literature of marketing - it only dates back a century or so.  Equally, important, review the state-of-the-art, i.e. as represented by authoritative introductory textbooks in the cognate disciplines synthesised by marketing.  You might be very surprised by how much you will learn.

Michael J Baker                                                                    June 2004