The Measurement Standard
Jim Macnamara’s ‘Measuring Up’
Transparency, not ‘Black Box’ Measurement Needed
At the Summit on Measurement this year in Portsmouth, New Hampshire, a gathering of some of the best and brightest working in the field of measuring PR and corporate communication that is really the world’s leading event of its type, discussion ventured on to ‘black box’ measurement – a fascination in our industry to come up with mysterious indexes and metrics that purport to be the Holy Grail of measurement.
Some argue that any day one of these will be the saviour of public relations – a silver bullet that shows the ROI (Return on Investment) of PR and sends management into spasms of appreciative joy as they reach for their corporate check books to up their PR budget.
As a line from an iconic Australian movie, The Castle, said: “Tell ’em they’re dreaming”.
A look over our shoulder, some peripheral vision as they say, will show that in advertising and marketing, research has forged an important role through exactly the opposite. Rather than creating mysterious near undecipherable metrics, mainstream research has relied on open transparent standards in conducting surveys, focus groups, interviews and other methodologies.
Survey researchers use tried and tested question techniques such as Likert scales and readily disclose, and are required to disclose under the Codes of research societies, their sample sizes, scales, weightings if any, and error rates. The same goes in scientific experimental research where strict documentation of procedures is required.
Does this openness lead to sameness in research and lack of differentiation? No. Open standards and conformity to standards creates confidence among clients and ability to understand the information they are presented with. Research companies compete by offering value-add in terms of interpreting data and drawing conclusions in relation to the client’s objectives, rather than throwing up raw numbers.
Meanwhile, much of the public relations measurement confraternity seems obsessed with finding some magic formula – a Media Prominence Index, a Reputation Index based on a fixed formula, an ROI score or some other single metric that pronounces a campaign or project a success and its creator a hero.
The search for the Holy Grail in measurement is a blind alley that is confusing clients, fragmenting the industry and eroding the already sub-optimal level of confidence in public relations.
The optimum path for public relations measurement is for bodies such as the Association for the Measurement and Evaluation of Communication (AMEC) and the Institute for Public Relations in the US to set and actively promote basic standards for communication research. Standards do not necessarily restrict competitiveness. For instance, standards should require, at a minimum:
· Open transparent disclosure of methodology including scales and calculations used;
· Publication of sample sizes and error rates in research.
In addition, standards setters should ideally have the courage to require:
· Avoidance of generalised invalid methods such as Advertising Value Equivalents for editorial publicity when it is known to be variable and potentially negative;
· Focus on measuring impact and effects (outcomes) rather than simply counting output data such as audience reach, opportunities to see and volume of messages.
The computer, DVD/video and automotive industries have shown how important standards are. Until Microsoft and IBM and other computer companies got together and agreed on basic standards for Operating Systems, communication protocols and connectivity standards, computing was a ‘black art’ beyond the understanding of most consumers practised by men in white coats (yes, there were no women in computing then). Standards did not prevent computer companies competing – in fact, they provided a platform and an environment for growth and competition.
Imagine the automotive industry if there was no standards in terms of how we indicate when we wish to turn, whether the ignition key is turned clockwise or anti-clockwise, where the gear shift is located, and so on. Without standards, consumers are confused and markets do not grow.
The video industry provides a good example of the havoc that lack of standards creates. The presence of PAL, NTSC and SECAM standards, and the options of three-quarter inch, half-inch and quarter inch tape, contributed to its downfall and replacement by a uniform standard of digital video disks. With CD and DVD technologies conforming to basic standards worldwide, the music and video industry have enjoyed growth and improved efficiencies.
Meanwhile Nora Jones, Jack Johnson and Andrea Bocelli can each produce their own unique products within these standards and consumers flock to buy them in confidence.
It’s time for the PR industry to sing its song – but in key, conforming to the rules of music and not some new invented scale that leaves our audiences scratching their heads and wondering what it all means.
*****
* Dr Jim Macnamara BA, MA, PhD is Group General Manager – Research with Media Monitors – CARMA Asia Pacific following the acquisition of the CARMA Asia Pacific franchise by Australia’s largest media monitoring firm. He has a 30-year career in the media and PR is Asia Pacific and is the author of 11 books on media and communication.
1