How to Measure the Effectiveness of Advertising?

This article guides you about how to measure the effectiveness of advertising.

Following tests (fig. 22.1) are desired to test the effectiveness of advertising. Advertising effectiveness can be tested before the advertisement run (called as pretesting) and after the advertisement run (called as post testing).

Most measurement of advertising effectiveness is of an applied nature dealing with specific ads and campaigns. Most of the money is being spend by agencies on pretesting advertise­ments and much less on post evaluating their effects.

Many companies develop an advertising campaign, put it into the national market and then evaluate its effectiveness. It is better to limit the campaign to one or a few cities first and evaluate its impact before rolling a campaign throughout the country with a very large budget.

Pretesting:

Pretesting is used more commonly than post testing, because it is much more important to measure advertising effectiveness before an advertisement has been run than afterward. Therefore, pretesting should be viewed as a means of saving money, whereas post testing is often devoted to finding out what went wrong.

Many of the techniques used in pretesting may also be used in post testing, and vice versa. Two of the most common pretesting techniques are consumer jury tests and laboratory tests.

Pretesting should be conducted in the following situations:

1. To introduce a product or a brand.

2. When there are uncertainties or contradictory views expressed about the content of an advertisement.

3. To substitute an ongoing campaign with a new campaign.

Pretesting of advertisement take the following route:

(a) Concept testing

(b) Theme testing

(c) Copy research.

(a) Concept Testing:

Concept testing is a major feature of creative strategy, which has a bearing on ultimate effectiveness of advertising, is the basic communication concept around which a campaign may be developed. Concept testing would usually involve not more than 50 to 100 respondents using techniques such as qualitative interviews, free association tests and various statements comparison tests.

(a) Qualitative Interviews:

Quantitative interviews are of an informal nature and may be conducted individually, or in groups from amongst people drawn from a cross section of age, occupations and income levels.

(b) Free Association Tests:

Free association tests are used to pick to secondary associations to names or key words. These tests are conducting by having respon­dents mention the first thing they think of when a given name is mentioned.

 

(c)Statement Comparison Tests:

Statement comparison tests are used while testing various concept on concepts on small groups. These tests can be classified as (Fig. 22.2)

Rank Order Method:

Here the respondents ranking the different concepts or themes indicating their preference or desirability in relation to the product.

(ii) Paired Comparison Tests:

In this test the preference of respondents for either of two concepts is determined using a series of pairs of statements associated with product properties and characteristics.

(b) Theme Testing:

No specific guidelines are available for classifying themes for the purpose of analysis and research.

In general, the themes can be classified as under:

(i) Utilitarian

(ii) Focused

(iii) Informative

(iv) Non Specific

(v) Achievement Orientation

(vi) Descriptive & projective

(vii) New product, service scheme or idea.

(c) Copy Research:

The basic purpose of copy research is to establish whether the message content and presentation are likely to perform their allocated task efficiently and what changes and improvements may be helpful.

Copy research involves the following tests:

(i) Consumer jury test

(ii) Direct mail test

(iii) Portfolio test

(iv) Psychological scoring

(v) Mechanical or laboratory test.

(i) Consumer Jury Test:

Consumer jury tests are useful method for evaluating an advertisement’s effectiveness before it is run. A “Jury” composed of hypothetical customers is asked to evaluate an advertisement. The number of prospective customers on the jury may range from a dozen to several hundred.

The test may take a number of forms. Jurors may evaluate one advertisement or many. They may view advertisements individually or correctively and the evaluation process may take place at home in controlled conditions.

Both television commercials and print advertisements may be evaluated by the use of consumer jury testing method. In either case, the normal method is to have jurors look at several advertisements in order to determine which receives the best rating.

The two main procedures for evaluation are:

1. Order of merit

2. Paired comparisons

Order of merit:

Involves having each respondent rank several advertisements in order of preference. It is usually unadvisable to ask respondents to rank more than six advertisements at once. Since the average consumer can rarely work with more than six with any degree of accuracy.

The Paired comparison method:

Is especially advantageous when the researcher wishes to have jurors evaluate more than six advertisements. In this method, respondents compare two advertisements at a time, so that only two are being considered at any one time.

Each advertisement is compared with every other advertisement in the group. The winner in each comparison is noted on a score card and at the end, the number of times each ad “won” is totaled.

A major advantage of the paired comparison method over the order of merit method is that it enables the researcher to obtain advertisement ratings of greater consistency and accuracy. It also allows the evaluation of a larger number of advertisements.

However this advantage can disappear if the number of advertisements evaluated becomes too large. The number of advertisements being evaluated should not exceed ten, otherwise, the procedure becomes too tedious.

Total number of comparisons = n (n-1)/2

Here n equals the number of advertisements being evaluated. For example, in case where respondents were asked to rank eight advertisements by paired comparison method, a total of twenty eight comparisons would be involved.

n (n -1)/2 = 8(8-1)/2= 8(7)/2 = 56/2= 28

Evaluation of Consumer Jury Tests:

Though jury tests are useful and have wide application, here are certain fundamental difficulties inherent in their usage, which include:

1. Sample Selection Problems:

Here the difficulty lies in getting a that accurately reflects the market.

2. Measurement Criteria Problems:

On what basis is the advertisement being evaluated? What is the definition of “effective”? Does “effective” mean respondents like the ad best, find it most believable, think it as most interesting One?

3. Respondent Control Problems:

Unless respondents evaluate advertisements under controlled conditions, there is no guarantee that evaluation process is being done properly. In group evaluations the respondents may influence others.

4. Artificial Nature of the Test:

The most serious weakness of the consumer jury test is that it tends to be unrealistic. Since it is only a test, the conditions are not what they would be in the actual market place, and jurors’ answers are only hypothetical.

Despite these weaknesses, the jury test remains a popular method for pretesting advertise­ments. This method allows the researcher to get the evaluation in a relatively brief period of time than if the advertisement has been run through the mass media.

(ii) Direct Mail Test:

Also called as Post card test. In this test different copy appeals are condensed and printed on post cards. Each card makes some sort of free offer, the same on all cards. The post cards are sent to a large and representative sample of consumers of the product involved.

The per cent of people who write in to take advantage of the offer is regarded as an indication of the effectiveness of the appeal. Instead of a post card, a letter, booklet, or other promotional literature may be mailed to prospects.

The principle advantage of this test is that it measures action rather than opinion. The respondents take a positive action, presumably as a result of advertisement. Also it minimizes variables, because everyone receives the message under much the same conditions. Obviously, it is relatively inexpensive.

One limitation is that this method can never test more than general appeal. It does not necessarily follow that the most effective appeal in mail promotion will also be best for the mass media. It would probably not be valid test of an advertisement designed to build images rather than to stimulate direct action.

(iii) Portfolio Test:

A portfolio of different advertisements is prepared and is given to selected group of individuals who are asked to go through this portfolio and can take as much time as they wish. The portfolios are then taken back and respondents are asked to recall the advertisements. Some may also by aided in recalling. The opinion of the respondents are taken and conclusions drawn as to the extent that a message stands out or is understood.

While portfolio tests offer the opportunity to compare alternative ads, directly, a number of weaknesses limit their applicability.

i. Recall may not be the best test. Some researchers argue that for certain types of products—those of low involvement—ability to recognise the ad when shown may be a better measure than recall.

ii. Factors other than advertising creativity and presentation may affect recall. Interest in the product or product category, the fact that respondents know they are participating in a test or interviewer instructions (among others) may account for more differences than the ad itself.

One way to determine the validity of the portfolio method is to correlate its results with readership scores once the product is placed in the field. Whether such validity tests are being conducted or not is not readily known, although the portfolio method continues to remain popular in the industry.

(iv) Psychological Scoring:

Under this semantic differential is a rating scale that has been used extinsely in advertising measurement. Under this method, respondents are asked to mark on a seven point bipolar scale, their “feelings” toward a particular advertisement. In this way, various ad treatments can be measured in terms of consumer reaction. Fig. 22.3 shows the profile of three alternative advertisements.

Some advertising specialists feel that scales serve more as preliminary checklists for creative people than as research tools. Actually almost all copy writers devise some sort of checklist of the qualities to include in every advertisement and those to avoid.

Most checklists are based on some research evidence. For example, a checklist devised by Richard Manville is based primarily on the ability of certain ads to produce inquiries.

The Thompsonluce method is based on the correlation between readership and certain elements. A much publicized checklist of judging “advertising effectiveness” was devised by the Town send brothers in 1938. For several years it was much used by advertisers and agencies. Today’s more sophisticated advertising people are not inclined to accept anything so rigid.

A variety of so called motivational research techniques are used in pretesting advertisements, such as depth interviewing, the Rorschach (inkblot) test, Thematic Apperception Test (T.A.T.), word Association, and Role Playing. These methods seek to ascertain whether a particular advertisement or commercial conveys the intended meaning to consumers or not.

(v) Mechanical or Laboratory Tests:

These are designed to measure the psychological reaction of respondents to an advertisements. These tests are conducted in laboratory by using various mechanical and electronic equipment. These are primarily designed to test the attention getting power of the advertisement and do not measure other attributes.

The principal measuring devices used in such testing include:

1. Eye camera or Oculometer

2. Psycho-galvanometer (GRS Test)

3. Perceptoscope or pupilometer

4. Tachistoscope

5. Electro encephalography measures (EEG) Brain waves measures.

Eye Camera or Oculometer:

As it was first developed about in 1890, but it was not used in advertising research until 1938, when results of its use were reported by Look magazine. In an eye camera test, the respondent views an advertisement while the camera records how long and on what area of the advertisement the reader’s attention is focused.

Though the eye camera has two advantage of accurately measuring what part of the advertisements the respondent’s eye focused on. It does not measure interest in the advertisement. It is an instrument of questionable usefulness, despite its objectives.

Perception Research Services, a subsidiary of the Inter Public Group, uses eye movement cameras to measure respondent interest to each part of an advertisement. Interviewers then question respondents further about both the advertisement and the product. The firm has found his research most useful when creative work is still in process.

Psychogalvanometer:

(GSR Test) is another mechanical device used for pretesting advertise­ments under laboratory conditions. This instrument is similar to a lie detector, has been used by psychologists to measure respondent reactions and emotions. Emotional changes in respondents are recorded by measuring, through electronic impulses.

Perspiration changes in the palm of the hand. Presumably, an increase in perspiration indicates a reaction to a given advertisement. However, left unanswered is the question whether a favourable or an unfavourable reaction is being recorded. The measurement of galvaino skin responses is an extremely questionable technique in advertising research. It is useful mainly for advertise­ments of sensitive nature.

Perceptoscope or Pupilometer:

A third laboratory measuring device is also intended to measure respondent arousal or interest in a particular advertisement. This instrument records changes in the pupil size of the respondent’s eye. The principle involved is that pupil delation is indicative of interest and pupil contraction indicative of lack of interest in what is being viewed.

Though the percept scope is an instrument of great potential for measuring respondent’s interest in advertisements, it does not directly measure advertising effectiveness. These laboratory testing devices remain instruments of limited utility to the advertising researcher, however, with proper use by qualified experts, such devices can play a worth-while role as aids in the pretesting process.

Tachistoscope:

One or several advertisements are flashed on the screen for a fraction of second each and the subjects are asked to comment on what they have seen. This text measures the extent of perception of the message on the part of the receiver.

Electro Encephalography Measures (EEG) Brain Waves Measures:

EEG measures can be taken from the skill to determine electrical frequencies in the brain. These electrical impulses are used in two areas of research: alpha waves and Hemispheric laterlization.

Alpha activity:

It refers to the degree of brain activation. People are in an alpha state when they are inactive, resting or sleeping. The theory is that a person in an alpha state is less likely to be processing information (recall correlates negatively with alpha levels) and that attention and processing require moving from this state.

By measuring a subject’s alpha level while viewing a commercial, researchers can assess the degree to which attention and processing are likely to occur.