The Dorr Research Corporation Philosophy on Marketing
Research Methodology - Excerpts from a Communications Workshop Conducted for the
Vermont Advertising Club
by Richard S. Dorr
September 21, 1994
I love focused groups!
They're fun. They're useful. And, as a moderator, I get to meet a lot of nice people.
Clients and their advisors such as ad agencies can be stimulated by seeing and hearing people talk about their products.
Focused groups are a great way to identify people's problems or needs -- to brainstorm for ideas -- to sift through some preliminary concepts before selecting finalists to test in-depth -- to determine what to ask about and how to ask it in a more quantitative survey.
Yes, I love focused groups.
But, there are many times when focused groups are misused. What I'd like to discuss today are the appropriate applications of focused groups, how to avoid what I call "focused group myopia -- the indiscriminate use of focused groups for virtually every research need; qualitative alternatives to focused groups; and some practical tips in the use of focused groups and these alternatives.
In today's world of business research, focused groups enjoy special status. They are the one research method that best combines utility with entertainment. Many clients don't call with a request like, "How can we best determine why our sales have declined?" Their first words are more likely to be, "We need to conduct some groups."
They're so conditioned to the method that they don't even take the time to mention what kind of groups they want. In several cases, I've been tempted to suggest prayer groups.
This is the myopia to which I refer -- the almost blind devotion to a research form that has nearly reached cult proportions. These misguided managers -- and they are the exception rather than the rule -- may be saying, "We need information." Too often, what they really mean is, "Oh boy, I'm about to be entertained big time. I'll get away from the office, travel to some swell city, sit behind a one-way mirror, stuff myself with M&Ms, watch a dog and pony show, stay over a weekend allegedly to save my company money on the air fare but really to explore the best restaurants, and perhaps pick up a few marketing insights along the way."
One can hardly fault them for having this unfocused perspective on focused groups. Since the inception of the focused group technique by a fellow named Ernst Dichtor in the 1940s, the research community has cultivated the notion that there is somehow a mysterious and magic property to focused groups -- something the client shouldn't try to understand but should simply experience.
It was this claim of mystique that greeted me in the mid-sixties, when I had my first focused group experience. I was a fledgling researcher at an advertising agency in Boston. In those early days of research, "fledgling" was good enough to be the Research Director.
My background was in statistics and quantitative research. A lawn care client had a need for a probing technique to examine motivations, and I had heard of these things called focused groups.
I searched around until I found a guy -- I'll call him "Doc" -- who was a self-proclaimed expert on focused groups. I should have suspected something when I got to his office and found three signs on the door: psychologist, focused groups, and seances. But I was young and gullible -- and I loved M&Ms.
"Doc" explained that moderating focused groups was something akin to a mystical art form. He actually said these words: "We weave our magic, and the ideas multiply like rabbits jumping out of a hat."
He even explained his prized technique, which he called "The Grand Silence." "Whenever there's a lull in the discussion," he said, "I simply remain silent. You just watch. The dynamics of the group will take over. One of those people will crack and begin to spill his guts, because he'll sense that I expect more than the group has given. They'll actually be exerting pressure on themselves."
Wild horses couldn't have kept me away from that session. Eight owners of homes afflicted with crabgrass were "warmed up" by "Doc." Then, he started through the Discussion Guide, and it was readily apparent that he had a reluctant group on his hands. In fifteen minutes, he raised at least twenty topics, with most of the responses being either "Yes" or "No." Perspiration formed on his brow. Obviously in desperate straits, he launched into ... "The Grand Silence."
He tipped his chair back. His eyeballs rolled up into his head. I began counting the seconds, as I looked to and fro among the panelists. Who would crack?
Ten seconds went by....twenty....forty. Do you realize how long forty seconds of silence is? One woman shifted uneasily in her chair, and the creaking noise sounded deafening. I knew she would be the one.
More time passed as she summoned her courage. She took a deep breath. I eased forward in my chair, stealing a look at "Doc," whose eyeballs were still glued to the insides of his eyelids.
Finally, after a full 90 seconds of "The Grand Silence," she took the verbal plunge, and I can still hear the dynamic words ..... "Can I go to the bathroom?"
My point is not to denigrate focused groups. Basically, I'm here to praise them, not to bury what's been my bread and butter on more than 1,000 occasions. There are times when silence is an appropriate stimulant, and the dynamics of focused groups can prove invaluable.
I'd just like to interject a more balanced sense of reality when we think about research methodology. My main message is that neither focused groups nor any other marketing research technique is some sort of mystical "black box" that should be adopted as a panacea in all marketing research.
While wanting to keep this discussion from degenerating into a dry recitation of research theory, I must first establish what I feel is an important frame of reference in selecting the appropriate research method for any study. We feel there are three basic functions for which marketing research is most widely used:
• The first is exploration-for ideas, issues, problems, needs, motivations, etc. An example would be a recent study in which we were called upon to identify and investigate the priorities people exercise when they decide where to spend their vacations. In two other studies conducted concurrently, we probed for the factors accompanying two crazes that have been sweeping the country-premium coffee and microbrewed beer. Don't think it was easy to keep from confusing these two product categories, because there are many similarities. In both cases, new trends sweep from West to East, and flavored varieties are of increasing competitive importance. Was it the full, rich flavor of the coffee or the spicy bite of the beer that turned people on -- or was it the other way around?
• Another research application involves refinement -- enhancing a concept or idea to maximize its impact. An example of this function would be the many situations in which we are called upon to take advertising prototypes and subject diagnostic evaluation that will help our clients to fine tune them. We recently had occasion to do just this with conceptual ads for a race track and television story boards for a health care organization -- in other words, two studies involving two different forms of gambling. In another case, one of the major long-distance phone companies asked us to winnow some finalists from a list of promotional inducements they were considering as offers to small businesses.
• The third research function involves evaluation -- either testing reactions to something or taking a snapshot of some market situation. For example, a recent client had an idea for a new consumer financial service, and we conducted research to determine what proportion of the households in a given area might like the idea sufficiently to roll it out to the marketplace. Another form of evaluation involved taking a demographic and attitudinal "snapshot" of Massachusetts residents for an animal humane society. Sometimes evaluation can involve tracking over time, as in our continuing monitoring of the impact of the Massachusetts DPH anti smoking advertising initiative.
So we have these three prevailing research functions: exploration, refinement, and evaluation. If you think of them as a continuum, with exploration at one end, evaluation at the other end, and refinement somewhere in the middle, it becomes a fairly straightforward matter to identify the appropriate method for any given research objective. The rule is simple:
The closer the objective is to exploration, the more qualitative the research should be.
The closer the objective is to evaluation, the more quantitative the research should be.
In short, qualitative research is best equipped for dealing with issues of how and why. Qualitative research uses probing techniques to encourage respondent introspection and creativity. The credibility of a piece of qualitative research is usually evaluated within the context of validity-basically an assessment of how well respondents understand the issues we raise and how realistically they respond.
Quantitative research performs best in answering questions on who, what, when, where and how many. Its cornerstones are twofold: the standardization of a scripted questionnaire format and the sheer number of respondents. The credibility of quantitative research findings is usually evaluated within the context of reliability -- the margin of error that could exist between the research results and and corresponding results of the same study hypothetically replicated many times. Telephone surveys, mail questionnaires, and mall intercept studies are examples of quantitative research techniques.
Enough for the theory part. We're here to discuss focused groups. However, I'd prefer to broaden the topic a little and discuss qualitative research -- the research discipline of which focused groups are just one element.
For anyone who does not know, focused groups are open-ended discussions among small groups of participants who have been recruited to meet certain qualifications. They may peel potatoes left-handed, may be decision-makers for the pension programs of their companies, may be frequent travelers to Tibet, or I suppose they could be all of the above. Focused group sessions typically last an hour-and-a-half, are usually held in front of one-way mirrors so they may be observed by the people who are paying for the research and will benefit from the information that is derived.
For all practical purposes, there is one other form of qualitative research. I refer to the open-ended, one-on-one discussions that social scientists long ago termed "depth interviewing." The marketing research industry has adapted this approach to a much shorter interview format -- usually 30 to 45 minutes apiece. We at Dorr Research call it "intensive interviewing."
A focused group usually includes eight to ten participants, and a typical focused group study might involve anywhere from two to as many as 10 or 12 groups, with four probably being the average. As a rule of thumb, we have found that an intensive interviewing study should include about two-thirds as many respondents as the equivalent focused group study. If a given research venture would involve three focused groups (or 24 to 30 participants) , the corresponding number of intensive interviews should be between 15 and 20.
The biggest single difference between focused groups and one-on-one intensive interviewing is that heart and soul of focused groups is conversational dynamics among peers. Intensive interviewing implicitly prohibits peer interaction.
The main reason why focused groups are ideal for exploration is that people's comments tend to stimulate the responsiveness and creativity of one another.. The conversational "whole" becomes more than the sum of its parts.
As whatever we're researching gets farther away from the idea stage and closer to the possibility of market introduction, it increasingly behooves us to reflect market conditions as realistically as possible in our research. "Reality" does not normally include an advisory committee of peers when a consumer reaches for a box of soap powder in the supermarket.
Consequently, the very dynamics that represent a catalyst for focused groups can become an impediment when the research objectives involve refinement or evaluation. These are usually occasions when we do not want research respondents to influence on another, because that's not what they usually do in the real world. There are reasons why, when we took tests in school, influencing one another was called "cheating." With today's political correctness, it's probably called "creative adaptation." In the research industry, we call it "bias."
Given these caveats, the best application for focused groups occurs when the objective is exploration. Encouraging group participants to play "Can You Top This?" is an excellent way to generate ideas, detect problems, and simply to stimulate interesting conversation about a product, brand, or concept.
Getting into the area of refinement with focused groups can be tricky, because the basic nature of focused groups is to tend toward consensus. When refining something, the researcher must be able to obtain reactions to concepts as well as exploring for enhancements and then testing reactions to the retooled concepts. The number of independent interviews in an intensive interviewing study allows the researcher multiple forums for going through this iterative process of testing, further exploration, and additional testing.
Intensive Interviewing even performs well at the evaluative stage of research because of what is known as the "sequential analysis" process. As the results of each interview are overlaid on the cumulative results of all the preceding interviews-notwithstanding the effects of refinements along the way-certain trends become obvious. By the time the last interview is completed, a strong sense of redundancy has invariably invaded the accumulated findings of all these independent interviews. It is this "weight of evidence" that indicates a comforting sense of credibility in the conclusions. Ironically, the theory of sequential analysis was first associated with quantitaitve research techniques-specifically techniques associated with quality control procedures. Certain "acceptance" standards for products were preset, and random samples of the products were inspected until the cumulative results met or failed those standards within an acceptable level of tolerance. Sampling could then be discontinued after only a fraction of the inspection time typically associated with quality control. The same theory works with intensive interviewing.
I don't intend to suggest that intensive interviewing should replace all quantitative research at the evaluative stage. However, when the need is for evaluation of concepts, but the concepts being evaluated can and should be allowed to change over the course of the research, intensive interviewing is a lot more flexible and adaptable than scripted quantitative research. The sequential redundancy of the findings should never be confused with statistical reliability; but it suggests a stability in the results that gives clients a very strong comfort level in using the information.
I should mention one logistical difference between focused groups and intensive interviews. Intensive interviewing can be more tedious than groups. Twenty intensive interviews consumes a lot more time than four focused groups and can add a day or two to the schedule. Observers seem to consume much more coffee as intensive interviews wear on than they do during focused groups.
On balance, however, we believe intensive interviewing is the only research approach that is an arguable option in all three research functions: exploration, refinement and evaluation.
So, in a nutshell, my message to you today is this:
• If you want a great research exploration, idea generation or problem detection, conduct some great focused groups.
• If your research objectives require probing qualitative research with a heavy component of refinement or diagnostic evaluation, consider intensive interviewing.
• If your research calls for evaluation without an element of exploration or refinement, quantitative research is probably the method for you.
Thank you for your attention.