
Using Research to Evaluate Your School, Community College, College or
University
A Report Card for Higher Education: Measuring Target Audiences' Perception
of Educational Quality, Institutional Image, and Service Components
by Bob Topor
Topor Consulting Group International
1998
Magnitude estimation is a market research technique that provides a means
to evaluate judgments of key target audiences to determine perceived
educational quality and institutional organizational image. Comparisons
between institutions, institutional offerings, services and "products" can
be reported on a linear, ratio-determined proportional scale that can be
directly compared and validated. One can include the important marketing
idea of "value-added" perceptions in this assessment.
What is a Magnitude Scale and how can it work to help you?
A ratio scale of psychological and perceptual stimuli can be used to assign
numbers (values) to institutional offerings, curricular choices, majors,
alumni participation, fund raising issues and related perceptions. The
Magnitude Scale reports numbers derived from ratios defined by criteria of
services being evaluated. These studies can be used to determine valid
interpretation of consumers' judgments about a variety of perceptions,
ideas and issues. They can be used to measure broad subjects (institutional
image) as well as precise issues (such as user perceptions of a specific
course offering).
Using relatively small samples, this type of quantitative research can be
used to augment qualitative research (focus groups, for example). See "How
to Run a Focus Group for Higher Education," Marketing Higher Education
newsletter, Volume II, Number 6, June, 1988. (See Topor's book on this
subject at his web site). The combination of qualitative research (focus
groups) and quantitative research (magnitude estimation) can provide
powerful information to achieve many marketing objectives.
Some of the problems of quality assessment
Jacoby and Olsen in 1984, reporting in "Perceived Quality: How Consumers
View Stores and Merchandise," Lexington Books, Lexington, Massachusetts
describe critical shortcomings in the assessment of potential user
perceptions of quality and the lack of an appropriate standardized
measurement tool and reporting standard. Others have reported that research
findings cannot be generalized easily because of a lack of consistency
across studies and varied methodologies and research designs used to
identify attributes and perceived benefits comprising issues of quality.
This is particularly difficult for intangible products, i.e. education (as
compared to hard products). Higher education comprises, for the most part,
intangible products-difficult to define, measure, evaluate and report.
The concept of perceived values and service quality
Ideally, a set of judgments based on potential user perceptions and
responses about criteria can collectively establish "quality."
Traditionally services have been defined on the basis of a multidimensional
list of attributes for each category of an institution of higher education.
This application of perceived attributes can be applied to a specific issue
or idea to provide information contributing to a "score" to represent an
eventual total. This total is the collective sum of all the attributes used
to obtain a perceived quality total score. Some argue that this method of
market research distorts the eventual researched "picture because it
depends on a predetermined list of service attributes that are used to
define 'quality.'" The list may be inaccurate, incomplete, or prejudiced
producing distorted results. Researchers have often found little
consistency in the evaluation process and in the assessment of attributes
and their importance to eventual quality reporting.
Every effort to identify, quantify and understand perceived quality of
identified attributes is important. These data can provide valuable
information that can be used to augment, change, or establish perceptional
ideas amongst key target audiences. Alternatives of service offerings can
be determined. Real and useable information can be applied in public
relations, admissions, fund raising and alumni activities. These data can
be used to improve weak areas.
Measuring the consumer's "invisible report card"
Each person carries around in his or her head a "report card" that forms
the grading system by which an organization and its offerings are measured.
Image is derived from the cumulative assessment of these criteria. The
problem is that report cards differ in evaluative criteria. Sometimes quite
dramatically! Of course some evaluative criteria overlap from person to
person and from target audience member to target audience member. But all
criteria are not the same. More importantly, criteria used by external
audiences to evaluate your institution and its service (as well as its
image) may differ dramatically from those you use-or those used by your
academic colleagues.
* * * * *
A paradigm for image research and evaluation
The following list is designed to assist you as you establish a reference
list of potential criteria for market research intended to evaluate
institutional image.
Quality
Quality is not an issue that is monothematic. Quality is an aggregate of
many, many factors. No one idea most often defines quality in the minds of
target audiences although there may be one or two factors that are "top of
the mind" or prevalent as a member of your audience evaluates, stores,
re-evaluates and reports perceived quality.
Cost
Issues of cost prevail in higher education. Rightfully so, since public
perceptions and potential user perception of costs for your services and
offerings are a topical issue. As well, you should consider the idea of
"cost" in much broader ways than simple monetary dollars and cents for
tuition, housing and fees. For example, what is the "cost" for a person to
attend an event, even if it is "free?" Time, commitment and awareness may
be considered "costs" as you apply this cost evaluation idea to a wide
variety of educational activities including fund raising, community
relations and alumni work.
Peer Influences
The relationship of target market audience members with peers (friends,
acquaintances, counselors, family, etc.) contributes toward marketing
decisions. You should consider ways to influence these seemingly
"secondary" (indirect) audiences as ways to reach primary targets.
Communications quality
Feelings evoked, as well as the quality of information, contained in
outreach materials (publications, web sites, brochures, announcements,
letters, videotapes, speeches, etc.) serve to add positive information to
target audiences' perceptions or to add negative information that detracts
from your positive marketplace image. Pay close attention to individual as
well as cumulative effects produced by your outreach materials.
Communications continuity
Identified primary and secondary target audiences should receive regular
information designed to build perceptions, add to positive perceived
attributes (as identified by image market research), delivered with
consistent identification. Communications are the "engine" that drive
successful marketing. Keep your engine running! Don't make the mistake of
running "hot" and "cold" with outreach materials. That is, don't send off a
barrage of materials then leave receivers in the "dark" for months and
months (or years)! In marketing, as in other forms of personal
relationships, familiarity can result in strengthened relationships.
Isolation often results in abandonment.
Climate
Unlike effects produced by meteorological activities, the "climate" on your
campus (there often are many climates since each target segment may have
distinct perceptions and climatic interpretations) contributes to or
detracts from your audience's perceptions. Climates may vary based on any
number of factors: media reports and public relations being ways to
influence (positively or negatively) target audiences perceptions (and
resultant images) of your academy and its activities.
Associations
Often overlooked because of its persistent presence, the relationship or
association your "product" (offering) or institution has with others may
influence perceptions. Location, types of educational organization (i.e.,
"ivy league"), academic relationships (Liberal Arts school), size
(enrollment), kind of study (undergraduate, graduate, secondary school,
community college, research focus), and many other forms of often
externally attached categorization can have powerful impact on marketing
strategies you choose to develop. These associations need to be carefully
researched and understood. You may find that you have opportunities to take
advantage of these perceptual relationships or you may find that you need
to overcome some of the associations in order to achieve the marketing
results you require.
Relationship longevity
If you could imagine a scale with "length of relationship on the "x" axis
and "strength of relationship" on the "y" axis, you would find that the
longer the relationship the greater the possibility for strong association.
Note the word "possibility." Long associations do not necessarily guarantee
strong relationships. However, possibilities for stronger relationships can
be increased by continued exposure, positive experiences and fruitful
exchanges.
Accessibility
A powerful marketing tool to build user satisfaction and obtain comparative
differential advantage over your competitors. Being able to serve the user
in ways that are convenient and readily available will help in achieving a
positive image. There is, of course, the opposite of this idea (sometime
called the Gucci theory). That is the idea that people will want something
if it is difficult (or costly) rather than easy to achieve. This may be an
approach academies that have traditionally been perceived as the "best" may
elect to take. It is, however, a dangerous position especially as public
perceptions toward higher education become more jaded and problematic.
Convenience
Closely aligned to accessibility, convenience is a strong marketing idea.
How easy is it for your target market audiences to take advantage of your
services? For fund raisers: how easy is it for a donor to be recognized for
her/his contributions? In its simplest terms, how easy is it to get in
touch with a person at your school? In every situation, how well prepared
is your school to respond? Have you indeed made it convenient (as perceived
by the user)?
Performance
Excellent performance for higher education, as in race cars, is the
cumulative result of many things working well together. When one or two
things go astray, performance can be reduced-often dramatically. Well
"orchestrated" activities, events, offerings, services, and outreach
efforts will achieve high grades in performance.
Personality
Higher education is no more homogeneous than are people in it. Each
institution, like each person, has a personality and character that can be
defined through research. Comparative analysis will yield strong
information that can be used to market and build image and awareness.
Comfortability
The feelings you get when you settle in at home in your favorite easy chair
can be compared to the feelings evoked by institutions that know the value
of providing, nurturing and maintaining feelings that "make one feel at
home." These feelings can be evaluated, compared and measured.
Reliability
Trusted, credible, accredited, satisfactory, unsuspicious, and secure are
words that can be used to define reliability. Reliability is the idea that
your institution and its services can be relied on to deliver as promised.
This assumes a high level of quality in the "product" itself. The basic
functions of higher education; teaching, learning, research, and the
promulgation of knowledge need to be understood and measured using market
research.
Value
The perceived worth of a thing is its value. The degree to which a member
of your target audience perceives worth is what determines (in that
person's mind) the relative value of your institution, its people,
offerings, services, etc. Although abstract in nature, this idea of value
is very, very important since it can be usedto offset costs (tuition,
housing, fees, etc.). See "Perceived Value Added," Marketing Higher
Education newsletter, April 1991, Volume V, Number 4.
Competency (faculty, administration, staff)
Potent, capable, able, cogent, valid, effective, efficient, adequate,
predominant and strong are ideas that relate to competence. The perception
of competence on the part of your audiences is fed with many stimuli. The
cumulative "grade" placed on your school and its components can be the
result of many factors. Perception of competence will most often be the
result of many impressions and influences.
Credibility
Credibility results from the assurance that plausibility and probability
are logical outcomes of promises and expectations. Trustworthy and
believable are ideas that can be researched as perceived through the
feelings, thoughts and minds of your target audiences.
Security and confidentiality
Guardianship, stewardship, invulnerability of information and protection of
the individual are safeguard ideas that have become more and more important
to your audiences. Your institution's ability to deliver and promote ideas
of protection, guardian care, wardenship and safeguard contribute to an
image component that will become more and more important to many audiences.
Courtesy
Ideas of respect, good manners, politeness, gentility, cultivation,
culture, polish, amenity, good humor, amiability, affability and ceremony
contribute to the concept of courtesy. Individuals (and their actions)
contribute to or detract from courtesy as perceived by your target
audiences. Smart educational institutions pay careful attention to this
"detail."
Communication
Free and open exchanges of information are the "vehicles" on which positive
marketing can be built. Honest and accurate reflection of reality can be
used to build alliance and allegiance amongst your target audiences. Pay
careful attention to the idea of communications. High marks in
communications indicate opportunity for strong image building. Low marks
may indicate problems. Honesty counts.
Atmosphere
Ambiance, surrounding or prevailing conditions that are contained by your
campus, its location, activities, offerings, classrooms, facilities, and
occasions can evoke strong image perceptions and help maintain powerful
impressions. Traditional academic ceremonies (such as commencement and
baccalaureate events) can be used to build image.
Management and organizational structure
Behind the scenes organizational structure and culture are contributors to
image building since they often provide the "grease" that lubricates the
wheels of an efficient and progressive organization. A well managed and
well run organization is necessary to build optimism and a sense of well
being that can result in positive image perceptions.
Attitudes, esteem and self-perception
Self-perceptions result in a cumulative effect that is best described as
institutional self image. Just as with an individual, internal feelings
about the value and worth of the academy are very important since they fuel
many other activities and foster conclusive opinions on the part of
internal and external audiences.
Success perception
Periodic "successes" are important to the image of a place. Occasional
announcements of meritorious awards, prizes, recognitions and achievements
help to build positive image and promote feelings of accomplishments and
"goodness." Market research can be used to evaluate this phenomenon as
perceived by key target audiences.
Timely innovation
Leaders are often recognized because of visionary thinking and actions. One
of the tests of a healthy institution is this idea of innovative and timely
delivery of services, ideas and activities.
Advisement
The ability to advise, direct and consult users to assist them in their
pursuits is an attribute you should consider. Some institutions do a better
job of this than others. It may be fruitful to measure the perceptions of
your academy's advisement performance to determine if it is something you
want to consider in your marketing mix. See "A Practical Guide to
Enrollment Management in Higher Education " by Marguerite Dennis, 1998,
ISBN 0-89789-591-6.
Range of choice
It is not true that institutions of higher education offer equivalent or
similar ranges of choice. A broad range of options is an idea that you may
want to consider as you assess marketplace perceptions of your services.
Measure perceptions of your school's offerings as compared to others to
determine comparative differential advantage.
Application
More and more the idea of learning for its own sake has been vanishing.
There is greater demand for applicable learning. That is, learning that can
be applied to one's life's work, employment or contribute towards career
development. This is a measurable attribute for a school, college,
community college or university.
Governance
Sometimes overlooked because it is hidden behind closed doors of
administrative and trustee meetings, the idea of visible governance is
emerging as a criteria for the evaluation of quality. The best institutions
are led by individuals who demonstrate, through their actions and
communications, sound and visionary leadership, well tuned to the missions,
goals, values and aspirations of the academy.
Faculty and Students
Often overlooked, frequently misunderstood, probably belonging at the top
of this list of factors and criteria, the quality of faculty is probably
the single most important factor in determining an institution's strengths.
Quality faculty help an institution maintain its position of reputation and
provide powerful "fodder" for image building. Quality faculty attract
quality students. This combination is the heart and soul of an institution
of higher education.
* * * * *
Keep in mind that many of the preceding issues are "broad based" in nature.
As well, you may decide to establish qualitative evaluative criteria that
are very narrow and specific in nature. For example, you could establish
criteria to evaluate an extension or continuing education course as
perceived by target markets in a certain course of study between the ages
of 35 and 55. Issues could include perception of course usefulness, cost,
instructor performance, ease of location, applicability to real life, etc.
Your ability to establish evaluative criteria is limited only by your
imagination (and by information you derive from research). You may in fact
use research itself to determine the evaluative criteria. That is a good
way to get accurate projectionable information.

|