HomeMy WebLinkAbout2001-03-20; City Council; 16122; Citywide Paired Comparison Opinion SurveyAB# &&(,a
MTG. c.. -a 01
DEPT. CM
CITY OF CARLSBAD -AGENDA BILL c b ! TITLE-
Authorize Citywide Paired Comparison Opinion Survey to be
conducted by Cal State San Marcos Social Behavioral Research
Institute
DEPT. HD.
CITY All-Y.
CITY MGR
RECOMMENDED ACTION:
Authorize Citywide Paired Comparison Opinion Survey to be conducted by Cal State San
Marcos Social Behavioral Research institute and Allocate $20,000 from the City Council
Contingency Fund. (and adopt Resolution NO. 2001-89)
ITEM EXPLANATION:
In the summer of 2000, the City contracted Cal State San Marcos Social and Behavioral
Research Institute (SBRI) to perform the citywide public opinion survey. This survey was
conducted in the fall of 2000, with the draft results reported at the City Council goal
workshop in February 2001. At that workshop, the City Council asked SBRI to recommend
an approach to provide a deeper analyses of residents’ opinions regarding a specific set of
proposed programs / projects.
In response to the Council’s request, SBRI has proposed conducting a follow-up survey
based on a paired comparison / economic choice model. SBRl’s recommendation is to
perform a telephone survey of approximately 500 residents, first asking respondents to
state a preference given a choice between two City programs I projects. This paired choice
comparison questioning would continue through the entire set of proposed programs I
projects in order to provide rank order information. Secondly, the respondents would be
given the estimated program / project cost, and asked again to answer the same paired
choice comparisons, taking this information into account.
Using the paired comparison / economic choice methodology, we can derive information
about citizens priorities with and without cost values attached to them, and see how
priorities change when cost is taken into account. Based upon SBRl’s recommendation,
this type of methodology lends itself to ranking four (4) projects / programs. Staff proposes
using the following projects / programs in this analysis:
1. Fifty-meter Pool in south Carlsbad (with room for expansion into an aquatics
center)
2. Municipal Golf Course
3. Northwest Quadrant Community Park (a.k.a. Pine School site)
4. Citywide Trails Program
As stated above, the survey will interview 500 residents with 250 living North of Palomar
Airport Road and 250 living South of Palomar Airport Road.
The survey will take approximately 6-8 weeks to complete. The results of this survey will be
brought back to Council once the data has been collected and analyzed.
I
PAGE 2 OF AGENDA BILL NO. 1 lo! ) aa
FISCAL IMPACT:
The projected cost for this project is $20,000, to be allocated from the City Council
Contingency Fund
EXHIBITS:
1. Letter to Raymond R. Patchett, City Manager, from Allen Risely, Associate Director,
Social & Behavioral Research Institute, CSUSM dated February 19, 2001.
2. Resolution No. 2001-89
2
RESOLUTION NO.Z!ool-89
A RESOLUTION OF THE CITY COUNCIL OF
THE CITY OF CARLSBAD, CALIFORNIA,
To authorize preparation of a citywide Paired Comparison
Opinion Survey to be conducted by Cal State San Marcos
Social and Behavioral Research Institute and appropriating
funds.
WHEREAS, the City of Carlsbad wishes to be responsive to the
needs and preferences of our citizens; and
WHEREAS, the City of Carlsbad has a multi-million dollar Capital
Improvement Program designed to provide important public facilities to the
community; and
WHEREAS, the Capital Improvement Program should reflect the
priorities of the City Council and the citizens of Carlsbad; and
WHEREAS, one way to establish priorities is through the process
of surveying using a Paired Comparison model; and
WHEREAS, Cal State, San Marcos Social and Behavioral Research
Institute has the technical skills to complete a survey in a professional and useful
manner.
NOW, THEREFORE, BE IT RESOLVED by the City Council of the
City of Carlsbad, California, as follows:
1. That the above recitations are true and correct.
2. That Council authorizes the City Manager to conduct a citywide
Paired Comparison Opinion Survey using the services of Cal
‘State San Marcos Social and Behavioral Research Institute.
3
3. That funds in the amount of twenty thousand dollars are hereby
allocated from the Council contingency fund.
PASSED, APPROVED AND ADOPTED at a regular meeting of the
City Council of the City of Carlsbad held on the 20th day of
March I 2001 by the following vote, to wit:
AYES: Council Members Lewis, Kulchin, Nygaard and Hall.
NOES: None.
ABSENT: Council Member Finnila.
(SEAL) ti
.
Sail Marcos. California . USA 920960001
Phone: (760) 750-3258
FAX: (760) 750-3291 Social & Behavioral Research Institute
MEMORANDUM
DATE:
TO:
February 19,200l
Raymond R. Patchett; City Manager
City of Carlsbad
FROM: Allen Risley, Associate Director
Social & Behavioral Research Institute, CSUSM
SUBJECT: Discussion of Paired Comparison/Economic Choice Survey
Included in the discussion of the results of the 2000 Carlsbad Citizen Survey has been some
conversation regarding deeper analyses of residents’ opinions regarding a specific set of proposed
programs. This document has been developed in response to the City of Carlsbad’s request for
further information on alternative methodological approaches to studying the priorities of city
residents. The discussion to date has centered around two possible methods of data collection that
can be used to measure citizen preferences within the context of economic trade-offs: paired
comparison tests and a hypothetical budget test.
The information presented below is structured so as to inform discussion on the design of a
follow-up survey. With an eye towards budget and time constraints, questionnaire sections are
described in terms of their topical content, methodological approach and time requirements. Draft
cost figures for two possible questionnaire lengths are presented for you consideration and
discussion. It is expected that based on the two cost models, there will be discussion on which
types of question formats can be included in the survey design. Decisions on the question formats
will also drive the discussion of which topical areas (proposed programs) can be included in the
final survey draft.
Cost model 1: 500 Interviews (250 North/250 South), 10 minutes in length, 3 weeks of
data collection. $16,900
Cost model 2: 500 Interviews (250 North/250 South), 15 minutes in length, 5 weeks of
data collection. $20,100
The California State University
B~kerslield - Channel Islands - Chic0 * Dominguez Hills * Frcsno * Fullerton * Hayward * Humboldt l Long Beach * Los Angeles l Maritime Academy l
\ll)ntcrey l3ay . Northridge * Pomona * Sacramento * San Bernardino * San Diego - San Francisco - San Jose - San Luis Obispo * San Marcos * Sonoma l Stanislaus
TDD(760)750-3lS5
5
. Questionnaire Content Sections:
Introduction and Demographics (Time Budget - 3 minutes):
This section would include text and questions at the beginning of the survey and the end
that cover the following topics:
l Introduction of the survey topic.
l Screening of respondents (city residence, zip code).
l Demographics, including: age, gender, income, household size and presence of
children, race/ethnicity, length of city residence, rent vs. own.
Paired Comparisons (Time Budget - 3-4 minutes for 3 comparison items, 6-7 minutes for 4
comparison items, lo-12 minutes for 5 comparison items):
Paired comparison tests consist of a series of questions that pit each item from a set of
items against every other item in a series of one-on-one comparisons, where respondents
are asked to choose which item they prefer from each pair of items. The subsequent
analytical method allows for a tallying of the pairs “won” by each item in the set. We
would propose to conduct the paired comparison tests using the following guidelines:
l Include an initial transition discussion in the questionnaire where the interviewer
describes the pattern of questions that will follow.
Present the paired comparison questions twice for each (any) set of items: first
with no mention of the costs associated with the item, followed by a second
presentation of the paired comparisons with cost figures included. This allows for
three separate analyses:
n Analysis of pure baseline preferences for programs, regardless of
cost.
. Analysis of preferences for programs once costs are included.
. Analysis of the impact of costs to support for programs.
l The time impact of increasing the number of items in a set is geometric. Testing 3
comparison items requires 3 paired comparison questions; testing 4 comparison
items requires 6 paired comparison questions; and testing 5 comparison items
requires 10 paired comparison questions. Testing more than five items in a paired
comparison structure is not recommended because of the tremendous increase in
the number of individual questions that are required. Increasing the number of
questions negatively affects time constraints and contributes to respondent fatigue.
Hypothetical Budget (Time Budget - 2-3 minutes for 4 budget items, 3-4 minutes for 5
comparison items):
The hypothetical budget test is presented to the survey respondent in the fort-n of a
scenario, where the respondent is given a hypothetical “budget”, and a list of possible
programs that the budget can be spent on. Respondents are directed to spend the entire
budget, using whatever breakdown they desire. We would propose to conduct the
hypothetical budget test using the following guidelines:
l Include an initial transition discussion in the questionnaire where the interviewer
describes the scenario, lists the programs to be included, and presents the structure
for assigning dollar amounts.
l Total dollars “spent” and dollars “left to spend” are calculated on the fly by the
CAT1 system software, allowing the interviewer to continually prompt the
respondent with the impact of their spending choices.
l This data collection method allows respondents to create “distance” between
budget priorities, allowing for an assessment of the relative weight of priorities
held by respondents.
Any of the programs included in the previous telephone survey are appropriate for use with these
survey methodologies, provided that the length and style of description of the programs are
foughiy equivalent. The paired comparison test would also be appropriate for testing preferences
for the various types of development covered in the earlier survey:
Additional questions that are not a part of either the paired comparison or hypothetical budget
tests can also be included in the questionnaire.