Loading...
HomeMy WebLinkAbout2001-03-20; City Council; 16122; Citywide Paired Comparison Opinion SurveyAB# &&(,a MTG. c.. -a 01 DEPT. CM CITY OF CARLSBAD -AGENDA BILL c b ! TITLE- Authorize Citywide Paired Comparison Opinion Survey to be conducted by Cal State San Marcos Social Behavioral Research Institute DEPT. HD. CITY All-Y. CITY MGR RECOMMENDED ACTION: Authorize Citywide Paired Comparison Opinion Survey to be conducted by Cal State San Marcos Social Behavioral Research institute and Allocate $20,000 from the City Council Contingency Fund. (and adopt Resolution NO. 2001-89) ITEM EXPLANATION: In the summer of 2000, the City contracted Cal State San Marcos Social and Behavioral Research Institute (SBRI) to perform the citywide public opinion survey. This survey was conducted in the fall of 2000, with the draft results reported at the City Council goal workshop in February 2001. At that workshop, the City Council asked SBRI to recommend an approach to provide a deeper analyses of residents’ opinions regarding a specific set of proposed programs / projects. In response to the Council’s request, SBRI has proposed conducting a follow-up survey based on a paired comparison / economic choice model. SBRl’s recommendation is to perform a telephone survey of approximately 500 residents, first asking respondents to state a preference given a choice between two City programs I projects. This paired choice comparison questioning would continue through the entire set of proposed programs I projects in order to provide rank order information. Secondly, the respondents would be given the estimated program / project cost, and asked again to answer the same paired choice comparisons, taking this information into account. Using the paired comparison / economic choice methodology, we can derive information about citizens priorities with and without cost values attached to them, and see how priorities change when cost is taken into account. Based upon SBRl’s recommendation, this type of methodology lends itself to ranking four (4) projects / programs. Staff proposes using the following projects / programs in this analysis: 1. Fifty-meter Pool in south Carlsbad (with room for expansion into an aquatics center) 2. Municipal Golf Course 3. Northwest Quadrant Community Park (a.k.a. Pine School site) 4. Citywide Trails Program As stated above, the survey will interview 500 residents with 250 living North of Palomar Airport Road and 250 living South of Palomar Airport Road. The survey will take approximately 6-8 weeks to complete. The results of this survey will be brought back to Council once the data has been collected and analyzed. I PAGE 2 OF AGENDA BILL NO. 1 lo! ) aa FISCAL IMPACT: The projected cost for this project is $20,000, to be allocated from the City Council Contingency Fund EXHIBITS: 1. Letter to Raymond R. Patchett, City Manager, from Allen Risely, Associate Director, Social & Behavioral Research Institute, CSUSM dated February 19, 2001. 2. Resolution No. 2001-89 2 RESOLUTION NO.Z!ool-89 A RESOLUTION OF THE CITY COUNCIL OF THE CITY OF CARLSBAD, CALIFORNIA, To authorize preparation of a citywide Paired Comparison Opinion Survey to be conducted by Cal State San Marcos Social and Behavioral Research Institute and appropriating funds. WHEREAS, the City of Carlsbad wishes to be responsive to the needs and preferences of our citizens; and WHEREAS, the City of Carlsbad has a multi-million dollar Capital Improvement Program designed to provide important public facilities to the community; and WHEREAS, the Capital Improvement Program should reflect the priorities of the City Council and the citizens of Carlsbad; and WHEREAS, one way to establish priorities is through the process of surveying using a Paired Comparison model; and WHEREAS, Cal State, San Marcos Social and Behavioral Research Institute has the technical skills to complete a survey in a professional and useful manner. NOW, THEREFORE, BE IT RESOLVED by the City Council of the City of Carlsbad, California, as follows: 1. That the above recitations are true and correct. 2. That Council authorizes the City Manager to conduct a citywide Paired Comparison Opinion Survey using the services of Cal ‘State San Marcos Social and Behavioral Research Institute. 3 3. That funds in the amount of twenty thousand dollars are hereby allocated from the Council contingency fund. PASSED, APPROVED AND ADOPTED at a regular meeting of the City Council of the City of Carlsbad held on the 20th day of March I 2001 by the following vote, to wit: AYES: Council Members Lewis, Kulchin, Nygaard and Hall. NOES: None. ABSENT: Council Member Finnila. (SEAL) ti . Sail Marcos. California . USA 920960001 Phone: (760) 750-3258 FAX: (760) 750-3291 Social & Behavioral Research Institute MEMORANDUM DATE: TO: February 19,200l Raymond R. Patchett; City Manager City of Carlsbad FROM: Allen Risley, Associate Director Social & Behavioral Research Institute, CSUSM SUBJECT: Discussion of Paired Comparison/Economic Choice Survey Included in the discussion of the results of the 2000 Carlsbad Citizen Survey has been some conversation regarding deeper analyses of residents’ opinions regarding a specific set of proposed programs. This document has been developed in response to the City of Carlsbad’s request for further information on alternative methodological approaches to studying the priorities of city residents. The discussion to date has centered around two possible methods of data collection that can be used to measure citizen preferences within the context of economic trade-offs: paired comparison tests and a hypothetical budget test. The information presented below is structured so as to inform discussion on the design of a follow-up survey. With an eye towards budget and time constraints, questionnaire sections are described in terms of their topical content, methodological approach and time requirements. Draft cost figures for two possible questionnaire lengths are presented for you consideration and discussion. It is expected that based on the two cost models, there will be discussion on which types of question formats can be included in the survey design. Decisions on the question formats will also drive the discussion of which topical areas (proposed programs) can be included in the final survey draft. Cost model 1: 500 Interviews (250 North/250 South), 10 minutes in length, 3 weeks of data collection. $16,900 Cost model 2: 500 Interviews (250 North/250 South), 15 minutes in length, 5 weeks of data collection. $20,100 The California State University B~kerslield - Channel Islands - Chic0 * Dominguez Hills * Frcsno * Fullerton * Hayward * Humboldt l Long Beach * Los Angeles l Maritime Academy l \ll)ntcrey l3ay . Northridge * Pomona * Sacramento * San Bernardino * San Diego - San Francisco - San Jose - San Luis Obispo * San Marcos * Sonoma l Stanislaus TDD(760)750-3lS5 5 . Questionnaire Content Sections: Introduction and Demographics (Time Budget - 3 minutes): This section would include text and questions at the beginning of the survey and the end that cover the following topics: l Introduction of the survey topic. l Screening of respondents (city residence, zip code). l Demographics, including: age, gender, income, household size and presence of children, race/ethnicity, length of city residence, rent vs. own. Paired Comparisons (Time Budget - 3-4 minutes for 3 comparison items, 6-7 minutes for 4 comparison items, lo-12 minutes for 5 comparison items): Paired comparison tests consist of a series of questions that pit each item from a set of items against every other item in a series of one-on-one comparisons, where respondents are asked to choose which item they prefer from each pair of items. The subsequent analytical method allows for a tallying of the pairs “won” by each item in the set. We would propose to conduct the paired comparison tests using the following guidelines: l Include an initial transition discussion in the questionnaire where the interviewer describes the pattern of questions that will follow. Present the paired comparison questions twice for each (any) set of items: first with no mention of the costs associated with the item, followed by a second presentation of the paired comparisons with cost figures included. This allows for three separate analyses: n Analysis of pure baseline preferences for programs, regardless of cost. . Analysis of preferences for programs once costs are included. . Analysis of the impact of costs to support for programs. l The time impact of increasing the number of items in a set is geometric. Testing 3 comparison items requires 3 paired comparison questions; testing 4 comparison items requires 6 paired comparison questions; and testing 5 comparison items requires 10 paired comparison questions. Testing more than five items in a paired comparison structure is not recommended because of the tremendous increase in the number of individual questions that are required. Increasing the number of questions negatively affects time constraints and contributes to respondent fatigue. Hypothetical Budget (Time Budget - 2-3 minutes for 4 budget items, 3-4 minutes for 5 comparison items): The hypothetical budget test is presented to the survey respondent in the fort-n of a scenario, where the respondent is given a hypothetical “budget”, and a list of possible programs that the budget can be spent on. Respondents are directed to spend the entire budget, using whatever breakdown they desire. We would propose to conduct the hypothetical budget test using the following guidelines: l Include an initial transition discussion in the questionnaire where the interviewer describes the scenario, lists the programs to be included, and presents the structure for assigning dollar amounts. l Total dollars “spent” and dollars “left to spend” are calculated on the fly by the CAT1 system software, allowing the interviewer to continually prompt the respondent with the impact of their spending choices. l This data collection method allows respondents to create “distance” between budget priorities, allowing for an assessment of the relative weight of priorities held by respondents. Any of the programs included in the previous telephone survey are appropriate for use with these survey methodologies, provided that the length and style of description of the programs are foughiy equivalent. The paired comparison test would also be appropriate for testing preferences for the various types of development covered in the earlier survey: Additional questions that are not a part of either the paired comparison or hypothetical budget tests can also be included in the questionnaire.