Survey Administration

CSS uses rigorous methods to ensure smooth and accurate implementation of all aspects of the survey fielding process.

Preparing the Sample

CSS has expertise in complex sampling procedures—for example, in selecting patients to survey, giving priority based on a hierarchy of factors, such as, date of most recent visit, number of visits, number of visits by other family members, plan PCP assignment, physician specialty, and number of patients already sampled for the same physician.  We work with clients to help them design their sampling strategy and understand the impact of the design.

A key to project success is making certain that the sample frame is of as high quality as possible.  CSS has extensive experience dealing with sample frames from various sources, which themselves vary in data management expertise.  

On most projects, CSS strives to minimize the client burden by accepting the files in the client's standard format. 

Upon receiving the sampling frame files, CSS conducts a thorough quality assurance (QA) check of each submitter's data and works with individual submitters, as needed, to correct errors in their data. CSS typically runs reports for each data submitter showing the total record count and the number of records that contain each type of error that would disqualify those records for use in the survey. 

CSS also provides other counts on the sampling frame file to assist the client in checking that the sampling frame is complete.  For a typical survey of patients about doctors, for example, CSS provides for each doctor a summary report showing distributions by month of patient's last visit, gender, age range, first three digits of ZIP code, and payer identifier.

For some projects, CSS has provided each submitter an executable program that will import test files and check for errors in the patient records that would eliminate them from being used in the survey.  This allows data issues to be identified even before the data are submitted to CSS.  These errors might include formatting errors, missing data, and errors related to base survey rules (age range, assigned PCP, etc.). The software allows each submitter to check its data prior to submission and can serve as a useful tool in identifying any errors. 

In addition, CSS conducts visual checks on a dump of records. In the past, these checks have found surprising data problems that made the sample frames unusable but that would not have been identified in a routine computer check. 

Unless the client opts not to, CSS processes the sample frame data for each project through National Change of Address and phone look-up services to make sure contact information is as current as possible.

All sample members are assigned unique, confidential tracking numbers that are used to identify the status of each sample member throughout the survey process. A master sample tracking database will contain information on such matters as whether the questionnaire was returned, whether the survey was completed on the phone or over the Internet, whether the surveyed individual indicated that he/she would not like to be surveyed, whether the individual requested an alternative-language survey,  whether the mailing was returned as undeliverable,  whether the individual had a useable phone number after phone lookup, whether the individual could not be interviewed because of a language problem, whether the individual broke off during an Internet survey, and other relevant facts.

Mail Survey Fielding

Survey materials are designed to get maximum response rates at appropriate costs.  We advise clients on various package components and formats and their likely effects on results.

The survey package typically includes a survey, personalized letter, and a postage-paid business reply envelope in a carrier envelope. The appropriate identifiers (survey sponsor, plan, physician, CSS, or other entity) are used in the communications in the interest of full disclosure to the respondent and enhancement of response rate.

The format of the questionnaire depends on length and client preference (with the benefit of CSS's advice regarding postage cost, printing cost, and expected response rate).  Common formats include compact multi-page bound booklets, folded-down 14” by 17” sheets, six page 8.5” by 11”, single 8.5” by 11”, and many other variants in size as well as color, graphics, font, and font size.   

Mail survey packages are most often sent First Class using metered postage, but we offer other options as appropriate, including live postage stamps, different Postal Service rates and services (Priority, Standard Commercial, Standard Non-profit, and Certified), and non-Postal Service delivery (such as, FedEx). 

We also help clients choose and implement the best feasible options in terms of number of attempts and survey timing.

Online Fielding

For clients that can provide e-mail addresses, CSS can solicit responses by e-mail.  We have substantial experience in composing and designing such e-mail solicitations and in the steps necessary to maximize e-mail delivery.

Alternatively, individuals can be invited in a mail cover letter or postcard to go to a website to enter identifier information and complete the survey.  Online surveys are hosted by CSS, using an online instrument designed by CSS and approved by the client.  CSS ensures its online data collection is secure and accessible to web users.

CSS works with clients to design the online survey and appropriate page breaks to balance appearance and ease of use. As respondents progress through the survey, they are given the option of going back and changing answers or exiting the survey and later resuming where they left off. All skip patterns are transparent to the respondent.

Telephone Survey Fielding

CSS can use the telephone as the sole survey mode or as a follow-up tool to reach individuals who have not responded to mail or Internet attempts.
Telephone calls are handled using Computer Assisted Telephone Interviewing (CATI) technology. 

In mixed-mode surveys, during the entire telephone follow-up period, newly received completed questionnaires from the mail or Internet are accepted, and those respondents are suppressed from CATI interviewing. Likewise, respondents who are identified as refused, deceased, or ineligible during this time also are suppressed from CATI interviewing.

All CATI scripts are supplied to the client for approval.  Following client approval, the CATI system is programmed with built-in range and consistency checking and is programmed to follow all interview skip patterns. The CATI system is set up so that: 

• It will automatically schedule call-backs.

• An interview can be suspended and then resumed at a later time or date where left off.

• It will handle closed-ended questions, open-ended questions, and multiple response questions.

• It can accommodate multiple questionnaire versions with different question subsets so that the appropriate version comes up based on the respondent ID.

• The interviewer can go back as far as needed to correct incorrect entries.

• It can display comments from previous callers at the time the record comes up for the current caller.

Steps to enhance response rates and assure quality of data include the following:

·        Only experienced interviewers are allowed to work on any CSS project.

·        All interviewers are fully trained in basic interviewing techniques, such as reading all questions exactly as worded, maintaining a professional and neutral relationship with the respondent, and using non-directive probes if a respondent does not give an adequate answer.

·        A project-specific interviewer manual is distributed to every CATI interviewer.

·        CSS research staff members conduct project-specific training sessions at the CATI facility for all interviewers before they are allowed on the phone. Training ensures that interviewers understand the purpose of the project, the meaning of each question, and CSS-specified coding rules.

·        CSS creates a video-tape of a training session (including commonly asked questions) to be available for refresher training for interviewers.

·        All interviewers are required to do “mock” project-specific interviews before they are allowed on the phone.

·        Regular monitoring and feedback sessions are conducted with all interviewers.

·        At least 10 percent of completed interviews are monitored by on-site CATI supervisors.

·        Silent monitoring of interviews includes parallel monitoring of the interviewer's data entry on a computer monitor that matches the interviewer's monitor.

·        CSS's research staff remotely monitors a sample of interviews at least every other day of CATI data collection. Client representatives are welcome to join CSS staff in live remote monitoring of the interviewers.

·        Each interviewer is given regular performance evaluation forms rating the interviewer on such matters as the introduction, question-asking, repeating questions, probing/defining/clarifying, pace/timing, and overall clarity.

·        Interviewers ask for another number if the respondent is not available at the called number.

·        Interviewers call back at a scheduled time if the respondent is away at the time of a call.

·        Interviewers call back to resume interviews that must be suspended. 

·        Interviewers call back at a later date if a respondent is too ill to be interviewed.

·        CATI response data files are checked early in the CATI process for consistency with mail response data files.

Foreign Languages

CSS is experienced in providing translations and fielding surveys in many different languages.  Each translation is translated from English to the alternative language and then translated back to English by different translators to ensure that the alternative language text matches the original English text.  In addition to English, CSS has conducted surveys in Spanish, Chinese, Vietnamese, Russian, and Korean.