COMPARING RANDOM DIGIT DIAL (RDD) AND UNITED STATES POSTAL SERVICE (USPS) ADDRESS-BASED SAMPLE DESIGNS FOR A GENERAL POPULATION SURVEY - THE 2008 MASSACHUSETTS HEALTH INSURANCE SURVEY: This research compares respondents from a Random Digit Dial (RDD) sample design with respondents from a United States Postal Service (USPS) address list sample design for a general population study conducted to gather information on health insurance coverage in Massachusetts. The Urban Institute and International Communications Research collaborated in conducting the evaluation using a dual sample frame (RDD and USPS) approach. While the overall plan was to combine the two samples, a secondary goal was to choose large enough samples of both the RDD and the USPS so that they could produce independent estimates as well. Prior State surveys that have been used to evaluate health insurance coverage have largely been based on a RDD sample designs. However, with the coverage problems associated with the increasing numbers of households not having a landline telephone, survey sponsors have been exploring the potential of the USPS address lists to serve as a sampling frame for probability samples from the general population. This research looks at the relative coverage properties of the two frames along with response rates and measures of efficiency. The research provides insight into the coverage and cost/benefit trade-offs that researchers can expect from RDD sample designs that conduct all interviews over the phone versus using a full or combined USPS address sample design that utilizes a multi-mode (phone, web, mail) data collection approach.
CAN YOUR SPOUSE ACCURATELY REPORT YOUR ACTIVITIES - AN EXAMINATION OF PROXY REPORTING FROM THE 2008 SURVEY OF PUBLIC PARTICIPATION IN THE ARTS: The 2008 Survey of Public Participation in the Arts (SPPA) is sponsored by the National Endowment for the Arts and was conducted as a supplement to the May 2008 Current Population Survey (CPS). A big challenge in being a supplement to the CPS is that respondent selection procedures for SPPA supplement differ from the CPS procedures. CPS is administered to any person 16 or older who is able to report employment information on all persons 16 years or older in the household. While the SPPA collects information on all adults 18 or older and it is felt that many of the questions on the survey require self reports rather than proxy reports. In 2002, the Census Bureau interviewers attempted to complete the SPPA supplement with all persons 18 or older, but after 4 call attempts they accepted proxy reports. To make the SPPA a better fit for the CPS protocol, rather than attempt to interview all adults in the household , the 2008 SPPA accepted proxy responses for spouses or partners (for many of the questions). This change in design makes it much easier to measure the impact of proxy reports given that they were collected by design rather than out of necessity. This research will explore the extent to which proxy reporting may have resulted in over or under reporting participation. And when there are differences should you adjust your estimate? Of particular interest will be comparisons between husbands reporting about the wives activities and vice a versa. In addition, this research will explore whether the quality of proxy reporting varies by key population sub groups.
DEVELOPING A USER'S GUIDE FOR THE NATIONAL ENDOWMENT FOR THE ARTS (NEA) 2008 SURVEY OF PUBLIC PARTICIPATION IN THE ARTS (SPPA): The SPPA is the field's premiere repeated cross sectional survey of individual attendance and involvement in arts and cultural activity. The data are circulated to interested researchers, and they are the basis for a range of NEA reports and independent research publications. The SPPA provides primary knowledge on the extent and nature of participation in the arts in the United States. Continuous improvement of the SPPA so that its estimates are as accurate as possible is a worthy goal. This goal of this research is to provide SPPA data users with enough information about the data and the survey itself to be able to correctly use the public use data.
The 2008 SPPA was redesigned to better handle a number of important design issues that arose from the 2002 version. The main goal this time around was a design that would be less burdensome. There were two major design changes that reduced the average burden:
1. Rather than attempt to interview all adults in the household (as in 2002), the 2008 SPPA randomly sampled adults and accepted proxy responses for spouses or partners (for many of the questions). While proxy responses were accepted in 2002, this time the questions were properly worded so that it is clear who the question is referring to and proxy responses were clearly identified in the data file prepared by the Census Bureau.
2. Rather than administer the entire SPPA survey to all respondents, the questionnaire was separated into modules, so that any one respondent only answered the core arts attendance questions and 2 of the 4 other modules.
Understanding how these two design changes impact the data is essential for researchers planning to use the 2008 SPPA data.
AN ASSESSMENT OF THE RESPONSE RATE HISTORY OF THE NATIONAL SURVEY OF AMERICA'S FAMILIES: The decline in telephone survey response rates continues in the face of the most exhaustive efforts of survey data producers. Understanding the relationship between nonresponse procedures and survey estimates is central to the evaluation of nonresponse bias and can provide a clearer understanding of the future of telephone-based data collection. The goal of this research is to evaluate the response rates and survey operations history of the National Survey of America's Families (NSAF) across six years and three rounds of data collection.
For each round of data collection, a variety of initiatives were employed to improve declining response rates, including pre-notification letters, monetary incentives, changes in call scheduling, refusal conversion, and refusal sub-sampling. This analysis focuses on changes in data collection procedures, call disposition results and sample composition across rounds. This includes examining how changes made to incentive distribution plans, call schedules, maximum call attempts and refusal conversion protocols impacted response rates over time. It also includes assessing the impact that these procedural and response rate changes may have had on NSAF's sample composition across the three rounds.
SOCIAL AND DEMOGRAPHIC STUDY OF TELEPHONE SURVEY NONRESPONDENTS: Response rates for telephone surveys in the United States are declining. The impact of nonresponse on estimates derived from a telephone survey depends on the extent to which nonrespondents differ from respondents on survey items of interest. If all telephone households had equal probability of responding to a survey, the estimates would be unbiased and the decline in response rate would only affect the number of households that would have to be called to reach the targeted sample size. Whether the estimates are unbiased thus depends on the assumption that nonrespondents are missing at random (MAR).
There is little empirical evidence to evaluate the MAR assumption because we seldom have enough information about nonrespondents. However, classifying nonresponse as ignorable or non-ignorable can often be done using auxiliary information available for respondents and non-respondents. This research will use Census block group level data to predict characteristics of nonrespondents and develop linking equations to predict respondent characteristics. Using these models, we will estimate what percentage of the nonresponse can be considered ignorable.
This research will use data from the 1997 and 2002 rounds of the Urban Institute's National Survey of America's Families (NSAF), which was designed to provide a comprehensive look at the well-being of children and non-elderly adults. A major benefit of NSAF data is the large size of its random-digit-dial starting sample (483,260 telephone numbers in 1997, and 556,651 used in 2002) relative to other national random-digit-dial telephone surveys. In both rounds many household addresses were obtained through the use of reverse directory call back services, which trace addresses to telephone numbers. Households for which addresses were obtained are matchable to Census block group and to Census data for those block groups. For households where address was not obtained we will use telephone exchange information to find zip code and then assign Census data to households at the zip code level.
EXAMINING THE EFFECTS OF DATA QUALITY ON NONRESPONSE ADJUSTMENTS IN RDD SURVEYS: For random digit dial (RDD) telephone surveys, nonresponse adjustments are often carried out by modeling (either implicitly or explicitly) the survey participation decision, using auxiliary data that is available for both respondents and nonrespondents. While a considerable amount of research has been carried out examining issues of functional form with respect to such adjustments, relatively little effort has been expended on examining how the adjustments are affected by differences in data quality.
In many RDD telephone surveys, only aggregate data (defined at some unit of geography such as the census tract, county, zip code or telephone exchange) are available for both respondents and nonrespondents. This research uses data from the 2002 National Survey of America's Families (NSAF) to examine the degree to which different sources of aggregate data have the potential to affect nonresponse adjustments. The first source of auxiliary data consists of characteristics at the telephone exchange level. The second source consist of county level estimates of characteristics based on long-form data from the 2000 Census. The third source of auxiliary data that are available for both respondents and nonrespondents with mailable addresses consists of data at the census tract level from the 2000 Census.
When this research is completed the results will be available in the NSAF methodology series which is available on the Urban Institute's web site.
UPDATING GRAND SLAM STATISTICS ON TENNIS PLAYERS: The Grand Slam tennis program is a statistical model that allows you to simulate matches between former winners of grand slam tournaments (Australian Open, French Open, Wimbledon, and the U.S. Open). I am intermittently researching an updating the accuracy of the statistics used in the model and when appropriate I will add new players. You are welcome to download a free shareware copy of the "Grand Slam Tennis" program from this web site.
Timothy Triplett's Home Page: