Weighting

National Institutes of Mental Health (NIMH)
Collaborative Psychiatric Epidemiology Survey Program (CPES) Data Set.
Integrated Weights and Sampling Error Codes for Design-based Analysis

Steven G. Heeringa, Patricia Berglund
Statistical Design Group, Survey Research Center, University of Michigan

June 4, 2007

I. Introduction

Under contract to the National Institutes of Mental Health (NIMH), the Survey Research Center (SRC) has developed an integrated data base for the Collaborative Psychiatric Epidemiology (CPES) surveys: National Comorbidity Survey-Replication (NCS-R), National Survey of American Life (NSAL) and National Latino and Asian American Study (NLAAS). Heeringa, et al (2004) describe the sample designs and sample outcomes for the three CPES surveys. A general description of the survey methodology for the CPES surveys can be found in Pennell, et al. (2004).

This technical report outlines the method for integrating the design-based analysis weights and variance estimation codes for these three studies to permit analysts to approach analysis of the combined dataset as though it were a single, nationally-representative study.

The method of integrating the analysis of these three major survey programs was based on an adaptation of a multiple frame approach to estimation and inference for population characteristics (Hartley, 1962, 1974). There are several features and advantages to the method that are worth noting:

1. It was built on all of the study-specific weight development efforts conducted to date (Kessler et al. , 2004; Heeringa et al. 2004; Heeringa, et al. 2006).

2. It integrated overlapping representation of domains of the CPES survey population in a way that was mathematically transparent and easily understood by analysts of the combined data set. Given the large investments in study-specific weight development, this approach minimized the chance for conceptual or computational errors.

3. It was centered on the assumption that, conditional on the sample domain (e.g., block groups with 10-29.9% African American population) and the race/ethnicity of the respondent (e.g., Mexican-American), each study's sample representation based on the revised weight is proportional to the number of cases it "contributes" to the geographic domain x race/ethnicity cell.

II. CPES Survey Population

The CPES survey population was defined by the union of the survey populations for the three component studies. This included adults age 18 and older, living in households in the 48 coterminous United States (NCS-R, NSAL). The survey population for the Latino and Asian ancestry groups extended to the State of Hawaii as well.

III. Race/Ancestry Populations

CPES analysts are free to define respondent groupings for analysis; however, for purposes of weight development twelve specific race/ancestry groupings were initially specified. These groupings are listed in Table 1. Due to the small number of persons of Other ancestry interviewed in the NCS-R, those individuals were combined with the White race category for purposes of the CPES weight computation.

The breakdown of the full population into these 12 race/ancestry populations was a direct result of the specific eligibility and oversampling provisions of the NSAL and the NLAAS study designs. As shown in Table 1, NCS-R provided nearly universal coverage of all 12 race/ancestry groups. NSAL and NLAAS provided in-depth coverage of specific populations and with the exception of Afro-Caribbeans from Spanish language countries in the Caribbean (e.g., Cuba, Dominican Republic), the oversampling in each of these two studies did not overlap.

These 12 population groupings form the first dimension of a two-dimensional array that was used to apportion/adjust study-specific weights to create a new weight variable for integrated CPES analyses. These "population" groupings were defined at the respondent level. If individual respondents had multiple race/ancestry, they were assigned to a single category according to the priority order in the NLAAS and NSAL respondent classification rules (e.g. Afro-Caribbean taking preference over African-American, Vietnamese over Chinese). If ancestry for NCS-R cases could not be explicitly established at the level of detail required to map them into the NSAL or NLAAS population categories, they were stochastically assigned to a category based on the prevalence of each population in the Census Block Group in which the respondent's household was located.

IV. Sample Frame Geographic Domains

The second dimension of the CPES weight computation array was defined based on the geographic domain of the U.S. national sample frame with which individual area segments for the three component samples were associated (see Heeringa et al., 2004). The "domain" groupings were assigned at the area segment level. All respondents from the same segment, regardless of population, were assigned to the same domain. Table 2 defines the 11 domain categories that were used to classify area segments and thereby assign each CPES respondents to a geographic domain.

All segment assignments to geographic domains were performed using Census 2000 data for Block Groups. Like the population assignments for mixed ancestry respondents, area segment domain assignments based on this 11 category classification were not always unique. For example a Census Block Group might have contained a population that was >5% Vietnamese and also >5% Chinese. In cases where a Census Block Group qualified for more than one domain, the corresponding area segment was assigned to the lowest numbered category (e.g. the high density Vietnamese domain for this last example).

V. Integrated weight for the pooled CPES data set

Case-specific population weights had been developed for each CPES component survey (Kessler et al., 2004; Heeringa, 2004; Heeringa et al., 2005). Each project had carefully developed and refined its weight vector to enable robust probability sampling inference ("design-based") to its chosen survey population. NCS-R was unique among the three component studies in that it required two final analysis weights--one weight for the full sample of cases who participated in the Part 1 interview and a second for the subsample of cases that also completed Part 2 of the NCS-R. Consequently, the CPES combined data set also has two analysis weights--the first for analysis of common data items and the second for analysis of survey items that NCS-R only administered to Part 2 respondents.

The integrated weight development began with the existing final population weights for the NCS-R, NSAL, and NLAAS. The integrated weight development then proceeded according to the following steps:

Step 1. Each NLAAS, NSAL, and NCS-R case was assigned to a race/ancestry category based on the categories and priority order provided in Table 1 (see Section III).

Step 2. Each NLAAS, NSAL, and NCS-R area segment was assigned to a geographic domain based on the definitions and priority order shown in Table 2 (see Section IV). Each NLAAS, NSAL, and NCS-R respondent was assigned to a geographic domain based on its area segment classification.

Step 3. The final population weight values for the three data sets were obtained from the NLAAS, NSAL, and NCS-R investigators. Since the final NCS-R and NSAL weights had been "centered" or "normalized" (mean weight=1.0), they were restored to the original U.S. population scaling based on weighted totals from the March 2002 demographic supplement of the Current Population Survey ( CPS).

Step 4. Notation: each case in the CPES pooled data set was indexed as follows:

Step 5. From the pooled data set, EXCEL spreadsheets were used to compute the sums of nominal cases for each study by race/ancestry population by geographic domain cell. These counts were then aggregated across the three studies to produce CPES pooled case counts for each population x domain cell:

Step 6. The March 2002 CPS data enabled estimation of post-stratification control totals for each race/ancestry group, k=1,...,11; however, it did not provide the geographic detail needed to allocate the population total to the l=1,...,11 geographic domains. For this purpose, the weighted population distribution from the CPES study with the most robust estimates of geographic distribution was used. NLAAS was chosen as the basis for allocating the Asian and Hispanic populations to the 11 sample geographic domains. NSAL weighted sample distributions were used to apportion the African-American and Afro-Caribbean populations to the geographic domains. White and Other population totals were allocated to geographic domains based on the empirical distribution of weights in the NCS-R.

where:

= the CPES control total for race/ethnicity population k and domain l,

= the original study-specific weight for case i, study,

= the CPES study chosen to estimate the domain allocation for population k, and

= the March 2002 CPS population estimate for race/ethnicity category k.

Table 4 provides the final population controls for the race/ethnicity x domains cells of the CPES weight computation array.

Step 7: The original population weights from each study were post-stratified to the common race/ethnicity x domain population control totals derived from the March 2002 CPS (see Step 6 and Table 4).

where:

= the study specific weight adjusted to 2002 CPS population totals,

= the original study-specific population weight for case i, study j,

= the 2002 CPS estimate for race/ethnicity population k allocated to domain l.

Since the original study-specific weights for the major populations of interest had already included some form of population-based control, this rescaling to a common post-stratification standard did not require major adjustments.

Step 8. Since in Step 7 the individual study-specific weights were controlled to exact counts for each race/ancestry x geographic domain cell, the remaining step involved rescaling the study-specific weights to reflect the proportion of nominal cases that each study contributed to the cell in the pooled data set.

where:

= the CPES population weight for case i;

= the standard population weight for case i, study j, (assigned to population k and domain l).

Conditional on the assigned population (k) and domain classification (l), this rescaling provided a "proportionate to sample size" contribution from each study. It linearly rescaled the weights for each individual study. It did not alter the distribution of the study-specific population weights except to reduce the study specific mean by n+jkl/n++kl and the variance of the study weights (not relvariance) by a factor of (n+jkl/n++kl)2.

VI. Final Weights and Special Analysis Consideration for Weighted Analysis

VI.A Final Weights

As noted in Section V above, n=9282 NCS-R survey respondents completed Part 1 of the two-part CIDI-based interview; however, only a subsample of n=5692 NCS-R respondents went on to complete the more in-depth Part 2 questionnaire modules. All n=6082 NSAL and n=4649 NLAAS respondents completed the full interview schedule--the equivalent of NCS-R Parts 1 and 2. To account for the split schedule of NCS-R questionnaire administration, two CPES pooled analysis weights were computed using the calculation sequence described in steps (5) through (8) in Section V above. The first weight, termed the "Part 1" weight, is labeled CPESWTSH in the merged CPES data set. It is the population weight that should be used for analysis involving variables that are included in Part 1 of the NCS-R. The second weight, termed the "Part 2" weight, is stored in the merged data set as the variable, CPESWTLG. It is the population weight that should be used when the CPES analysis includes variables that NCS-R only asked of the Part 2 subsample of respondents.

Table 5 provides a summary of selected distributional statistics for the final CPES Part 1 and Part 2 analysis weights.

VI.B Special Considerations in Weighted Analysis of the CPES Data

The weights have been designed to enable analysts to compute unbiased or nearly unbiased estimates of population statistics and relationships (e.g. bivariate associations, regression relationships) for the larger CPES survey population of U.S. residents. Contemporary statistical software systems such as SAS, Stata, SPSS, and SUDAAN all provide the capability to conduct weighted analysis of the CPES survey data. CPES data analysts are encouraged to consult the user guides and help support for their chosen software package to learn the syntax and program specific features for conducting weighted analysis. The following paragraphs provide guidance on weighted analysis that is specific to the CPES data set.

VI.B. 1 Part 1 or Part 2 Weight ?

CPES analysts should consult the data documentation to determine if variables of interest in their analysis were obtained in Part 1 or Part 2 on the NCS-R interview. If the analysis includes only Part 1 variables, the CPESWTSH analysis weight should be used. It will include the full sample of NCS-R cases and provide greatest precision for sample estimates of population characteristics or relationships. If the analysis includes one or more variables that NCS-R collected only in Part 2, the appropriate weight for population estimation is CPESWTLG.

In the calculation of the Part 1 and Part 2 weights, the absolute contributions from NSAL and NLAAS to the pooled weight calculation remained unchanged--only NCS-R required changes to the nominal case counts and initial rescaling steps. However, due to the reduced NCS-R sample size for the Part 2 variables, the relative contributions of the NSAL and NLAAS to any give race/ethnicity x sample domain weighting cell did change. Therefore the final CPES Part 1 and Part 2 analysis weights (Step 8 above) differ for NSAL and NLAAS cases as they do for the NCS-R cases.

VI.B.2 Subsetting the CPES data by study

Occasionally, analysts may choose to extract CPES data for only one or two of the three component data sets. The CPES analysis weights will support this type of analysis; however, analysts should recognize that the sum of weights for this special CPES subset may not sum to the population control for that population. For example, consider an analysis which only used Afro-Caribbean data from the NLAAS and NSAL. Since a small number of Afro-Caribbeans interviewed in the NCS-R would be excluded from this analysis, the sum of weights for the combined NSAL and NLAAS cases would no longer match the CPES population control total for the Afro-Caribbean race/ethnicity population. A principle of weighted analysis of data is that population estimates and sampling errors (except for estimates of totals) should be invariant to any linear scaling of the weights (multiplication or division by a constant). Under the procedures used to compute the CPES Part 1 and Part 2 weights, this assumption of linear scaling applies when the data for one or two studies are used independently or are compared.

VI.B.3 Subsetting the CPES data based on characteristics or respondents

In general, CPES analysts can apply the analysis weights for subpopulation analysis (e.g., estimation for women of Mexican-American ancestry). Provided all qualifying cases in the CPES data are included in the subpopulation analysis, the estimates would be unbiased and the sum of the CPES weights would be an unbiased estimator of the 2002 population count for that subset of the larger U.S. population. Experience has shown that due to the sheer numbers of observations and richness of the variable set, data sets such as the CPES generate interest in rare populations or populations for which the original samples were not optimal (e.g. women of Mexican-American ancestry living in the West Census Region and covered by a regional health maintenance organization (HMO) program). CPES analysts who have concerns about the appropriateness of the CPES for subpopulation analysis they are proposing to conduct are encouraged to consult a survey statistician.

VI.B.4 Item Missing Data for Analysis Variables

The original NCS-R, NSAL, and NLAAS analysis weights included adjustments for survey nonresponse. Through the process used to create the combined analysis weights, these adjustments for differential nonresponse are preserved in the CPES Part 1 and Part 2 weights. However, the CPES weights do not include adjustments for item missing data in the CPES data set. With a few special exceptions, most statistical software packages employ "case-wise" deletion as the means to address the problem of missing values for the variables. That is, any case with a missing value on one or more variables (e.g. fitting a multivariate logistic regression model) will cause the case to be dropped from the analysis. If the amount of such case-wise deletion is substantial, the unbiasedness of the weighted estimation may be compromised. Analysts are encouraged to use standard data checking techniques to establish the patterns of missing data in their analysis variables and assess the extent and impact of software-driven case-wise deletion on the integrity of their analysis. If the variables of interest have high rates of item missing data, analysts may consider consulting a survey statistician to consider remediation approaches such as stochastic imputation (Little and Rubin, 2002).

VII. Procedures for Sampling Error Estimation in Design-based Analysis of the CPES Data.

The CPES data set is the product of the merger of three probability samples of the U.S. population and therefore shares the primary stage sample stratification and clustering features of the component sample designs. The NCS-R, NSAL and NLAAS sample designs were very similar in their basic structure to the multi-stage designs used for major survey programs such as the U.S. Health Interview Survey (HIS), the National Survey of Family Growth (NSFG) or the other national scientific surveys. The survey literature refers to the these samples as complex designs, a loosely-used term meant to denote the fact that the sample incorporates special design features such as stratification, clustering and differential selection probabilities (i.e., weighting) that analysts must consider in computing sampling errors for sample estimates of descriptive statistics and model parameters. Standard programs in statistical analysis software packages assume simple random sampling (SRS) or independence of observations in computing standard errors for sample estimates. In general, the SRS assumption results in underestimation of variances of survey estimates of descriptive statistics and model parameters. Confidence intervals based on computed variances that assume independence of observations will be biased (generally too narrow) and design-based inferences will be affected accordingly. Likewise, test statistics (t, X², F) computed in complex survey data analysis using standard programs will tend to be biased upward and overstate the significance of tests of effects.

This section focuses on sampling error estimation and construction of confidence intervals for survey estimates of descriptive statistics such as means, proportions, ratios, and coefficients for linear and logistic regression models.

VII.A Sampling Error Computation Methods and Programs

Over the past 50 years, advances in survey sampling theory have guided the development of a number of methods for correctly estimating variances from complex sample data sets. Sampling error programs that implement these complex sample variance estimation methods are available to CPES data analysts. The two most common approaches (Rust, 1985) to the estimation of sampling error for complex sample data are through the use of a Taylor Series linearization of the estimator (and corresponding approximation to its variance) or through the use of resampling variance estimation procedures such as Balanced Repeated Replication (BRR) or Jackknife Repeated Replication (JRR).

VII.B Taylor Series linearization method:

When survey data are collected using a complex sample design with unequal size clusters, most statistics of interest will not be simple linear functions of the observed data. The linearization approach applies Taylor's method to derive an approximate form of the estimator that is linear in statistics for which variances and covariances can be directly and easily estimated. Stata Release 8 and 9, SAS V8.2/V9.0, SUDAAN Version 9, and the most recent releases of SPSS are commercially available statistical software packages that include procedures that apply the Taylor Series method to sampling error estimation and inference for complex sample data.

Stata (StataCorp, 2005) is a more recent commercial entry to the available software for analysis of complex sample survey data and has a growing body of research users. Stata includes special versions of its standard analysis routines that are designed for the analysis of complex sample survey data. Special survey analysis programs are available for descriptive estimation of means (SVY MEAN), ratios (SVY RATIO), proportions (SVY TAB), and population totals (SVY TOTAL). STATA programs for multivariate analysis of survey data include linear regression (SVY REGRESS), logistic regression (SVY LOGIT) and probit regression (SVY PROBT). STATA program offerings for survey data analysts are constantly being expanded. Information on the STATA analysis software system can be found on the Web at: http://www.stata.com.

Programs in SAS Version 9 (SAS, 2003; http://www.sas.com/) also use the Taylor Series method to estimate variances of means (PROC Surveymeans), proportions and cross-tabular analysis (PROC SurveyFreq), linear regression (PROC SurveyReg), and logistic regression (PROC SurveyLogistic).

SUDAAN (RTI, 2004) is a commercially available software system developed and marketed by the Research Triangle Institute of Research Triangle Park, North Carolina (USA). SUDAAN was developed as a stand-alone software system with capabilities for the more important methods for descriptive and multivariate analysis of survey data, including: estimation and inference for means, proportions, and rates (PROC DESCRIPT and PROC RATIO); contingency table analysis (PROC CROSSTAB); linear regression (PROC REGRESS); logistic regression (PROC LOGISTIC); log-linear models (PROC CATAN); and survival analysis (PROC SURVIVAL). SUDAAN V9.0 and earlier versions were designed to read directly from ASCII and SAS system data sets. The latest versions of SUDAAN permit procedures to be called directly from the SAS system. Information on SUDAAN is available at the following Web site address: http://www.rti.org/.

SPSS Version 14.0 (http:// www.spss.com/) users can obtain the SPSS Complex Samples module which supports Taylor Series linearization estimation of sampling errors for descriptive statistics (CSDESCRIPTIVES), cross-tabulated data (CSTABULATE), general linear models (CSGLM), and logistic regression (CSLOGISTIC).

VII.C Resampling methods:

BRR, JRR, and the bootstrap comprise a second class of nonparametric methods for conducting estimation and inference from complex sample data. As suggested by the generic label for this class of methods, BRR, JRR, and the bootstrap utilize replicated subsampling of the sample database to develop sampling variance estimates for linear and nonlinear statistics. WesVar PC (Westat, Inc., 2000) is a software system for personal computers that employs replicated variance estimation methods to conduct the more common types of statistical analysis of complex sample survey data. WesVar PC was developed by Westat, Inc. and is distributed along with documentation to researchers at Westat's Web site: http://www.westat.com/wesvarpc/ . WesVar PC includes a Windows-based application generator that enables the analyst to select the form of data input (SAS data file, SPSS for Windows data base, ASCII data set) and the computation method (BRR or JRR methods). Analysis programs contained in WesVar PC provide the capability for basic descriptive (means, proportions, totals, cross tabulations) and regression (linear, logistic) analysis of complex sample survey data. WesVar also provides the best facility for estimating quantiles of continuous variables (e.g., 95%-tile of a cognitive test score) from survey data. WesVar Complex Samples 4.0 is the latest version of WesVar. Researchers who wish to analyze the CPES data using WesVar PC should choose the BRR or JRR (JK2) replication option.

STATA V9 has introduced the option to use JRR or BRR calculation methods as an alternative to the Taylor Series method for all of its svy command options. SUDAAN V9.0 also allows the analysts to select the JRR method for computing sampling variances of survey estimates.

IVEWare is another software option for the JRR estimation of sampling errors for survey statistics. IVEWare has been developed by the Survey Methodology Program of the Survey Research Center and is available free of charge to users at: http://www.isr.umich.edu/src/smp/ive/ . IVEWare is based on SAS Macros and requires SAS Version 6.12 or higher. The system includes programs for multiple imputation of item missing data as well as programs for variance estimation in descriptive (means, proportions) and multivariate (regression, logistic regression, survival analysis) analysis of complex sample survey data.

These new and updated software packages include an expanded set of user-friendly, well-documented analysis procedures. Difficulties with sample design specification, data preparation, and data input in the earlier generations of survey analysis software created a barrier to use by analysts who were not survey design specialists. The new software enables the user to input data and output results in a variety of common formats, and the latest versions accommodate direct input of data files from the major analysis software systems.

VII.D Sampling Error Computation Models

Regardless of whether the linearization method or a resampling approach is used, estimation of variances for complex sample survey estimates requires the specification of a sampling error computation model. CPES data analysts who are interested in performing sampling error computations should be aware that the estimation programs identified in the preceding section assume a specific sampling error computation model and will require special sampling error codes. Individual records in the analysis data set must be assigned sampling error codes that identify to the programs the complex structure of the sample (stratification, clustering) and are compatible with the computation algorithms of the various programs. To facilitate the computation of sampling error for statistics based on CPES data, design-specific sampling error codes will be routinely included in all versions of the data set. Although minor recoding may be required to conform to the input requirements of the individual programs, the sampling error codes that are provided should enable analysts to conduct either Taylor Series or Replicated estimation of sampling errors for survey statistics. In programs that use the Taylor Series Linearization method, the sampling error codes (SESTRAT and SECLUSTR) will typically be input as keyword statements (SAS V9.1, SUDAAN V9.0) or as global settings (Stata V9) along with the analysis weight and will be used directly in the computational algorithms. Programs that permit BRR or JRR computations will require the user supplied sampling error codes to construct "replicate weights" that are required for these approaches to variance estimation.

Two sampling error code variables are defined for each case based on the sample design stratum and primary stage unit (PSU) cluster in which the sample respondent resided: Sampling Error Stratum Code (SESTRAT) and Sampling Error Cluster Code (SECLUSTR). The CPES SESTRAT codes were derived directly from a concatenation of the existing sampling error stratum codes for the NCS-R, NSAL and NLAAS sample designs. A total of 180 sampling error strata were defined. These were allocated to the individual contributing samples according to the coding scheme shown in Table 6.

All original sampling error strata definitions for the NCS-R and NLAAS were preserved unchanged in the mapping to the CPES sampling error stratum code. In general, the assignment of NSAL cases to CPES sampling error strata also followed the original NSAL coding. The single exception involved a NSAL sampling error stratum that included multiple clusters. This stratum was divided into several pseudo-strata each with a pair of combined clusters. This minor change enables CPES analysts to use any of the sampling error calculation methods (Taylor, BRR or JRR) without having to perform additional recoding of the sampling error variables.

Likewise, with one exception, the values of SECLUSTR for CPES sampling error strata are identical to those in the original NCS-R, NSAL and NLAAS data sets. The exception was the cluster numbering for the one NSAL sampling error stratum with multiple clusters. Clusters in this stratum were randomly grouped into pairs and assigned to pseudo-strata as described in the preceding paragraph. The result is that the CPES SECLUSTR code takes a value of either 1 or 2 and exactly two sampling error clusters are assigned to each sampling error stratum.

VII.E Syntax for CPES Design-based Variance Estimation Using STATA and SAS

The following two sections provide a short overview of the general syntax and command file structure for computing sampling errors using STATA and SAS programs that have been designed for the analysis of complex sample survey data. Analysts are referred to the user guides and the on-line help facilities of these two software systems for documentation of the individual programs.

VII.E.1 Stata command syntax

As described above, CPES data analysts who are familiar with the STATA software system can utilize STATA's "svy" commands for the analysis of complex sample survey data. STATA Version 9 syntax for some of the more commonly used analysis programs is illustrated below (shown for the Part 2 weight option) :

This statement defines the sample design variables for the duration of the analysis session. SVY commands issued after this statement will automatically incorporate these design specifications.

To conduct analyses, the following STATA commands and syntax are used (please refer to STATA V9 Reference Manual for specific command syntax and output options):

[estimates, standard errors, design effects for means]

[estimates, standard errors for proportions of single variable categories, or crosstabulations of two variables with tests of independence]

[simple linear regression model for a continuous dependent variable]

[simple logistic regression model for a binary dependent variable]

To estimate the single statistics or regression models for subpopulations of the survey population in STATA, the following optional syntax is used (illustrated for svytab):

where var is a categorical variable that defines the subpopulations for which separate estimates are desired (e.g. gender).

VII.E.2 SAS Version 9 Command Syntax

SAS Version 9 includes four programs for the analysis of complex sample survey data: PROCS Surveymeans, SurveyFreq, SurveyReg and SurveyLogistic. The general syntax for specifying the CPES design structure in the SAS system is as follows:

Users are referred to the SAS/STAT(R) 9.1 User's Guide (SAS, 2004) for documentation on program specific statements, keywords and options

VIII. Weights for Study pairs

Final weights were also developed for analyzing pairs of the CPES studies (NCS-R and NSAL, NCS-R and NLAAS, and NSAL and NLAAS) and for Part I and Part II sub-samples (only for pairs that include NCS-R respondents). This will allow for generating population estimates by analyzing data from study pairs only. Table 7 below summarizes the paired weights and key descriptive statistics for the weight distributions.

a. Short and long sub-samples don't not apply as NCS-R sample is not included.

Weights for analysis of CPES study pairs are based on the final CPES 3-study weights and were developed according to the following steps:

Step 1. Each NLAAS, NSAL and NCS-R case was assigned to a race/ancestry category based on the categories and priority order provided in Table 1 (see Section III).

Step 2. Each NLAAS, NSAL and NCS-R area segment was assigned to a geographic domain based on the definitions and priority order shown in Table 2 (see Section IV). Each NLAAS, NSAL and NCS-R respondent was assigned to a geographic domain based on its area segment classification.

Step 3. For each pair of studies, race x domain cell counts were obtained. Due to the lack of over-sampling of certain race groups in specific pairs (Asians in NCS-R and NSAL) some of the CPES race x domain cells had no cases or a small number of cases. Such small cell counts could affect the robustness of post-adjustments and are usually dealt with by collapsing. Collapsing was mainly done over similar race groups (e.g., Vietnamese, Filipino, Chinese and other Asian groups) within a domain. Similar domains were then collapsed (examples of collapsed domains include Census Block Group > 5% Cuban and include Census Block Group > 5% Puerto Rican) if the cell count was still small (mainly <10) after race group collapsing. The same collapsed groups were used for the long and the short sub samples (whenever applicable). Collapsed groups for each pair of studies are shown in Tables 8-12.

Step 4. CPS 2002 totals were calculated for each collapsed group. Weighted counts using the final CPES weight (short form when dealing with short sample and long form when dealing with long sub-sample) were also generated for each collapsed group.

Step 5. A post-stratification adjustment factor (CPS 2002 total divided by the weighted count using CPES final weight) was calculated and applied to the final CPES weight to generate the paired weights. Respondents' weights for cases belonging to the same race x domain received the same factor.

IX. CPES Weights Chart

To better understand weights in CPES, please consult our CPES Weights Chart and take a look at our FAQs on weights and complex design.

X. References and Readings

Alegria M, Takeuchi D, Canino G, Duan N, Shrout P, Meng X, Vega W, Zane N, Vila D, Woo M, Vera M, Guarnaccia P, Aguilar-Gaxiola S, Sue S, Escobar J, Lin K, Gong F. Considering context, space and culture: the National Latino and Asian American Study. IJMPR 2004; 13(4): 208-20.

Alegria M, Vila D, Woo M, Canino G, Takeuchi D, Vera M, Febo V, Guarnaccia P, Aguilar-Gaxiola S, Shrout P. Cultural relevance and equivalence in the NLAAS instrument: integrating etic and emic in the development of cross-cultural measures for a psychiatric epidemiology and services study of Latinos. IJMPR 2004; 13(4): 270-88.

Alegria, M., Takeuchi, D.T., Canino, G.,Duan, N.,Shrout, P.E., Vega, W., Zave, N., Guarnaccia, P., Aguilar-Gaxiola, Ver, M., Sue, S., Escobar, J., Lin, Keh-Ming, Jang, M. amd Gong, F. (2004). " Considering Context, Space and Culture: The National Latino and Asian American Study". International Journal of Methods in Psychiatric Research, Vol. 13, No.2, pp. 208-220.

American Association for Public Opinion Research. Standard Definitions: Final Dispositions of Case Codes and Outcome Rates for Surveys. American Association for Public Opinion Research (AAPOR) standard: http://www.aapor.org , 2004.

Atrostic BK, Bates N, Burt G, Silberstein A. Non-response in US government household surveys: consistent measures, recent trends, and new insights. Journal of Official Statistics 2001; 17: 209-26.

Blaise(r) Survey Processing System: Version 4.5. Statistics Netherlands, 2000-2004.

Bogen K. The effect of questionnaire length on response rates - a review of the literature. Proceedings of the Section on Survey Research Methods, American Statistical Association 1996: 1020-5.

Bradburn NM. Respondent burden. Proceedings of the Section on Survey Research Methods. American Statistical Association 1978: 35-40.

Cannell C, Marquis K, Laurent A. A summary of studies. Vital Health Stat 1977; 2: 69.

Chambers, R.L. and Skinner C.J. (editiors). (2003). Analysis of Survey Data. JohnWiley and Sons, New York.

Cheung GQ, Liu Y. Displaying Chinese characters in Blaise. Proceedings of the Eighth International Blaise Users Conference 2003, Copenhagen, Denmark.

Cochran WG. Sampling Techniques, 3 edn. New York: John Wiley & Sons, 1977.

Cochran, W.G. (1977). Sampling Techniques. New York: John Wiley & Sons.

Corbin J, Morse JM. The unstructured interactive interview: issues of reciprocity and risks when dealing with sensitive topics. Qualitative Inquiry 2003; 9(3): 335-54.

de Leeuw E, de Heer W. Trends in household survey nonresponse: a longitudinal and international comparison. In R Groves, DA Dillman, JL Eltinge and RJA Little (eds) Survey Non-response. New York: Wiley, 2002, pp. 41-54.

Groeneveld R. Using non-Latin alphabets in Blaise. Proceedings of the Eighth International Blaise Users Conference 2003. Copenhagen, Denmark.

Groves RM, Couper MP. Non-response in Household Interview Surveys. New York: Wiley, 1998.

Groves RM, Fowler FJ, Couper MP, Lepkowski J, Singer E, Tourangeau R. Survey Methodology. New York: Wiley, 2004.

Groves RM. Survey Errors and Survey Costs. New York: John Wiley & Sons, 1989.

Guenzel PJ, Berckmans TR, Cannell CF. General Interviewing Techniques: A Self-Instructional Workbook for Telephone and Personal Interviewer Training. Ann Arbor, Michigan: Institute for Social Research, Survey Research Center, 1983.

Hansen MH, Hurwitz WN, Madow WG. Sample Survey Methods and Theory, Volumes I and II. New York: John Wiley & Sons, 1953.

Hansen MH, Hurwitz WN. The problem of nonresponse in surveys. Journal of the American Statistical Association 1946; 41: 517-29.

Hartley, H.O. (1962). "Multiple frame surveys." Proceedings of the Socials Science Section of the American Statistical Association Meeting, Minneapolis, Minnesota.

Hartley, H.O. (1974). "Multiple Frame Methodology and Selected Applications." Sankhya, Series C, 3, pp. 99-118.

Heeringa SG, Connor J, Darrah D. The 1980 SRC/NORC National Sample. Ann Arbor: Survey Methodology Program, Survey Research Center, University of Michigan, 1984.

Heeringa SG, Connor J, Redmond G. The 1990 SRC National Sample. Ann Arbor: Survey Methodology Program, Survey Research Center, University of Michigan, 1994.

Heeringa SG, Groves RM. Responsive Design for Household Surveys. Ann Arbor: Survey Methodology Program, Institute for Social Research, University of Michigan, 2004.

Heeringa SG, Liu J. Complex sample design effects and inference for mental health survey data. IJMPR 1997; 7(1): 56-65.

Heeringa SG, Wagner J, Torres M, Duan N, Adams T, Berglund P. Sample designs and sampling methods for the Collaborative Psychiatric Epidemiology Studies (CPES) IJMPR 2004; 13(4): 221-40.

Heeringa, S. , (2006). Technical Sample Design Documentation: National Study of American Life (NSAL). Technical Report. Statistical Design Group, Institute for Social Research, University of Michigan, Ann Arbor.

Heeringa, S., (2004). Technical Sample Design Documentation: 2002-2003 National Latino and Asian American Study (NLAAS). Technical Report. Statistical Design Group, Institute for Social Research, University of Michigan, Ann Arbor.

Heeringa, S., Wagner, J., Torres, M.,Duan, N., Adams, T., Berglund, P. (2004). "Sample designs and sampling methods for the Collaborative Epidemiology Studies (CPES)", International Journal of Methods in Psychiatric Research, Vol. 13, No. 4, pp. 221-240.

Heeringa, S.G. and Liu, J. (1997). "Complex sample design effects and inference for mental health survey data." International Journal of Methods in Psychiatric Research, Volume 7, Number 1, 56-65.

Henderson AS, Jorm AF. Do mental health surveys disturb? Psychol Med 1990; 20: 721-4.

Hess I. Sampling for social research surveys: 1947-1980. Ann Arbor: Institute for Social Research, University of Michigan, 1985.

Jackson J. Life in Black America. London: Sage Publications, 1991.

Jackson JS, Torres M, Caldwell CH, Neighbors HW, Nesse R, Taylor RJ, Trierweiler, SJ, Williams DR. The National Survey of American Life: a study of racial, ethnic and cultural influences on mental disorders and mental health. IJMPR 2004; 13(4): 196-207.

Jorm AF, Henderson AS, Scott R, MacKinnon AJ, Korten AE, Christensen H. Do mental health surveys disturb? Further evidence. Psychol Med 1994; 24: 233-7.

Kalton, G. (1977), "Practical methods for estimating survey sampling errors," Bulletin of the International Statistical Institute, Vol 47, 3, pp. 495-514.

Kessler R, Berglund P, Chiu WT, Demler O, Heeringa S, Hiripi E, Jin R, Pennell B, Walters E, Zaslavsky A, Zheng H. The US National Comorbidity Survey Replication (NCS-R): an overview of design and field procedures. IJMPR 2004; 13(2): 69-92.

Kessler R. The National Comorbidity Study of the United States. International Review of Psychiatry, 1994; 6: 365-76.

Kessler RC, Üstün TB. The World Mental Health (WMH) survey initiative version of the World Health Organization Composite International Diagnostic Interview (CIDI). IJMPR 2004; 13(2): 93-121.

Kessler RC, Wittchen H-U, Abelson J, Zhao S. Methodological issues in assessing psychiatric disorders with self-reports. In AA Stone, JS Turkkan, CA Barchrach, JB Jobe, HS Kurtzman and VS Cain (eds) The Science of Self-Report: Implications for Research and Practice. Mawah NJ: Lawrence Erlbaum Associates, 200; 229-55.

Kessler, R., Berglund, P., Chiu, W.T., Demler, O., Heeringa, S.,Hiripi, E., Jin, R.,Pennell, B., Walters, E., Zaslavsky, A., Zheng, H. (2004). "The U.S. National Comorbidity Survey Replication (NCS-R): an overview of design and field procedures." International Journal of Methods in Psychiatric Research, Vol. 13, No.2, pp. 69-92.

Kish L. A procedure for the objective selection of the respondent within the household. Journal of the American Statistical Association 1949; 44: 380-7.

Kish L. Statistical Design for Research. New York: John Wiley & Sons, 1987.

Kish L. Survey Sampling. New York: John Wiley & Sons, 1965.

Kish, L. (1965), Survey Sampling. New York: John Wiley & Sons, Inc.

Lessler JT, Kalsbeek WD. Nonsampling Errors in Surveys. New York: John Wiley & Sons, 1992.

Little, R.J.A. and Rubin, D.B. (2003). Statistical Analysis with Missing Data, 2nd Edition, John Wiley and Sons, New York.

Pennell, B.P., Bowers, A. Carr. D., Chardoul, S. Cheung, G-Q, Dinkelmann, K, Gebler, N., Hansen, S.E, Pennell, S. Torres, M. (2004). "The Development and Implementation of the National Comorbidity Survey Replication, the National Survey of American Life, and the National Latino and Asian American Survey", International Journal of Methods in Psychiatric Research, Vol. 13, No. 4, pp. 241-269.

Rao, J.N.K & Wu, C.F.J. (1988.), "Resampling inference with complex sample data," Journal of the American Statistical Association, 83, pp. 231-239.

Research Triangle Institute (2003). SUDAAN User's Manual, Release 9.0. Research Triangle Park, NC: Research Triangle Institute.

Rust, K. (1985). "Variance estimation for complex estimators in sample surveys," Journal of Official Statistics, Vol. 1, No. 4.

SAS Institute, Inc. (2003). SAS/STAT(R) User's Guide, Version 9, Cary, NC: SAS Institute, Inc.

Sharp LM, Frankel J. Respondent burden: a test of some common assumptions. Public Opinion Quarterly 1983; 47: 36-53.

Singer E, Groves RM, Corning AD. Differential incentives: beliefs about practices, perceptions of equity, and effects on survey participation. Public Opinion Quarterly 1999; 63: 251-60.

Singer E, Van Hoewyk J, Gebler N, Raghunathan T, McGonagle K. The effect of incentives in interviewermediated surveys. Journal of Official Statistics 1999; 15: 217-30.

Skinner, C.J., Holt, D., & Smith, T.M.F. (1989). Analysis of Complex Surveys. New York: John Wiley & Sons.

STATA Corp. (2004). STATA Statistical Software: Release 9.0. College Station, TX: STATA Corporation.

Turnbull JE, McLeod JD, Callahan JM, Kessler RC. Who should ask? Ethical interviewing in psychiatric epidemiology studies. American Journal of Orthopsychiatry 1988; 58(2): 228-39.

Westat, Inc. (2000). WesVar 4.0 User's Guide. Rockville, MD: Westat, Inc.

Wittchen H-U. Reliability and validity studies of the WHD Composite International Diagnostic Interview (CICDI): a critical review. J Psychiatr Res 1994; 28(1): 57-84.

Wolter, K.M. (1985 ). Introduction to Variance Estimation. New York: Springer-Verlag.

World Health Organization. Composite International Diagnostic Interview, Version 1.0. Geneva, Switzerland: World Health Organization, 1990.

World Health Organization. Composite International Diagnostic Interview, Version 2.1. Geneva, Switzerland: World Health Organization, 1997.