Hudson Institute Files

1

HUDSON INSTITUTE FILES

DOCUMENTATION

/***The John D. and Catherine T. MacArthur Foundation encouraged the Hudson Institute to study the provision of affordable rental housing over the 28-year period from 1985 through 2013 and supported that research. A first report covering the 1985 through 2005 period, The Long-Term Dynamics of Affordable Rental Housing, was released in 2010. The final report,The Affordable Housing Stock: A Longitudinal Analysis, 1985-2013,was released in 2017. Both reports relied on data from the American Housing Survey (AHS), which is sponsored by the U.S. Department of Housing and Urban Development (HUD) and conducted by the Census Bureau.

When the second report was completed, the authors prepared two files that others could use. This document describes the two files, explains how they were created, and defines the variables added in the course of the analysis.

The first Hudson Institute File, HUDSON_1, contains 95,444 observations and 2,162 variables. HUDSON_1:

• merges all 15 AHS national surveys based on the sample drawn in 1985,
• extracts key AHS variables,
• appends relevant variables from HUD’s Housing Affordability Data System (HADS) and the Census Bureau’s Where Did They Go (WDTG) file[1],
• creates variables used in the merging and cleaning of the individual AHS files, and
• eliminates units from supplement samples that appear in only some AHS surveys.

The second Hudson Institute File, HUDSON_2, contains 65,540 observations and 3,278 variables. HUDSON_2 modifies HUDSON_1 by:

• eliminating
• cases absent from too many AHS surveys
• cases with too much missing information on affordability
• other defective cases
• using longitudinal information on assistance statues to identify assisted units and combining that information with HUD/Census matches to create an “assisted” variable,
• providing adjusted median incomes for identified metropolitan areas from HUD publications (prior to 2003, HUD relied on regression based estimates to augment the AHS information),
• providing adjusted median income estimates for 2007, 2009, 2011, and 2013 to make the median income series consistent across time, given HUD’s switch from the decennial census to the ACS in 2007,
• developing a new bedroom variable that reduces survey-to-survey variation in the count of bedrooms,
• allocating values to the key status variable with missing information on rental affordable so there are no “blanks,”
• creating weights for longitudinal analysis,
• creating numerous special variables for the MacArthur analysis.

The AHS sample was drawn in 1985; 66 percent of our units come from the original sample. The original sample was re-interviewed every two years through 2013 and HUD and the Census Bureau added new units to the AHS sample with each survey to represent units that had entered the housing stock through new construction or by other means. The remainder of our sample comes from these additions through 2013, the end year of our analysis and the last year of data collection for this AHS sample.

HADSis a set of files derived by HUD from the 1985 and later national American Housing Survey (AHS). This system categorizes housing units by affordability and households by income, with respect to the Adjusted Median Income, Fair Market Rent (FMR), and poverty income. More information about the AHS and HADS can be found at The Where Did They Go (WDTG) file contains historical information on all the AHS national cases. The Census website, which calls this file the casehistory file, contains documentation – see

The authors are making HUDSON_2 available for researchers who want to do longitudinal analysis of housing policy issues and who want to take advantage of some of the special features developed in the course of the MacArthur study. HUDSON_1 is designed for researchers who want to do things differently than we did but who want to avoid the merging and extracting work.

Here is the relevant history of the research. The Hudson Institute received a grant from the John D. and Catherine T. MacArthur Foundation to analyze the dynamics of affordable rental housing using the American Housing Surveys (AHS) from 1985 through 2005. The report from that research -- The Long-Term Dynamics of Affordable Rental Housing – can be found at Subsequently, the MacArthur Foundation expanded the grant to allow the Hudson Institute to extend the analysis beyond 2005 to 2013, the last year of data collection for the sample drawn in 1985.

The analysis required the creation of a file that linked information from all 15 AHS national surveys, eliminated irrelevant cases or cases with insufficient information, incorporated information from the HUD's Housing Affordability Data System (HADS) and from HUD's annual estimates of median family income, contained specially created variables to facilitate the analysis, and imputed values for missing key variables.

This Documentation is an annotated Word© version of the SAS© code used to create the Hudson Institute Files. The annotations explain the various steps in creating the files and define the new variables created for this analysis. The creation of HUDSON_1 ends on page 98. Users who want to modify what we did can modify our code and apply that modified code, beginning on page 99, to HUDSON_1.

Users should consult the AHS codebook, available on the Census Bureau’s website, the HADS documentation available on the HUD USERS website, and the WDTG documentation on the previously mentioned Census website for definitions ofthe AHS, HADS, and WDTG variables.

HOW TO FIND ANNOTATIONS IN THE DOCUMENTATIONS

Using the FIND feature of Word to search for “/***”, one can locate the numerous annotations in the Documentation. “IMPORTANT” identifies the parts we think are key to understanding how the file was created and how to use key variables.

Most of the analysis in Chapter 3 is based on the STATC variable and the CCOUNT0, …, CCOUNT8 variables are cross tabbed successively with other variables, such as, JCWCAT, to produce the unit-year analysis in Chapter 4.

HERE ARE IMPORTANT CONVENTION IN THE NAMING OF VARIABLES.

We rename each AHS and HADS variable to identify which survey it is drawn from. Generally, the name of a variable is in capital letters and, if the variable can take different values in different surveys, its name includes a prefix of the form INXX_ where XX are the last two digits of the survey from which the variable is drawn. For example, IN89_TENURE is the tenure of the unit in 1989 and IN07_BEDRMS is the number of bedrooms in the unit in 2007. We also apply the INXX_ convention to the variables we create if they can take different values in different surveys, e.g. IN95_STATC which identifies the status of a unit in 1995.

We attach the suffix _H to HADS variables, such as, IN03_ASSISTED_H which uses respondent answers to classify a unit as assisted or unassisted in 2003.

Four of the WDTG variables (NOINTXX, PWTXX, STATUSXX, TENUREXX) end in XX to identify survey year, e.g. PWT89 is the pure weight of a unit in 1989. CONTROL, PATTERN, YRINTRO, and YEARMC are the remaining WDTG variables; these four variables take values that do not vary between surveys.

The table on the following pages is a partial listing of the variables on the file. If a variable occurs in multiple surveys, we list it only once using XX instead of numbers, e.g., INXX_BEDRMS or PWTXX. The variables we created have no “labels.” The user will need to use the FIND function in Word to locate where we introduce this variable. Generally, the Documentation will provide a short discussion of that variable, followed by the code that created the variable. Variables with labels are either AHS variables, HADS variables (look for _H), or WDTG variables (look for XX at the end of the variable.

The Hudson Institute Files and the SAS© used to create the file can be downloaded at

HOW THE FILES WERE USED

The tables contained in The Long-Term Dynamics of Affordable Rental Housing were produced using the equivalent of HUDSON_2. Simple SAS© runs were used to produce Excel© output. The Excel© output was used to construct the report tables. As mentioned above, most of the analysis in Chapter 3 is based on the STATC variable and the CCOUNT0, …, CCOUNT8 variables are cross tabbed successive with other variables, such as, JCWCAT, to produce the unit-year analysis in Chapter 4.

IMPORTANT HINT: For the weighted analysis, we divided HIFWGT by 10 to prevent undesirable rounding, particularly for counts of the owner stock, and then multiplied the Excel© results by 10.

WE NAMED OUR LIBRARY “OUT”

1

***/

1

/*** The first part of the code starts with the linked file we created for the 1985-2005 analysis (AHS05) and adds in the 2007, 2009, 2011, and 2013 public use files (PUFs). We used the complete 2011 PUF, including the TYPEC relational table that contains key information on units that left the sample prior to 2011. We select from the 2011 PUF only those cases that are part of the national file (NATLFAG = 1). (The 2011 PUF differs from other PUFs; it combines the national AHS with metropolitan AHS surveys conducted in 1011.)

FIRST: THE LIBRARIES FOR THE INPUT SAS DATA SETS
The folder C:\AHS_SAS_DATA has the AHS , HADS, and HISTORY input data sets for 1985 through 2003 and this library is called OUT1
The folderC:\AHS_SAS_DATA\NEW_AHS_FILES\AHS05 has the AHS , HADS, and HISTORY input data set for 2005 and this library is called OUT5

The folderC:\AHS_SAS_DATA\NEW_AHS_FILES\AHS07 has the AHS , HADS, and HISTORY input data set for 2007 and this library is called OUT7
The folderC:\AHS_SAS_DATA\NEW_AHS_FILES\AHS09 has the AHS , HADS, and HISTORY input data set for 2009 and this library is called OUT9
The folder C:\HUDSON_2015\AHS_2011_PUF_V1_4 has the AHS , HADS, and HISTORY input data set for 2011 and this library is called OUT11
The folder C:\HUDSON_2015\AHS2013_NATIONAL_PUF_V1_2 has the AHS , HADS, and HISTORY input data set for 2013 and this library is called OUT13

We will eventually add (page 89) to the 2013 PUF the casehistory relational file, also called the Where Did They Go (WDTG) file, that contains historical information on all the national case. The Census website for the casehistory file contains documentation for the WDTG file.

We eliminated from both the 2011 and 2013 PUFs, the special supplemental samples of HUD-assisted housing (HUDSAMP=1). These important supplemental samples are not useful for longitudinal analysis because we have no information on their status prior to 2011. Throughout the creation of HUDSON INSTITUTE FILE, we require that each unit have a complete history from 1985 or its year of entry into the housing stock (if later than 1985) to 2013 (or its year of permanent lost to the housing stock in earlier than 2013). ***/

options pagesize=6000 linesize=110 compress=binary nocenter MPRINT MLOGIC NODATE NONUMBER;

libname OUT05 'C:\AHS_SAS_DATA\NEW_AHS_FILES\AHS05\';

libname OUT07 'C:\AHS_SAS_DATA\NEW_AHS_FILES\AHS07\';

libname OUT09 'C:\AHS_SAS_DATA\NEW_AHS_FILES\AHS09\';

libname OUT1 'C:\AHS_SAS_DATA\';

libname OUT11 'C:\HUDSON_2015\AHS_2011_PUF_V1_4\';

libname OUT13 'C:\HUDSON_2015\AHS2013_NATIONAL_PUF_V1_2\';

libname OUT 'C:\HUDSON_2016\';

ods HTML file="C:\HUDSON_2016\CONTENTS_HUDSON_1_DATA_FILE.XLS";

IF IN11_NATLFLAG = 1 ;RUN;

DATA TYPEC_2011; SET OUT11.renamed_2011_typec;

IF IN11_NATLFLAG = 1;RUN;

DATA AHS11; SET AHS2011 TYPEC_2011; IF NOT(IN11_HUDSAMP = "1"); RUN;

DATA AHS13_HUDADMIN; SET OUT.ahs2013;

IN13_HUDADMIN=HUDADMIN;RUN;

DATA AHS13_HUDADMIN; SET AHS13_HUDADMIN;

KEEP CONTROL IN13_HUDADMIN; RUN;

PROC SORT DATA=AHS13_HUDADMIN; BY CONTROL;

PROC SORT DATA=OUT13.RENAMED_AHS2013_HADS2013;BY CONTROL;RUN;

DATA OUT13.RENAMED_AHS2013_HADS2013;

MERGE OUT13.RENAMED_AHS2013_HADS2013 AHS13_HUDADMIN;BY CONTROL;RUN;

DATA AHS2013; SET OUT13.RENAMED_AHS2013_HADS2013;

IF NOT(IN13_HUDSAMP = '1');RUN;

/*** The following is a macro designed to change all character to their numerical equivalent. ***/

%macro fixchar(variab);

if &variab='B' then &variab='.B';

if &variab='D' then &variab='.D';

if &variab='R' then &variab='.R';

if &variab='_' then &variab='._';

if &variab=' B' then &variab='.B';

%mend;

/*** The next part of the code, which is quite long (pages 17-87), extracts variables and make weights and householder variables consistent, pre-1997 and 1997 and later.

Pre-97 weights were percents; 1997and later weights are numbers.

The pre-1997 householder variables are recoded to be

consistent with 1997 and later definitions. ***/

DATA AHS85; SET OUT1.AHS85;

IN85_PWT=IN85_PWT/100;

IN85_CONTROL=CONTROL;

IN85_RACE1=IN85_RACE;

IN85_SPAN1=IN85_SPAN;

IN85_SEX1=IN85_SEX;

IN85_AGE1=IN85_AGE;

KEEP

CONTROL

IN85_CONTROL IN85_NUNIT2

IN85_PWT IN85_NUNITS

IN85_WEIGHT IN85_PER IN85_IPov_H

IN85_AGE1 IN85_RACE1 IN85_FMR_H

IN85_AMTE IN85_REGION IN85_Burden_H

IN85_AMTF

IN85_AMTG IN85_ROOMS IN85_L50_H

IN85_AMTO IN85_L80_H

IN85_AMTI IN85_LMED_H

IN85_AMTT IN85_COSTMED_H

IN85_AMTW IN85_SEX1