1
You must memorize the information on pages2 through 9.
You do not need to memorize the other pages, but doing so would improve your performance in this course.
Deviating from what is written can introduce error. Therefore, you are safest memorizing verbatim.
Probabilities
EventSomething that either will or will not happen. Example: It will either rain or it will not rain today.
TrialThe opportunity for an event to occur or not occur. Example: Each day is a trial because each day it either will or will not rain.
Independent eventsTwo events are independent if the occurrence of one event does not alter the probability of the other event occurring. Example: It will either rain or it will not rain, and you will get either heads or tails when you flip a coin. Whether it is raining or not has no effect on the probability of getting heads or tails.
Dependent eventsTwo events are dependent if the occurrence of one event alters the probability of the other event occurring. Example: It will either rain or it will not rain, and you will either wear a coat or not. The probability of you wearing a coat is higher when it is raining and lower when it is not raining.
Mutually exclusiveA set of events is mutually exclusive when only one of the events can occur. Example: For the change in a stock price from one day to the next, the events “stock price rises” and “stock price falls” are mutually exclusive.
Jointly exhaustiveA set of events is jointly exhaustive when at least one of the events must occur. Example: For the change in a stock price from one day to the next, the events “stock price rises”, “stock price falls”, and “stock price doesn’t change” are jointly exclusive.
Marginal probabilityThe probability of a single event occurring.
Pr(A)
Complement probabilityThe probability of a single event not occurring.
Pr(A’) = 1 – Pr(A)
Joint probabilityThe probability of two or more events occurring.
Pr(A and B) = Pr(A) Pr(B) if A and B are independent events.
Pr(A and B) = Pr(A) + Pr(B) – Pr(A or B)
Disjoint probabilityThe probability of at least one of two or more events occurring.
Pr(A or B) = Pr(A) + Pr(B) – Pr(A and B)
Conditional probabilityThe probability of an event occurring given than another event has occurred.
Pr(A | B) = Pr(A and B) / Pr(B)
Unconditional probabilityThe probability of a conditional event occurring when we do not know whether or not the other event has occurred.
Pr(A) = Pr(A | B) Pr(B) + Pr(A | B’) Pr(B’)
Bayes’ TheoremA theorem used to reverse the conditionality of a probability.
Pr(B | A) = Pr(A | B) Pr(B) / Pr(A)
CombinationA set of objects. Example: A set of lottery numbers is a combination because the numbers matter but the order of the numbers does not.
PermutationA set of ordered objects. Example: A phone number is a permutation because both the numbers and the order of the numbers matter.
Randomness
Random variableA variable that can take on any of a set of values within a range.
Probability distributionThe set of probabilities of a random variable taking on each value within its range of possible values.
PopulationThe set of all possible observations of a random variable.
SampleA subset of all possible observations of a random variable.
ObservationAninstance of a random variable taking on a value.
Discrete random variableA random variable that can only take on certain values within a range. Example: A die is a discrete random variable because, although its range is 1 through 6, the die can only take on integer values over this range.
Continuous random variableA random variable that can take on any value within a range. Example: People’s heights are continuous random variables because a person’s height (when measured with sufficient accuracy) can be any of infinite number of values.
Statistics
MeanThe average of a set of numbers. Let xi be the ith observation in a set of N observations. The mean is , where
MedianThe middle value in an ordered set of numbers.
ModeIn a set of numbers, the value that occurs most frequently.
Standard deviationThe average amount by which a set of numbers deviates from its mean.Let xi be the ith observation in a set of N observations. The standard deviation is s, where
Standard errorThe standard deviation of an estimate.
VarianceThe standard deviation (or standard error) squared.
First QuartileThe value that separates the lowest 25% of a set of data from the rest. Similarly, there are second, third, and fourth quartiles.
First DecileThe value the separates the lowest 10% of a set of data from the rest. Similarly, there are second, third, etc. deciles.
RangeThe difference between the maximum and minimum values in a data set.
Population meanThe mean of all observations in a population.
(The population mean is usually represented as μ.)
Sample meanThe mean of all observations in a sample. The sample mean is an approximation of the population mean.
(The sample mean is usually represented with a vertical bar as .)
Population varianceThe variance of all observations in a population.
(The population variance is usually represented as σ2.)
Sample varianceThe variance of all observations in a sample. The sample variance is an approximation of the population variance.
(The sample variance is usually represented as s2.)
Hypotheses
Null hypothesisSomething that is assumed to be true.
Alternative hypothesisThat which is assumed to be true when the null hypothesis is false.
p-valueThe probability of erroneously rejecting the null hypothesis.
The probability of observing the data you observed when, in fact, the null hypothesis is true.
Significance levelA criterion for separating “small” p-values from “large” p-values.
Test statisticA statistic that can be compared to a probability distribution to find a p-value.
(A generic term for t-score, z-score, F-score, etc.)
Confidence intervalA range over which observations of a random variable are likely to occur or a range that is likely to contain a population measure. A confidence interval is: .
Critical valueA criterion for separating “small” areas from “large” areas within a probability distribution.
Goals of Econometric Analysis
Hypothesis testingDetermining whether there is adequate evidence to refute an assertion about the value of a parameter.
ForecastingEstimating the value that a dependent variable would take on under conditions that do not yet exist.
Sensitivity analysisEstimating the change in the value that a dependent variable that would accompany a change in the value of an independent variable.
Desirable Estimator Properties
UnbiasedCoefficient Estimate
Definition:
Interpretation:On average, the sample parameter estimate is equal to the population parameter.
Implications of absence …
…for hypothesis testing:If the direction of bias is unknown, hypothesis testing is not possible. If the direction of bias is known, a bound on the hypothesis test can be established.
…for forecasting:Possible.
…for sensitivity analysis:If the direction of bias is unknown, sensitivity analysis is not possible. If direction of bias is known, a bound on the sensitivity can be established.
Consistent Coefficient Estimate
Definition:
Interpretation:As the number of observations increases, the expected difference between the sample parameter estimate and the population parameter approaches zero.
Implications of absence…
…for hypothesis testing:Possible with large data sets.
…for forecasting:Possible with large and small data sets.
…for sensitivity analysis:Possible with large data sets.
Efficient Coefficient Estimate
Definition:
Interpretation:The sample standard error of the sample parameter estimate is the lowest attainable sample standard error (within the class of linear unbiased estimators).
Implications of absence…
…for hypothesis testing:Hypotheses rejected in the presence of inefficiency will be rejected in the presence of efficiency. Nothing is known about hypotheses not rejected in the presence of inefficiency.
…for forecasting:Possible, though the confidence interval for the forecast will be larger than if the parameter estimate(s) were efficient.
…for sensitivity analysis:Possible, though the confidence interval for the sensitivity will be larger than if the parameter estimate(s) were efficient.
Unbiased Standard Error Estimate
Definition:
Interpretation:On average, the sample standard error of the sample parameter estimate is equal to the population standard error of the sample parameter estimate.
Implications of absence…
…for hypothesis testing:Not possible.
…for forecasting:Possible, though it will not be possible to establish a confidence interval for the forecast.
…for sensitivity analysis:Possible, though it will not be possible to establish a confidence interval for the sensitivity.
Statistical Anomalies (by order of severity)
Minor Anomalies
Multicollinearity
Description:Two or more regressors are correlated.
Implication:OLS parameter estimates are inefficient.
Identification:Variance inflation factor.
Correction:Drop one or more of the highly correlated regressors.
Extraneous Regressor
Description:A regressor that does not belong in the regression has been included.
Implication:OLS parameter estimates are inefficient.
Identification:Improved when regressor is omitted.
Correction:Drop the regressor from the model.
Serial Correlation
Description:Error terms are correlated across time.
Implication:OLS parameter estimates are unbiased.
Standard errors of the OLS parameter estimates are biased.
Identification:Portmanteau Q-statistics for the autocorrelation and partial autocorrelation correlograms. Durbin-Watson statistic will identify AR(1) processes. Apparent serial correlation can be caused by omitted variable / misspecification anomaly.
Correction:Cochrane-Orcutt procedure.
Note:If the errors are serial correlated, the regressors are serially correlated, and the correlations are of the same sign, then standard errors of the parameter estimates will be biased toward zero.
Moderate Anomalies
Suppressor Variables
Description:Two independent variables are uncorrelated with the dependent variable, but are negatively correlated with each other.
Implication:Significances of suppressor variable parameter estimates are spurious.
Identification:Coefficient for each suppressor variable is insignificant when the other suppressor variable is dropped from the model.
Correction:Drop both suppressor variables from the model.
Heteroskedasticity
Description:Variance of the error term is not constant.
Implication:Standard errors of the OLS parameter estimates are biased and inconsistent. The degree of bias in the standard errors is typically small enough to have little effect on significance tests except in cases of extreme heteroskedasticity.
Identification:White test. Breusch-Pagan test.
Correction:Weighted least squares.
Omitted Regressor / Model Misspecification
Description:A regressor that belongs in the regression has been excluded.
Implication:OLS parameter estimates are biased and inconsistent.
Identification:Theory. Coefficient on omitted regressor is significant when regressor is included. Anomaly might exhibit itself as serial correlation, or as stochastic regressor (if the omitted regressor is correlated with an included regressor).
Correction:Include the regressor.
Regime Shift
Description:Parameters change values at some point in the data set.
Implication:OLS parameter estimates are biased and (possibly) inconsistent.
Identification:Chow breakpoint or Chow forecast test.
Correction:Include the appropriate dummy variable(s).
Non-Linearity in Regressors
Description:Dependent variable is a non-linear function of regressors.
Implication:OLS parameter estimates are biased and inconsistent.
Identification:Theory. Significant coefficient for the non-linear form of the regressor when both the linear and non-linear forms are included in the regression. Box-Cox transformation.
Correction:Include non-linear form of the regressor. Box-Cox transformation.
Non-Linearity in Parameters
Description:Dependent variable is a non-linear function of parameters.
Implication:OLS parameter estimates are biased and inconsistent.
Identification:Theory. Improved R2 when using non-linear least squares.
Correction:Non-linear least squares.
Stochastic Regressors / Measurement Error
Description:One or more regressors are measured with error.
Implication:OLS parameter estimate of the stochastic regressor is biased toward zero. Estimates of the other parameters may be biased up or down.
Identification:Theory.Hausman test. May be the result of an omitted regressor that is correlated with an included regressor.
Correction:Two-stage least squares estimation. (Use two-stage residual inclusion method if stage two, but not stage one,is non-linear.)
Major Anomalies
Non-Stationarity
Description:One or more variables have unit roots.
Implication:OLS parameter estimates are biased and inconsistent. Standard errors of OLS parameter estimates are biased toward zero.
Identification:Augmented Dickey-Fuller Test. Phillips-Perron Test. Informal identification: R2 exceeds the Durbin-Watson statistic; AR(1) coefficient close to or exceeds 1.
Correction:First difference, second difference, percentage change, log, cointegration.
Invalid Restrictions
Description:Restrictions placed on model parameters are inconsistent with the true model.
Implication:Parameter estimates are biased and inconsistent.
Identification:F-Test.
Correction:Alter the restrictions.
Binary Dependent Variable
Description:The dependent variable takes on only two values.
Implication:OLS parameter estimates are biased and inconsistent. Standard errors of the OLS estimates are biased and inconsistent.
Identification:Examine the dependent variable.
Correction:Use logit, probit, or tobit procedures.
Multinomial Nominal Dependent Variable
Description:The dependent variable takes on discrete values. The ordering of the values contains no information.
Implication:OLS parameter estimates are biased and inconsistent. Standard errors of the OLS estimates are biased and inconsistent.
Identification:Examine the dependent variable.
Correction:Use multinomial logit, probit, or tobit procedures.
Multinomial Ordered Dependent Variable
Description:The dependent variable takes on a small number of discrete values. The ordering of the values contains information, but the distances between values contain no information.
Implication:OLS parameter estimates are biased and inconsistent. Standard errors of the OLS estimates are biased and inconsistent.
Identification:Examine the dependent variable.
Correction:Use ordered logit, probit, or tobit procedures.
Censored Dependent Variable
Description:A phenomenon is detected and either (1) measured when within given bounds, or (2) not measured when outside given bounds. The dependent variable is the measure of the phenomenon.
Implication:OLS parameter estimates are biased and inconsistent.
Identification:Examine the conditions under which the dependent variable is measured.
Correction:Use maximum likelihood procedures for censored data.
Truncated Dependent Variable
Description:A phenomenon is detected and measured only when within given bounds. The dependent variable is the measure of the phenomenon.
Implication:OLS parameter estimates are biased and inconsistent. Standard errors of the OLS estimates are biased and inconsistent.
Identification:Examine the conditions under which the dependent variable is measured.
Correction:Use maximum likelihood procedures for truncated data.
Estimation Procedures Assumptions
Maximum Likelihood (ML)
Regressors are uncorrelated with the error term.
Errors are normally distributed with zero mean.
Generalized Method of Moments (GMM)
Dependent variable is a linear function of the model parameters.
Errors are normally distributed with zero mean.
Generalized Least Squares (GLS)
Dependent variable is a linear function of the model parameters.
Regressors are uncorrelated with the error term.
Errors are normally distributed with zero mean.
Two-Stage Least Squares / Instrumental Variables (2SLS / IV)
Errors have constant variance.
Errors are uncorrelated across observations.
Dependent variable is a linear function of the model parameters.
Errors are normally distributed with zero mean
Weighted Least Squares (WLS)
Errors are uncorrelated across observations.
Dependent variable is a linear function of the model parameters.
Regressors are uncorrelated with the error term.
Errors are normally distributed with zero mean.
Ordinary Least Squares (OLS)
Errors have constant variance.
Errors are uncorrelated across observations.
Dependent variable is a linear function of the model parameters.
Regressors are uncorrelated with the error term.
Errors are normally distributed with zero mean.
Simultaneous Equations
Multiple regression equations comprise a single set of models.
Regressors in some of the equations are dependent variables in other equations.
Errors have constant variance.
Errors are uncorrelated across observations.
Dependent variablesare linear functions of the model parameters.
Errors are normally distributed with zero mean.
Seemingly Unrelated Regression (SUR)
Multiple regression equations comprise a single set of models.
Errors have constant variance.
Errors are uncorrelated across observations within equations but correlated across equations.
Dependent variablesare linear functions of the model parameters.
Regressors are uncorrelated with the error term.
Errors are normally distributed with zero mean.
Logit/Probit/Tobit
Dependent variable is discrete.
Errors have constant variance.
Errors are uncorrelated across observations.
Regressors are uncorrelated with the error term.
Errors have zero mean.
Covariance Notation
An easier method for performing calculations with covariances (as compared to summation notation) is as follows. Let X, Y, W, and Z be random variables. Let α and β be constants.
Estimators