The Relationship Between Reliability (Rxx) and Validity (Rxy) Coefficients

The Relationship Between Reliability (Rxx) and Validity (Rxy) Coefficients

The Relationship Between Reliability (Rxx) and Validity (Rxy) Coefficients[1]

By

Billy J. McCroskey, Ph.D.

Given two independent predictors of the same criterion, the predictor with the higher reliability will also have the higher validity coefficient. Reliability coefficients tend to be greater than validity coefficients because reliability places an upper limit on validity. The rule for maximum possible validity due to the upper limit placed on it by reliability is: The correlation between the predictor and independent criterion can never be higher than the square root of the product of the reliability of the predictor and the reliability of the criterion. For purposes of understanding and comparing Reliability (Rxx) and Validity (Rxy) coefficients, the table below best illustrates their relationship in terms of equivalent ranges and magnitude.[2]

RELIABILITYVALIDITYVERBAL SCALE EQUIVALENTS

0.97-0.99+0.70-0.99+ Extremely High

0.94-0.960.60-0.69Very High

0.90-0.930.50-0.59High

0.80-0.890.40-0.49Moderate

0.70-0.790.30-0.39Low

0.00-0.690.00-0.29Does not meet Standard[3]

Maximum Possible Validity = SQRT[Rxx(Predictor) times Rxx(Criterion)]

Table 2: Correlational Analysis of Software Estimates of Transferability of Skills Relative to 115 IARP Vocational Expert Participants' Estimates of Transferability[4]

Spearman’s Rho

Software PackageN Predictive Validity Coefficientp-Value

MVQSTM MTSP 2003TM56 0.56 0.00

VolcanoTM 4.1 MTS-R[5]56 0.48 0.00

VolcanoTM 4.1 MTS-O[6]56 0.39 0.00

SkillTranTM63 0.27 0.03

Control31 0.00 0.00

OASYSTM62 -0.15 0.26

LifeStepTM26- 0.21 0.31

The Dunn, et. al. TSA study was based on an Injured Worker with typical back restrictions in Bexar County, Texas.

In a second TSA study based on 10Non-Injured Workers[7] completed at the same Fall 2003 IARP Conference, Grimley, Lowe, Williams and Hahn (2004) found:

Table 3: Pearson R Predictive Validity Coefficients

TSA System

Summary and Conclusions

In the two IARP TSA Validy Studies (See Tables 2 & 3 above), MVQS Job-Person Matching TSA Levels from MTSP 2003 and Volcano 4.1 were found to be valid relative to Vocational Expert Ratings at the Fall 2003 IARP Conference in San Antonio, TX. These scientific findings support the High Reliability and High to Extremely High Face, Content, Predictive and Construct Validity of MVQS 2003 and Volcano 4.1 products for use with both injured and non-injured workers. Results generalize to MVQS2005.02 and Volcano 5.02 programs.

MVQS 2005.02 Product Costs (Separately & Bundled Together)[9]

MVQS 2005.02 (McDOT, McPLOT and MTSP) Program Package $2,680.00

Volcano 2005.02 Program Package$1,415.00

Four 30-hour CEU Training Courses Package$ 660.00

All Program Packages Bundled together (Save $1,530.00) $3,225.00

Contact: Dr. Billy J. McCroskey

8209 Halifax Court North

Brooklyn Park, MN 55443-2538

Ph: 763-569-0680

Reference for the above Help Screen cite: Statistica 98 Edition

Page: 1

[1] For more on Reliability and Validity: See Cronbach, Lee J., Essentials of Psychological Testing, 2nd ed., Harper & Brothers, New York, 1960, p. 141. See also, Guilford, J. P., Fundamental Statistics, 4th edition, McGraw-Hill, New York, 1965, pp. 438-462. See also, StatSoft, Inc. (1998). Statistica. [a Computer Program]. Tulsa, OK: Author.

[2] McCroskey, B. J. & Hahn, S. J. (1998). The Vocational Quotient (VQ) as a Predictor of Earning Capacity: 1996-97 Criterion-Referenced Validity Follow-up Studies. Journal of Forensic Vocationology, 4(1), p. 26.

[3]Does not meet or exceed the Confidence Standard for "Scientific" Reasonable Vocational Expert Certainty.

[4] Dunn, P., Williams, J. & Bast S. (2004). A Comparison of Intuitive and Computer-Generated Estimates of Skill Transferability. A Predictive Validity For TSA Systems relative to Vocational Expert TSA Ratings at the Fall 2003 IARP Conference in San Antonio, Tx, Using Spearman’s Rho. (Unpublished Manuscript – Updated with MVQS MTSP 2003 and Volcano MTS-R Values).

[5] MVQS MTSP 2003 & MTS-R ( McCroskey Transferable Skills 100 point Ratio Scales) were data mined and added for comparison.

[6] MTS-O = McCroskey Transferable Skills Ordinal scale (the MTS-R ratio scale reduced to a 5-point ordinal scale).

[7] Grimley, Cynthia P., Lowe, Jan K., Williams, John M. & Hahn, S. J. (2004). Analysis of IARP 2003 Vocational Expert (VE) Criterion Sampling Distribution TSA Ratings Predicted by Volcano 4.1 Transferable Skills Analysis (TSA) Estimates. Unpublished Manuscript.

[8] Exerpted from the 2000 ABVE Study for comparison with IARP Study Results. See, Grimley, C. P., Williams, J. M., Hahn, S. J. & Dennis, K L. (2000). Scientific Prediction of Transferable Skills.Journal of Forensic Vocationology, 6(1), pp. 7-16.

[9] MVQS 2005.02 prices subject to change with or without prior notice. Call 763-569-0680 for the most current prices.