Fields of Study: Algebra, Data Analysis and Probability, Representations

Psychological Testing

Testing is used for many different purposes within psychology—among them to evaluate intelligence, diagnose psychiatric illness, and identify aptitudes and interests. Although the results of testing are rarely used as the sole criterion to make a diagnosis or other decision about an individual they are often used in conjunction with information gained from other sources such as interviews and observations of behavior. There are many types of psychological tests but most share the goal of expressing an essentially unobservable quality such as intelligence or anxiety in terms of numbers. The numbers themselves are not meant to be taken literally: no one seriously believes that a person’s intelligence is equivalent to their IQ score, for instance. Instead the numbers are useful tools which to help evaluate a person’s situation: for instance, how does the intellectual development of one particular child relate to that of other children of his age? Of course, the results of psychological testing should be evaluated with the social context of the individual in mind and with full respect for human diversity.

Psychometrics

Psychometrics is a field of study which applies mathematical and statistical principles to devise new psychological tests and evaluate the properties of current tests. The two most common approaches to psychometrics today are classical test theory and item response theory (IRT).

Classical test theory is the older of the two approaches and the calculations required can be performed with a pencil and paper, although today computer software is often used. Classical test theory assumes that all measurements are imperfect and thus contain error: the goal is to evaluate the amount of error in a measurement and develop ways to minimize it. Any observed measurement (for instance a child’s score on an intelligence test) is made up of two components: true score and error. This may be written as an equation:

X = T + E

where X is the observed score, T is the true score (the score representing the child’s true intelligence) and E is the error component (resulting from imperfect testing). Classical test theory assumes that that error is random and thus will sometimes be positive (resulting in a higher observed score than true score) and sometimes negative (resulting in a lower observed score than true score) so that over an infinite number of testing occasions the mean of the observed scores will equal the true score. Although normally a test is administered only once to a given individual this is a useful model which facilitates evaluation of the reliability and validity of different tests.

Item-response theory (IRT) is a different approach to psychological testing which assumes that observed performance on any given test item can be explained by a latent (unobservable) trait or ability so that individuals may be evaluated in terms of the amount of that trait they contain and items may be evaluated in terms of the amount of the trait required to answer them positively. For an item on an intelligence test (intelligence being the latent trait), persons with higher intelligence should be more likely to answer the question correctly. The same principle applies to IRT-based tests evaluating other psychological characteristics: for instance if an item in a psychological screening test is meant to diagnose depression a person with more depressive symptoms should be more likely to answer it positively. IRT is a mathematically complex method of analysis which depends on the use of specialized computer software but has become a popular means to evaluate psychological tests as computers have become more affordable. Although the mathematical models of IRT differ from that of classical test theory, the goals are the same: to devise tests which measure characteristics of individuals with a minimum of error.

Reliability and Validity

Reliability refers to the consistency of a test score: if a test is reliable it will yield consistent results over time and without regard to the irrelevant conditions such as the person administering the test. Internal consistency is considered an aspect of reliability: it means that all the items in a test measure the same thing. Temporal reliability is also called test-retest reliability because it is typically evaluated by having groups of individuals take the same test on several occasions and seeing how their scores compare: some differences are expected due to random nature of the error component but there should be a strong relationship between the observed scores of individuals on multiple occasions.

Inter-rater reliability refers to the consistency of a test or scale regardless of who administers it. For instance, psychiatric conditions are often evaluated by having an observer rate an individual’s behavior using a scale, and the results for different observers evaluating the same individual at the same time should be similar. For instance, three psychologists using a scale to evaluate the same child for hyperactivity should reach similar conclusions. Both types of reliability are typically evaluated by correlating test results on different occasions (temporal) or the scores returned by different raters (inter-rater).

Internal consistency can be measured in several ways. The split-half method involves having a group of individuals take a test then splitting the items into two groups (for instance, odd numbered items in one group and even in the other) and calculating the correlation between the total scores of the two groups. Cronbach’s alpha (coefficient alpha) is a refinement of the split-half method: it is the mean of all possible split-half coefficients.

Validity refers to whether a test measures what it claims to be measuring. Three types of validity are typically discussed: content, predictive and construct. Content validity refers to whether the test includes a reasonable sample of the subject or quality (for instance, mathematical aptitude or quality of life) it is intended to measure and is usually established by having a panel of experts evaluate the test in relation to its purpose. Predictive validity means that test scores correlate highly with measures of similar outcomes in the future: for instance a test of mechanical aptitude should correlate with a new hire’s success working as an auto repairman. Construct validity refers to a pattern of correlations predicted by the theory behind the quantity being measured: the scores on a test should correlate highly with scores on other tests which measure similar qualities and less highly with those which measure different qualities.

SEE ALSO: Diagnostic Testing; Educational Testing; Intelligence Quotients.