1
HOMEWORK EIGHT: INSTRUMENTATION
Topic: Instrumentation
Isabel Cabrera
EDCI-6300.61 Foundations of Research in Education
Dr. Alberto Jose Herrera
The University of Texas at Brownsville
March 10, 2012
Exercise on Topic 32: Measures of Reliability
1. Researchers need to use at least how many observers to determine inter observer reliability?
A researcher should at least have two observers independently observe in order to check the reliability.
2. When there are two quantitative scores per participant, researchers can compute what statistic to describe reliability?
When there are tow quantitative scores per participant the researcher can check on the degree of relationship by computing a correlation coefficient.
3. Do researchers usually measure at two different points in time to estimate inter observer reliability?
No, it is in the test-retest reliability when researchers measure at two different points in time.
4. Do researchers usually measure at two different points in time to estimate test-retest reliability?
Yes, when researchers measure at two different points in time, it is said that the researcher is using the test-retest reliability.
5. According to this topic, most published tests have reliability coefficients that are about how high?
Most published tests have reliability coefficients of .80 or higher.
6. According to this topic, serviceable reliability coefficients may be how low if researchers are examining group averages?
The serviceable reliability coefficients may be as low as .50 if researchers are examining group averages.
Exercise on Topic 33: Internal Consistency & Reliability
1. Which two methods for estimating reliability require two testing sessions?
In the test-retest reliability is a test that is administered twice and the parallel forms reliability is when two alternative forms of a test are administered.
2. Does the split-half method require “one” or “two” administrations of a test?
The researcher administers one test, but scores the items in the test as though they were two separate tests by scoring all the odd numbers as one test score and then using all the even numbers as another test score.
3. What is meant by an “odd- even split”?
The odd-even split is a process that results in two different scores per examinee by dividing the questions in half by using the even numbered problems as one score and the odd numbered problems as another score for the examinee.
4. If a split-half reliability coefficient equals 0.00, what does this indicate?
When the researcher correlates the two sets of scores, he/she is yielding what is known as a split-half reliability coefficient. It can range from 0.00, which indicates a complete absence of reliability.
5. What is the highest possible value for a split-half reliability coefficient?
1.00 is the highest possible value for a split-half reliability coefficient.
6. To obtain alpha, mathematical procedures are used to obtain the equivalent of what?
In order to obtain alpha, the mathematical procedures are used to obtain the equivalent of the average of all possible spit-half reliability.
7. Does alpha estimate the consistency of scores over time?
No, alpha measures consistency among the items within a test at a single point in time.
Exercise on Topic 34: Norm- & Criterion-Referenced Tests
1. A norm-referenced test is designed to facilitate a comparison of an individual’s performance with what?
A norm-referenced (NRT) test is designed to facilitate a comparison of an individual’s performance with that of a norm group.
2. Are norm groups always a national sample?
Norm groups are often national samples of examinees, but it also may be an entire local population.
3. What is the definition of a criterion-referenced test?
A CRT is a test designed to measure the extent to which individual examinees have met performance standards of specific criteria.
4. In which type of test are items answered correctly by about 50% of the participants favored in item selection?
In the NRT, items answered correctly by about 50% of the participants are favored in item selection.
5. In which type of test are items typically selected on the basis of content they cover without regard to items difficulty?
In the CRT, item difficulty typically is of little concern. Instead, they are used to determine the desired level of performance.
6. Which type of test should be used in research where the purpose is to describe specifically what examinees can and cannot do?
The criterion-referenced test is used to describe specifically what examinees know and can do or in other words performance level. In contrast, the norm-referenced tests are designed in comparison to the norm group.
Exercise on Topic 35: Measures of Optimum Performance
1. Which type of test is designed to predict achievement in general?
An intelligence test is designed to predict achievement in general.
2. An algebra prognosis test is an example of what type of test?
An algebra prognosis test is an example of an aptitude test because it is designed to predict achievement in algebra by measuring basic math skills that are used in algebra.
3. A test designed to measure how much students learn in a particular course in school is what type of test?
The achievement test is a measure of how much students learn in a particular course in school.
4. A test designed to predict success in learning a new set of skills is what type of test?
An aptitude test is designed to predict success in learning a new set of skills. For example, the Scholastic Aptitude Test (SAT) is a test to predict success in college.
5. A list of desirable characteristics of a product or performance is known as what?
A check list is a list of desirable characteristics of a product or performance, each of which is awarded on a point system or rating scale.
6. How can researchers increase the reliability of scoring essays, products, and performances?
Researchers can make sure that the scorers know specifically what characteristics of the essays, performances, or products they are to consider and how much weight to give to each characteristic in arriving at the the scores in order to increase reliability of the test.
7. According to this topic, are intelligence tests a good measure of innate ability?
No, intelligence tests are not a good measure of innate ability. At best, intelligence tests measure skills that have been acquired in some specific cultural milieu.
8. According to this topic, how valid are commercially published aptitude tests?
Commercially published aptitude tests have low to modest validity with coefficients of about .20 to .60.
Exercise on Topic 36: Measures of Typical Performance
1. Do researchers usually want participants to show their best when measuring personality traits?
No, when researchers measure personality traits, they want to determine participants’ typical levels of performance not to show their best.
2. In this topic, what is the main reason for administering personality measures anonymously?
The main reason for administering personality measures anonymously is to reduce social desirability in participants’ responses in order to increase the validity of the test.
3. What do researchers reduce by observing behavior unobtrusively?
Researchers want to observe behavior unobtrusively in order to reduce the influence of social desirability to increase the validity of personality measures.
4. Loosely structured stimuli are used in which type of personality measure?
Projective techniques provide loosely structured or ambiguous stimuli such as the ink blots to measure certain personality traits such as aggressiveness.
5. According to this topic, which type of personality measure is seldom used in personality research?
Projective techniques test is seldom used in personality research because they are time consuming, interpretations are usually in words, and their validity is highly suspect.
6. What is the range of choice in Likert-type scale?
In the Rensis Likert type scale the range is from “Strongly agree” to “Strongly disagree.” Points are rewarded to each of the statements.
7. Is content validity a relevant concern when assessing the validity of a Likert-type scale for measuring attitudes?
Yes, content validity is a relevant concert when assessing the validity of a Likert-type scale for measuring attitudes. The researcher must have several statements concerning attitudes in order to have an analysis to assure that the contents are comprehensive to contribute to its content validity.