38 minute read
Chapter 3: Differences, Consistency, and the Meaning of Test Scores
from Psychometrics: An Introduction 4th Edition R. Michael Furr ISBN 9781071824085, 1071824082. All Chapt
by ACADEMIAMILL
Test Bank
Multiple Choice
1. A group of respondents took a psychological test. In their distribution of test scores, the variance is 4.56. Which of the following is the most accurate statement about that value?
a. It means that the people’s scores on the test are likely connected to scores on another test b. It reflects a small amount of variability (people did not differ greatly from each other, in terms of their test scores) c. It reflects a large amount of variability (i.e., people do differ quite a lot from each other, in terms of their test scores) d. This value is possible, but it is difficult to know whether it reflects a large or small amount of variability.
Ans: D
Learning Objective: 3-3: Summarize the importance of central tendency, variability, and distributions in statistics.
Cognitive Domain: Analysis
Answer Location: Variability and Distribution of Scores
Difficulty Level: Medium a. exponential b. gamma c. normal d. chi-squared
2. What kind of distribution is usually considered valid in statistical procedures?
Ans: C
Learning Objective: 3-3: Summarize the importance of central tendency, variability, and distributions in statistics.
Cognitive Domain: Comprehension
Answer Location: Distribution Shapes and Normal Distributions
Difficulty Level: Medium a. exponential b. gamma c. normal d. chi-squared
3. What type of distribution is considered a theoretical ideal?
Ans: C
Learning Objective: 3-3: Summarize the importance of central tendency, variability, and distributions in statistics.
Cognitive Domain: Comprehension
Answer Location: Distribution Shapes and Normal Distributions
Difficulty Level: Medium a. 0 b. 1 c. -1 d. 2
4. What is the value of a perfectly symmetrical distribution?
Ans: A
Learning Objective: 3-3: Summarize the importance of central tendency, variability, and distributions in statistics.
Cognitive Domain: Comprehension
Answer Location: Distribution Shapes and Normal Distributions
Difficulty Level: Medium a. 0 b. 1 c. -1 d. 2
5. What is the most likely value of a negatively skewed distribution?
Ans: C
Learning Objective: 3-3: Summarize the importance of central tendency, variability, and distributions in statistics.
Cognitive Domain: Comprehension
Answer Location: Distribution Shapes and Normal Distributions
Difficulty Level: Medium a. People who score relatively high on one of the tests tend to score relatively high on the other test, and people who score relatively low on one of the tests tend to score relatively low on the other test. b. People who score relatively high on one of the tests tend to score relatively low on the other test. c. Scores on the first test have nothing to do with scores on the second test. d. This value is impossible, you cannot get a correlation -.39.
6. A group of respondents took two psychological tests, and the correlation between the two distributions of scores is -.39. Which of the following is true?
Ans: B
Learning Objective: 3-4: Describe how associations are made between variables.
Cognitive Domain: Analysis
Answer Location: Correlation
Difficulty Level: Medium a. 1.20 b. .00 c. .75 d. -.80
7. Which Pearson correlation coefficient realistically shows the strongest relationship between two variables?
Ans: D
Learning Objective: 3-4: Describe how associations are made between variables.
Cognitive Domain: Application
Answer Location: Correlation
Difficulty Level: Hard a. taller people tend to have higher self-esteem than do shorter people b. shorter people tend to have higher self-esteem than do taller people c. none of the students have high self-esteem d. a person with high self-esteem is equally likely to be tall or short
8 Say, in a sample of students, there is a zero correlation between height and selfesteem. This means that _________.
Ans: D
Learning Objective: 3-4: Describe how associations are made between variables.
Cognitive Domain: Analysis
Answer Location: Correlation
Difficulty Level: Medium a. r will be increased b. r will be decreased c. r will remain the same d. cannot be determined from the information given
9 You conducted a study that examines the correlation between IQ and income, and you find a value of r = 0.75. At the end of the study, you find out all the IQ scores were scored 10 points too high. What will the value of r be after you rescore the IQ data, subtracting 10 from each person’s score?
Ans: C
Learning Objective: 3-4: Describe how associations are made between variables.
Cognitive Domain: Analysis
Answer Location: Correlation
Difficulty Level: Medium a. SAT scores are higher among females than among males b. Both SAT scores and CGPA scores are higher (on average) among females than among males c. SAT scores are more strongly associated with CGPA among females than among males d. The distribution of SAT scores is more skewed among females than among males.
10 Imagine that we examine the predictive accuracy of SAT scores by correlating those scores with college GPA. Imagine we find that the correlation between SAT scores and CGPA is r = .40 for females and r = .20 for males. What is the most accurate interpretation of these findings?
Ans: C
Learning Objective: 3-4: Describe how associations are made between variables.
Cognitive Domain: Analysis
Answer Location: Correlation
Difficulty Level: Medium
11 The variance: a. is the sum of participants' scores divided by the number of scores b. reflects the degree to which participants' scores differ from each other c. is an index of skewness d. is inferior to the range as an index of central tendency
Ans: B
Learning Objective: 3-5: Compare and contrast variance and covariance in computing composite scores.
Cognitive Domain: Comprehension
Answer Location: Variance and Covariance for “Composite Variables”
Difficulty Level: Medium a. You are more neurotic than your friend. b. Your friend is more neurotic than you are. c. Both of you are above the accepted neurotic level. d. It is impossible to make that kind of distinction.
12. If you and your best friend are answering questions related to neuroticism. If you obtained a score of 24 on the scale and your friend obtained 85 on the scale, what could you interpret?
Ans: B
Learning Objective: 3-7: Describe how test scores are represented, including z scores, standard scores, and percentile ranks.
Cognitive Domain: Analysis
Answer Location: Interpreting Test Scores
Difficulty Level: Medium
13 A z score: a. is the inverse of the mean b. is a measure or error variance c. reflects how far a participant's score falls from the mean d. is a measure of the variance
Ans: C
Learning Objective: 3-7: Describe how test scores are represented, including z scores, standard scores, and percentile ranks.
Cognitive Domain: Comprehension
Answer Location: z Scores (Standard Scores)
Difficulty Level: Medium a. I have a high level of self-esteem. b. I have a low level of self-esteem. c. Nothing; such a score would be impossible. d. Nothing, because you have no interpretive frame of reference for scores on that questionnaire.
14. Imagine that I responded to a new self-esteem questionnaire, and my raw score was 153.25. Based on that information, what would you conclude about me?
Ans: D
Learning Objective: 3-7: Describe how test scores are represented, including z scores, standard scores, and percentile ranks.
Cognitive Domain: Analysis
Answer Location: z Scores (Standard Scores)
Difficulty Level: Medium a. My level of self-esteem is greater than the level of my knowledge of psychometrics. b. My level of self-esteem is less than the level of my knowledge about psychometrics. c. Nothing, because you don’t know whether the two measures are comparable in any way. d. Nothing; such a score would be impossible.
15. Now, imagine that I also responded to a questionnaire measuring “knowledge of psychometrics,” and my raw score is 36 on that test. Based on that information and my self-esteem score, what could you conclude about me?
Ans: C
Learning Objective: 3-7: Describe how test scores are represented, including z scores, standard scores, and percentile ranks.
Cognitive Domain: Analysis
Answer Location: z Scores (Standard Scores)
Difficulty Level: Medium a. My self-esteem is 50% of the total possible self-esteem as measured by the test b. My self-esteem score is one-half of a standard deviation above the mean score. c. Compared to the average person, my self-esteem score is relatively low. d. I score higher than 50% of the other people who have taken the test.
16 Say my z score (standard score) on a self-esteem test is .50. How can you interpret that value?
Ans: B
Learning Objective: 3-7: Describe how test scores are represented, including z scores, standard scores, and percentile ranks.
Cognitive Domain: Analysis
Answer Location: z Scores (Standard Scores)
Difficulty Level: Medium a. -1 b. .90 c. 1 d. -5
17. Imagine that your raw score on a Neuroticism scale was 45, where the average score was 50, with a standard deviation of 5. What is your z score (i.e., standard score)?
Ans: A
Learning Objective: 3-7: Describe how test scores are represented, including z scores, standard scores, and percentile ranks.
Cognitive Domain: Application
Answer Location: z Scores (Standard Scores)
Difficulty Level: Hard a. Your score is higher than 99% of your classmates’ scores. b. Your score is almost 1 standard deviation higher than the average class score. c. You earned 99% of the possible points on the test. d. Compared to the average person, your score is relatively low.
18. After taking this exam, you find out that your standard score (z score) is .99. What’s the correct interpretation for this?
Ans: B
Learning Objective: 3-7: Describe how test scores are represented, including z scores, standard scores, and percentile ranks.
Cognitive Domain: Analysis
Answer Location: z Scores (Standard Scores)
Difficulty Level: Medium a. The average 10-year-old reads better than the average 7-year-old. b. All 10-year-olds read better than all 7-year-olds. c. There are greater differences among the 10-year-olds’ reading abilities than among the 7-year-olds’ reading abilities d. There are fewer differences among the 10-year-olds’ reading abilities than among the 7-year-olds’ reading abilities.
19. A developmental psychologist studies the development of reading ability. He studies a sample of 7-year-olds and a sample of 10-year-olds. Not surprisingly, he finds that the 10-year-old group has a higher mean level of reading ability than the 7-year-old group. He also finds that the 10-year-old group has a larger standard deviation than the 7-yearold sample. What does the difference in standard deviations tell us?
Ans: C
Learning Objective: 3-7: Describe how test scores are represented, including z scores, standard scores, and percentile ranks.
Cognitive Domain: Analysis
Answer Location: Converted Standard Scores (Standardized Scores)
Difficulty Level: Medium a. My score is 60% of the total possible self-esteem as measured by the test b. My score is .60 of a standard deviation above the mean score. c. Compared to the average person, my self-esteem score is relatively low. d. I score higher than 60% of the other people who have taken the test.
20 Say my percentile rank on a “knowledge of psychometrics” test is .60. How can you interpret that value (mark all that are correct)?
Ans: D
Learning Objective: 3-7: Describe how test scores are represented, including z scores, standard scores, and percentile ranks.
Cognitive Domain: Analysis
Answer Location: Percentile Ranks
Difficulty Level: Medium a. You know 95% of the total possible level of “Knowledge of Psychological Testing.” b. Your score is .95 of a standard deviation above the mean. c. Compared to the average person, your score is relatively low. d. You score higher than 95% of the other people who have taken the quiz.
21 After taking this quiz, let’s say you learn that your percentile rank on the test is .95. How can you interpret this value?
Ans: D
Learning Objective: 3-7: Describe how test scores are represented, including z scores, standard scores, and percentile ranks.
Cognitive Domain: Analysis
Answer Location: Percentile Ranks
Difficulty Level: Medium a. normal distributions b. converted standard scores c. percentile ranks d. normalization transformations
22. In what procedure must test developers must make two assumptions: levels of the psychological attribute are normally distributed, and the actual test data that they obtained in their sample are imperfect reflections of the distribution of the construct itself?
Ans: D
Learning Objective: 3-7: Describe how test scores are represented, including z scores, standard scores, and percentile ranks.
Cognitive Domain: Comprehension
Answer Location: Normalized Scores
Difficulty Level: Medium
23. When creating a test, the test creator will sometimes administer the test to a large group of people to develop test norms. That group is called a a. reference sample b. variance group c. covariance group d. herd
Ans: A
Learning Objective: 3-8: Discuss how test developers “norm” tests to interpret the scores.
Cognitive Domain: Comprehension
Answer Location: Test Norms
Difficulty Level: Medium a. The test norms reflect the one score that would be expected from a normative (i.e., typical) respondent. b. The test norms reflect the one score that would be expected from a normal (i.e., non-pathological) respondent. c. Any new test-taker can be compared to those test norms as a frame of reference for interpreting her/his score. d. Test norms allow researchers to use a test in their studies, as they allow researchers to compute correlations between the test’s scores and other variables of interest.
24. What is the value of having test norms?
Ans: C
Learning Objective: 3-8: Discuss how test developers “norm” tests to interpret the scores.
Cognitive Domain: Comprehension
Answer Location: Test Norms
Difficulty Level: Medium a. probability sampling b. nonprobability sampling c. correlation coefficient d. normal distribution
25. What refers to the use of certain procedures that ensure a representative sample?
Ans: A
Learning Objective: 3-8: Discuss how test developers “norm” tests to interpret the scores.
Cognitive Domain: Knowledge
Answer Location: Representativeness of the Reference Sample
Difficulty Level: Easy True/False
1. The numerator of the variance is the “sum of squares.” Ans: T
Learning Objective: 3-3: Summarize the importance of central tendency, variability, and distributions in statistics.
Cognitive Domain: Comprehension
Answer Location: Variability
Difficulty Level: Medium
2. The variance is viewed as more intuitive than standard deviation. Ans: F
Learning Objective: 3-3: Summarize the importance of central tendency, variability, and distributions in statistics.
Cognitive Domain: Analysis
Answer Location: Variability
Difficulty Level: Medium
3. Standard deviation can never be less than 0, but variance can be less than 0.
Ans: F
Learning Objective: 3-3: Summarize the importance of central tendency, variability, and distributions in statistics.
Cognitive Domain: Analysis
Answer Location: Variability
Difficulty Level: Medium
4. When a person obtains a relatively high score on one variable and then again on another, the variables have a negative association.
Ans: F
Learning Objective: 3-4: Describe how associations are made between variables.
Cognitive Domain: Comprehension
Answer Location: Interpreting the Association Between Two Variables
Difficulty Level: Medium
5. A scatterplot represents the association between two variables in terms of an upward or downward trend of points.
Ans: T
Learning Objective: 3-4: Describe how associations are made between variables.
Cognitive Domain: Knowledge
Answer Location: Scatterplots: Visually Representing the Association Between Two Variables
Difficulty Level: Easy
6. The variance of a binary item is maximized when half the people provide a positively valenced response and the other have provided a negatively valenced response.
Ans: T
Learning Objective: 3-6: Identify what a binary item is and how tests based on binary items are scored.
Cognitive Domain: Comprehension
Answer Location: Binary Items
Difficulty Level: Medium
7 Z score = 0 is the mean.
Ans: T
Learning Objective: 3-7: Describe how test scores are represented, including z scores, standard scores, and percentile ranks.
Cognitive Domain: Comprehension
Answer Location: z Scores (Standard Scores)
Difficulty Level: Medium
8. Z score < 0 is above the mean.
Ans: F
Learning Objective: 3-7: Describe how test scores are represented, including z scores, standard scores, and percentile ranks.
Cognitive Domain: Comprehension
Answer Location: z Scores (Standard Scores)
Difficulty Level: Medium
9. Z score < 0 is below the mean.
Ans: T
Learning Objective: 3-7: Describe how test scores are represented, including z scores, standard scores, and percentile ranks.
Cognitive Domain: Comprehension
Answer Location: z Scores (Standard Scores)
Difficulty Level: Medium
10 Z score = 0 is the same as absolute value.
Ans: F
Learning Objective: 3-7: Describe how test scores are represented, including z scores, standard scores, and percentile ranks.
Cognitive Domain: Comprehension
Answer Location: z Scores (Standard Scores)
Difficulty Level: Medium
Short Answer
1 Which is more central to the point of psychological testing and assessment? 1) To detect psychological differences, or 2) to reduce an individual’s entire character to one number?
Ans: Answer may vary.
Learning Objective: 3-1: Describe the implicit and explicit nature of variability found in psychological measurement.
Cognitive Domain: Analysis
Answer Location: The Nature of Variability
Difficulty Level: Medium
2 What is the term for a set of scores (e.g. a set of test scores)?
Ans: Answer may vary.
Learning Objective: 3-2: Explain why individual differences are fundamental to psychological measurement.
Cognitive Domain: Knowledge
Answer Location: Importance of Individual Differences
Difficulty Level: Easy
3. What is the conceptual term meaning “differences”?
Ans: Answer may vary.
Learning Objective: 3-3: Summarize the importance of central tendency, variability, and distributions in statistics.
Cognitive Domain: Knowledge
Answer Location: Variability and Distribution of Scores
Difficulty Level: Easy
4. What is the term for a statistical value used to quantify the degree of differences?
Ans: Answer may vary.
Learning Objective: 3-3: Summarize the importance of central tendency, variability, and distributions in statistics.
Cognitive Domain: Knowledge
Answer Location: Variability and Distribution of Scores
Difficulty Level: Easy
5 What is a conceptual term for the typical score in a distribution?
Ans: Answer may vary.
Learning Objective: 3-3: Summarize the importance of central tendency, variability, and distributions in statistics.
Cognitive Domain: Knowledge
Answer Location: Central Tendency
Difficulty Level: Easy
6. What is a statistical value used to quantitatively represent the typical score in a distribution?
Ans: Answer may vary.
Learning Objective: 3-3: Summarize the importance of central tendency, variability, and distributions in statistics.
Cognitive Domain: Comprehension
Answer Location: Central Tendency
Difficulty Level: Medium
7. What are the two most commonly used indexes of variability in behavioral research lying at the heart of psychometric theory in particular?
Ans: Answer may vary.
Learning Objective: 3-3: Summarize the importance of central tendency, variability, and distributions in statistics.
Cognitive Domain: Comprehension
Answer Location: Variability
Difficulty Level: Medium
8 What is the conceptual term for the consistency between two distributions of scores?
Learning Objective: 3-4: Describe how associations are made between variables.
Ans: Answer may vary.
Cognitive Domain: Knowledge
Answer Location: Quantifying the Association or Consistency Between Distributions
Difficulty Level: Easy
9 What is the term for a statistical value used to quantify the consistency between two distributions of scores?
Ans: Answer may vary.
Learning Objective: 3-4: Describe how associations are made between variables.
Cognitive Domain: Knowledge
Answer Location: Covariance
Difficulty Level: Easy
Chapter 4: Test Dimensionality and Factor Analysis
Test Bank
Multiple Choice
1. What is another way to think about a dimension? That it is a _________.
a. test b. item c. trait d. response
Ans: C
Learning Objective: 4-1: Understand the different types of dimensionality tests and their implications for scoring and psychometric evaluation.
Cognitive Domain: Knowledge
Answer Location: Test Dimensionality
Difficulty Level: Easy a. a psychological dimension b. a response to a test item c. the assumption that the trait affects responses to test items d. the assumption that the item affects the trait
2. Consider the figure below. Which of the following is the best meaning for the circle?
Ans: A
Learning Objective: 4-1: Understand the different types of dimensionality tests and their implications for scoring and psychometric evaluation.
Cognitive Domain: Comprehension
Answer Location: Unidimensional Tests
Difficulty Level: Medium a. a psychological dimension b. a response to a test item c. the assumption that the trait affects responses to test items d. the assumption that the item affects the trait
3. Consider the figure below. Which of the following is the best meaning for the box?
Ans: B
Learning Objective: 4-1: Understand the different types of dimensionality tests and their implications for scoring and psychometric evaluation.
Cognitive Domain: Comprehension
Answer Location: Unidimensional Tests
Difficulty Level: Medium a. a psychological dimension b. a response to a test item c. the assumption that the trait affects responses to test items d. the assumption that the item affects the trait
4. Consider the figure below. Which of the following is the best meaning for the arrows’ directionality?
Ans: C
Learning Objective: 4-1: Understand the different types of dimensionality tests and their implications for scoring and psychometric evaluation.
Cognitive Domain: Comprehension
Answer Location: Unidimensional Tests
Difficulty Level: Medium a. unidimensional b. multidimensional with correlated dimensions c. multidimensional with uncorrelated dimensions d. hyperdimensional with redundant dimensions
5. Which kind of test has one and only one score?
Ans: A
Learning Objective: 4-1: Understand the different types of dimensionality tests and their implications for scoring and psychometric evaluation.
Cognitive Domain: Knowledge
Answer Location: Unidimensional Tests
Difficulty Level: Easy a. unidimensional b. multidimensional with correlated dimensions c. multidimensional with uncorrelated dimensions d. hyperdimensional with redundant dimensions
6. What kind of test (in terms of dimensionality) is best represented by the figure below?
Ans: B
Learning Objective: 4-1: Understand the different types of dimensionality tests and their implications for scoring and psychometric evaluation.
Cognitive Domain: Comprehension
Answer Location: Multidimensional Test With Correlated Dimensions (Tests With Higher-Order Factors)
Difficulty Level: Medium
7. When responses to a test’s items reflect more than one psychological dimension, the test is a. unidimensional b. hyperdimensional c. multidimensional d. psychologically invalid Ans: C
Learning Objective: 4-1: Understand the different types of dimensionality tests and their implications for scoring and psychometric evaluation.
Cognitive Domain: Comprehension
Answer Location: Multidimensional Tests With Correlated Dimensions (Tests With Higher-Order Factors)
Difficulty Level: Medium a. unidimensional b. multidimensional with correlated dimensions c. multidimensional with uncorrelated dimensions d. hyperdimensional with redundant dimensions
8. Recall the BFI (Big Five Inventory, with scores representing Neuroticism, Extraversion, Openness to Experience, etc.). What type of test was that? Recall that we computed 5 and only 5 scores.
Ans: C
Learning Objective: 4-1: Understand the different types of dimensionality tests and their implications for scoring and psychometric evaluation.
Cognitive Domain: Analysis
Answer Location: Multidimensional Tests With Uncorrelated Dimensions
Difficulty Level: Medium
9 EFA is most directly based on: a. correlations among items b. differences between groups of people c. a conceptual analysis of item meaning d. respondents’ comments about their view on test dimensionality
Ans: A
Learning Objective: 4-4: Describe factors, factor loadings, and rotations, and how they offer deeper insight into factor analysis.
Cognitive Domain: Comprehension
Answer Location: A Deeper Perspective on Factors, Factor Loadings, and Rotation
Difficulty Level: Medium
10. Imagine that you are examining a test, and you find that responses to one item are highly correlated with responses on another item. This finding indicates: a. The two items likely reflect the same factor. b. The two items likely reflect different factors. c. Responses to the first item are likely affected by the same psychological attribute that affects responses to the second item. d. There were different groups of people that answered each item
Ans: A
Learning Objective: 4-1: Understand the different types of dimensionality tests and their implications for scoring and psychometric evaluation.
Cognitive Domain: Application
Answer Location: Multidimensional Tests With Correlated Dimensions (Tests With Higher-Order Factors)
Difficulty Level: Hard a. unidimensional b. multidimensional with correlated dimensions c. multidimensional with uncorrelated dimensions d. hyperdimensional with redundant dimensions
11. Which kind of test is often scored in a way that produces a total score, combined across several subsets, which are often combined?
Ans: B
Learning Objective: 4-1: Understand the different types of dimensionality tests and their implications for scoring and psychometric evaluation.
Cognitive Domain: Comprehension
Answer Location: Multidimensional Tests With Correlated Dimensions (Tests With Higher-Order Factors)
Difficulty Level: Medium a. unidimensional b. multidimensional with correlated dimensions c. multidimensional with uncorrelated dimensions d. hyperdimensional with redundant dimensions
12. "Higher-order" and "second-order" structures are most likely associated with which kind of test?
Ans: B
Learning Objective: 4-1: Understand the different types of dimensionality tests and their implications for scoring and psychometric evaluation.
Cognitive Domain: Comprehension
Answer Location: Multidimensional Tests With Correlated Dimensions (Tests With Higher-Order Factors)
Difficulty Level: Medium a. For a multidimensional test with correlated dimensions, the subscales can be combined into a total scale score, but this is not true for a multidimensional test with uncorrelated dimensions. b. For a multidimensional test with uncorrelated dimensions, the subscales can be combined into a total scale score, but this is not true for a multidimensional test with correlated dimensions c. For a multidimensional test with correlated dimensions, each subscale should be examined separately from the others, but this is not true for a multidimensional test with uncorrelated dimensions. d. For a multidimensional test with uncorrelated dimensions, each subscale should be examined separately from the others; however, for a multi-dimensional test with correlated dimensions, the subscales should always be combined and not examined separately.
13 In terms of scoring a test, what is a key difference between the two kinds of multidimensional tests?
Ans: A
Learning Objective: 4-1: Understand the different types of dimensionality tests and their implications for scoring and psychometric evaluation.
Cognitive Domain: Analysis
Answer Location: Multidimensional Tests With Uncorrelated Dimensions
Difficulty Level: Medium a. unidimensional b. multidimensional with correlated dimensions c. multidimensional with uncorrelated dimensions d. hyperdimensional with redundant dimensions
14. Which kind of test is combined across several subsets, with dimensions that are not associated with each other (or only weakly associated)?
Ans: C
Learning Objective: 4-1: Understand the different types of dimensionality tests and their implications for scoring and psychometric evaluation
Cognitive Domain: Knowledge
Answer Location: Multidimensional Tests With Uncorrelated Dimensions
Difficulty Level: Easy a. test dimensionality b. factor loadings c. rotations d. scree plots
15 By identifying sets of items that go together, items that are relatively strongly correlated with each other but weakly correlated with other items, what are we able to understand?
Ans: A
Learning Objective: 4-2: Explain how test developers use factor analysis in order to address the core questions in the three dimensionality tests.
Cognitive Domain: Comprehension
Answer Location: Factor Analysis: Examining the Dimensionality of a Test
Difficulty Level: Medium a. scree plots b. cluster analysis c. multidimensional scaling d. factor analysis
16. Which of the following is the statistical tool most commonly used to test dimensionality?
Ans: D
Learning Objective: 4-2: Explain how test developers use factor analysis in order to address the core questions in the three dimensionality tests.
Cognitive Domain: Comprehension
Answer Location: Factor Analysis: Examining the Dimensionality of a Test
Difficulty Level: Medium a. scree plots b. cluster analysis c. multidimensional scaling d. factor analysis
17. "Exploratory" and "confirmatory" are two types of what way to examine the dimensionality of a test?
Ans: D
Learning Objective: 4-2: Explain how test developers use factor analysis in order to address the core questions in the three dimensionality tests.
Cognitive Domain: Comprehension
Answer Location: Factor Analysis: Examining the Dimensionality of a Test
Difficulty Level: Medium a. a psychological trait or dimension b. responses to an item c. clusters of participants d. something that two items do NOT have in common with each other
18. In an EFA, what does a “Factor” represent?
Ans: A
Learning Objective: 4-3: Summarize the process of conducting an exploratory factor analysis.
Cognitive Domain: Comprehension
Answer Location: Conducting and Interpreting an Exploratory Factor Analysis
Difficulty Level: Medium
19 When conducting an Exploratory Factor Analysis and attempting to understand the psychological meaning of each factor, researchers usually rely upon: a. tests of statistical significance b. reasoned judgment, based on factor loadings c. the “flattening-point” of a scree test d. inferring what non-loading tests share in common
Ans: B
Learning Objective: 4-3: Summarize the process of conducting an exploratory factor analysis.
Cognitive Domain: Analysis
Answer Location: Conducting and Interpreting an Exploratory Factor Analysis
Difficulty Level: Medium a. 7 b. 16 c. 3 d. 1
20 Imagine that you conducted an EFA of a 16-item test, and you obtained the scree plot below. Based on this, how many factors does the test seem to have?
Ans: C
Learning Objective: 4-3: Summarize the process of conducting an exploratory factor analysis.
Cognitive Domain: Application
Answer Location: Conducting and Interpreting an Exploratory Factor Analysis
Difficulty Level: Hard a. The on-factor loadings are too high (note, “on-factor” here means an item’s largest factor loading). b. The on-factor loadings are generally lower than we’d like. c. There’s a problem: none of the factor loadings are significant at p <.05. d. There’s likely a mistake an item loading on F1 should probably have a negative loading on that factor.
21 You examine the dimensionality of a nine-item test intended to assess relationship satisfaction. After doing an EFA, you get the following set of factor loadings. Which is the most appropriate statement below?
Ans: D
Learning Objective: 4-3: Summarize the process of conducting an exploratory factor analysis.
Cognitive Domain: Application
Answer Location: Conducting and Interpreting an Exploratory Factor Analysis
Difficulty Level: Hard a. varimax and promax b. quartimax and oblimin c. varimax and equamax d. promax and simplimax
22. Which of the following are oblique rotations?
Ans: D
Learning Objective: 4-3: Summarize the process of conducting an exploratory factor analysis.
Cognitive Domain: Application
Answer Location: Conducting and Interpreting an Exploratory Factor Analysis
Difficulty Level: Hard a. unidimensional b. multidimensional with correlated dimensions c. multidimensional with uncorrelated dimensions d. hyperdimensional with redundant dimensions
23 Imagine that you conduct an EFA of responses to a set of test items, and you find a two-factor structure, with an inter-factor correlation of .68. Based on these results, what would you conclude about the dimensionality of your test?
Ans: B
Learning Objective: 4-4: Describe factors, factor loadings, and rotations, and how they offer deeper insight into factor analysis.
Cognitive Domain: Analysis
Answer Location: Conducting and Interpreting an Exploratory Factor Analysis
Difficulty Level: Medium a. unidimensional b. multidimensional with correlated dimensions c. multidimensional with uncorrelated dimensions d. hyperdimensional with redundant dimensions
24. Imagine that you conduct an EFA of responses to a set of test items, and you find a two-factor structure, with an inter-factor correlation of .00. Based on these results, what would you conclude about the dimensionality of your test?
Ans: C
Learning Objective: 4-4: Describe factors, factor loadings, and rotations, and how they offer deeper insight into factor analysis.
Cognitive Domain: Analysis
Answer Location: Conducting and Interpreting an Exploratory Factor Analysis
Difficulty Level: Medium
25. The factors extracted in factor analysis are made more clear and interpretable by: a. converting raw scores to z scores before analysis b. adding variables that have low correlations with other variables c. applying a square root transformation to the raw data before analysis d. statistically rotating factors
Ans: D
Learning Objective: 4-4: Describe factors, factor loadings, and rotations, and how they offer deeper insight into factor analysis.
Cognitive Domain: Analysis
Answer Location: A Deeper Perspective on Factors, Factor Loadings, and Rotation
Difficulty Level: Medium
1. It is psychometrically problematic if a test includes some items that reflect one dimension and other items that reflect another dimension. For example, it’s problematic if a test includes some items reflecting depression and other items reflecting anxiety.
Ans: F
Learning Objective: 4-1: Understand the different types of dimensionality tests and their implications for scoring and psychometric evaluation.
Cognitive Domain: Analysis
Answer Location: Test Dimensionality
Difficulty Level: Medium
2 It is psychometrically ideal for each item (on a test) to reflect only one dimension, and problematic if an item is affected by more than one dimension. For example, it’s problematic if responses to Item 1 on a test are affected by both depression and anxiety.
Ans: T
Learning Objective: 4-1: Understand the different types of dimensionality tests and their implications for scoring and psychometric evaluation.
Cognitive Domain: Analysis
Answer Location: Test Dimensionality
Difficulty Level: Medium
3. No psychological test should produce more than a single score for each respondent. Ans: F
Learning Objective: 4-1: Understand the different types of dimensionality tests and their implications for scoring and psychometric evaluation.
Cognitive Domain: Comprehension
Answer Location: Test Dimensionality
Difficulty Level: Medium
4. A score in a psychological test should represent a single psychological feature or attribute.
Ans: T
Learning Objective: 4-1: Understand the different types of dimensionality tests and their implications for scoring and psychometric evaluation.
Cognitive Domain: Knowledge
Answer Location: Test Dimensionality
Difficulty Level: Easy
5. For a unidimensional test, we would have one score for each respondent.
Ans: T
Learning Objective: 4-1: Understand the different types of dimensionality tests and their implications for scoring and psychometric evaluation.
Cognitive Domain: Knowledge
Answer Location: Unidimensional Tests
Difficulty Level: Easy
6. When evaluating the psychometric quality of a test, we should evaluate the psychometric quality for each scale score on the test.
Ans: T
Learning Objective: 4-1: Understand the different types of dimensionality tests and their implications for scoring and psychometric evaluation.
Cognitive Domain: Comprehension
Answer Location: Multidimensional Tests With Correlated Dimensions (Tests With Higher-Order Factors)
Difficulty Level: Medium
7. For a multidimensional test, we would have one score for each respondent.
Ans: F
Learning Objective: 4-1: Understand the different types of dimensionality tests and their implications for scoring and psychometric evaluation.
Cognitive Domain: Knowledge
Answer Location: Multidimensional Tests With Correlated Dimensions (Tests With Higher-Order Factors)
Difficulty Level: Easy
8. It is theoretically possible for a test to produce some scale/dimension scores that have good psychometric quality and other scale/dimension scores that have poor psychometric quality.
Ans: T
Learning Objective: 4-1: Understand the different types of dimensionality tests and their implications for scoring and psychometric evaluation.
Cognitive Domain: Analysis
Answer Location: Multidimensional Tests With Correlated Dimensions (Tests With Higher-Order Factors)
Difficulty Level: Medium
9. Each subtest in a multidimensional test is, itself, unidimensional.
Ans: T
Learning Objective: 4-1: Understand the different types of dimensionality tests and their implications for scoring and psychometric evaluation.
Cognitive Domain: Comprehension
Answer Location: Answer Location: Multidimensional Tests With Correlated Dimensions (Tests With Higher-Order Factors)
Difficulty Level: Medium
10. When scoring a multidimensional test that has uncorrelated dimensions, each score should reflect only a single dimension.
Ans: T
Learning Objective: 4-1: Understand the different types of dimensionality tests and their implications for scoring and psychometric evaluation.
Cognitive Domain: Knowledge
Answer Location: Multidimensional Tests With Uncorrelated Dimensions
Difficulty Level: Easy
Short Answer
1. What is one of the three key questions regarding test dimensionality?
Ans: Answer may vary.
Learning Objective: 4-1: Understand the different types of dimensionality tests and their implications for scoring and psychometric evaluation.
Cognitive Domain: Application
Answer Location: Three Dimensionality Questions: What They Are and Why They Matter
Difficulty Level: Hard
2. What type of psychological test includes items that reflect only a single attribute of a person, a single psychological dimension?
Ans: Answer may vary.
Learning Objective: 4-1: Understand the different types of dimensionality tests and their implications for scoring and psychometric evaluation.
Cognitive Domain: Knowledge
Answer Location: Unidimensional Tests
Difficulty Level: Easy
3. What kind of test would you classify intelligence tests, like the Wechsler Intelligence Scale for Children?
Ans: Answer may vary.
Learning Objective: 4-1: Understand the different types of dimensionality tests and their implications for scoring and psychometric evaluation.
Cognitive Domain: Analysis
Answer Location: Answer Location: Multidimensional Tests With Correlated Dimensions (Tests With Higher-Order Factors)
Difficulty Level: Medium
4 Most traditional IQ tests are built upon the idea that there is an ability, called “g” or “general intelligence,” that affects people’s scores on a range of cognitive tasks. This is why those tests produce a single “IQ score” that we’re all familiar with
A few years ago, researchers conducted an EFA of scores on 12 cognitive tasks (N = 44,600 participants). They state that their analysis revealed a three-factor structure among these tasks. This apparent finding is clearly at odds with the traditional view that one single factor (i.e., “general intelligence”) accounts for much of the covariation among diverse sets of cognitive tasks. Based on this finding, the researchers concluded that the idea that there is a single general factor of intelligence is “a myth” and is simply incorrect.
This is a provocative claim if it’s true, then we should dramatically rethink our view of intelligence and intelligence testing. In an online supplement to the article, the researcher reported the 12 eigenvalues from their EFA. Using these, we can sketch out the following scree plot:
Looking at the scree plot, how many factors would you say there are?
Ans: Answer may vary.
Learning Objective: 4-3: Summarize the process of conducting an exploratory factor analysis.
Cognitive Domain: Analysis
Answer Location: Conducting and Interpreting an Exploratory Factor Analysis
Difficulty Level: Medium
5 Statistical values that typically range between -1 and +1 and reflect the degree/direction of connection between an item and a factor are best labeled as what term?
Ans: Answer may vary.
Learning Objective: 4-3: Summarize the process of conducting an exploratory factor analysis.
Cognitive Domain: Knowledge
Answer Location: Conducting and Interpreting an Exploratory Factor Analysis
Difficulty Level: Easy
6 Statistical values that are often used to understand the number of factors when conducting an EFA are best labeled as what term?
Ans: Answer may vary.
Learning Objective: 4-3: Summarize the process of conducting an exploratory factor analysis.
Cognitive Domain: Knowledge
Answer Location: Conducting and Interpreting an Exploratory Factor Analysis
Difficulty Level: Easy
7 A figure that presents the magnitudes of eigenvalues is best labeled as what term?
Ans: Answer may vary.
Learning Objective: 4-3: Summarize the process of conducting an exploratory factor analysis.
Cognitive Domain: Knowledge
Answer Location: Conducting and Interpreting an Exploratory Factor Analysis
Difficulty Level: Easy
8. When the results of a factor analysis indicate that each item is strongly connected to one and only one factor, it is called what?
Ans: Answer may vary.
Learning Objective: 4-3: Summarize the process of conducting an exploratory factor analysis.
Cognitive Domain: Knowledge
Answer Location: Conducting and Interpreting an Exploratory Factor Analysis
Difficulty Level: Easy
9 The particular statistical/mathematical procedure that you might use when beginning an EFA is best labeled as what term?
Ans: Answer may vary.
Learning Objective: 4-3: Summarize the process of conducting an exploratory factor analysis.
Cognitive Domain: Knowledge
Answer Location: Conducting and Interpreting an Exploratory Factor Analysis
Difficulty Level: Easy
10 The procedure by which you might clarify the meaning of factors, by achieving simple structure, is best labeled as what term?
Ans: Answer may vary.
Learning Objective: 4-4: Describe factors, factor loadings, and rotations, and how they offer deeper insight into factor analysis.
Cognitive Domain: Knowledge
Answer Location: A Deeper Perspective on Factors, Factor Loadings, and Rotation
Difficulty Level: Easy
Chapter 5: Reliability
Test Bank
Multiple Choice
1. Conceptually, what does reliability refer to?
a. the degree to which differences in test scores are consistent with differences in true scores b. the degree to which we correctly interpret the meaning of test scores c. the degree to which a test is unbiased and works equally well for all groups of respondents d. the raw amount of true differences among respondents
Ans: A
Learning Objective: 5-1: Describe reliability according to classical test theory.
Cognitive Domain: Knowledge
Answer Location: Overview of Reliability and Classical Test Theory
Difficulty Level: Easy a. true scores are determined additively by Observed Score and Measurement Error b. measurement error is nonrandom/systematic c. measurement error is random/unsystematic d. the measure in question is a valid representation of the underlying latent variable
2 There are two key assumptions of Classical Test Theory. What is one of them?
Ans: C
Learning Objective: 5-2: Differentiate between observed scores, true scores, and measurement error.
Cognitive Domain: Comprehension
Answer Location: Observed Scores, True Scores, and Measurement Error
Difficulty Level: Medium a. 0 b. 1 c. .5 d. unknown, it depends on the test
3 As an implication of one of the assumptions of classical test theory, what is the correlation between true scores on a test and error scores on that test?
Ans: A
Learning Objective: 5-2: Differentiate between observed scores, true scores, and measurement error.
Cognitive Domain: Analysis
Answer Location: Observed Scores, True Scores, and Measurement Error
Difficulty Level: Medium
4. In terms of individuals’ observed score, measurement error will tend to make: a. each individual’s score smaller than it should be b. each individual’s score larger than it should be c. the observed scores imprecise estimates of the true scores d. the observed scores precise, true scores
Ans: C
Learning Objective: 5-2: Differentiate between observed scores, true scores, and measurement error.
Cognitive Domain: Comprehension
Answer Location: Observed Scores, True Scores, and Measurement Error
Difficulty Level: Medium a. Amy’s observed score will be exactly equal to Barry’s. b. Amy’s observed score will be 5 points less than Barry’s. c. Amy’s observed score will be 5 points higher than Barry’s. d. Amy’s observed score will be 10 points higher than Barry’s.
5. Let’s say that Amy’s true score for a self-esteem test was 45, and Barry’s true score was 40. If the test scores have good reliability, what is the difference that we should see between Amy’s and Barry’s observed test scores?
Ans: C
Learning Objective: 5-2: Differentiate between observed scores, true scores, and measurement error.
Cognitive Domain: Analysis
Answer Location: Observed Scores, True Scores, and Measurement Error
Difficulty Level: Medium a. This is a good level of reliability. b. This is a poor level of reliability. c. It depends, depending on the test and its purpose, this could good or poor. d. This level of reliability is theoretically impossible.
6. Imagine you were told that a new test had a reliability of .87. How should this be interpreted?
Ans: A
Learning Objective: 5-3: Summarize how reliability depends on the variance in observed scores, true scores, and measurement error.
Cognitive Domain: Analysis
Answer Location: Variances in Observed Scores, True Scores, and Error Scores
Difficulty Level: Medium a. increase true score variance b. decrease error variance c. increase the correlation between true scores and error scores d. increase the average error score
7. Let’s say that you administered a test to a set of respondents. You then improved the test in a way that would ultimately improve its reliability. You then administered the revised test to the same set of original respondents. According to CTT, which of the following would happen?
Ans: B
Learning Objective: 5-3: Summarize how reliability depends on the variance in observed scores, true scores, and measurement error.
Cognitive Domain: Analysis
Answer Location: Variances in Observed Scores, True Scores, and Error Scores
Difficulty Level: Medium a. the test is unbiased and works equally well for all groups of respondents b. responses to the test items were not strongly affected by measurement error c. the test’s (observed) scores are uncorrelated with other scores that they should be uncorrelated with d. The test is a good (precise) measure of intelligence
8 Say that I develop a test, and I try to convince you that it is a measure of intelligence. You then gather responses to the test, estimate reliability, and find that those test scores have high (estimated) reliability. What can we conclude about those scores, assuming that they do indeed have high reliability?
Ans: B
Learning Objective: 5-3: Summarize how reliability depends on the variance in observed scores, true scores, and measurement error.
Cognitive Domain: Analysis
Answer Location: Variances in Observed Scores, True Scores, and Error Scores
Difficulty Level: Medium
9. More technically, reliability can be defined as: a. the ratio of error score variance to observed score variance b. the (squared) correlation between observed scores and true scores c. the correlation between true scores and error scores d. the correlation between true scores on one test and true scores on another test
Ans: B
Learning Objective: 5-4: Identify the four ways to think about reliability in classical test theory.
Cognitive Domain: Knowledge
Answer Location: Four Ways to Think of Reliability
Difficulty Level: Easy a. -1 b. 0 c. 1 d. the value cannot be determined with this information
10. A true score variance of 0 means a reliability of _____.
Ans: B
Learning Objective: 5-4: Identify the four ways to think about reliability in classical test theory.
Cognitive Domain: Comprehension
Answer Location: Reliability as the Ratio of True Score Variance to Observed Score Variance
Difficulty Level: Medium a. -1 b. 0 c. 1 d. there is no such thing
11. What is perfect reliability?
Ans: C
Learning Objective: 5-4: Identify the four ways to think about reliability in classical test theory.
Cognitive Domain: Comprehension
Answer Location: Reliability as the Ratio of True Score Variance to Observed Score
Variance
Difficulty Level: Medium a. .40 b. .55 c. .70 d. .85
12. Which of the following is a reliability score you would most be satisfied with?
Ans: D
Learning Objective: 5-4: Identify the four ways to think about reliability in classical test theory.
Cognitive Domain: Application
Answer Location: Reliability as the Ratio of True Score Variance to Observed Score Variance
Difficulty Level: Hard a. true score b. reliability c. error variance d. parallel test
13. What term represents the degree to which error affects different people in different ways?
Ans: C
Learning Objective: 5-4: Identify the four ways to think about reliability in classical test theory.
Cognitive Domain: Knowledge
Answer Location: Reliability as Lack of Error Variance
Difficulty Level: Easy a. the coefficient of reliability b. the error variance c. the standard error of measurement d. classical test theory
14. What is the index of reliability?
Ans: A
Learning Objective: 5-4: Identify the four ways to think about reliability in classical test theory.
Cognitive Domain: Knowledge
Answer Location: Reliability as the (Squared) Correlation Between Observed Scores and True Scores
Difficulty Level: Easy
15. If the correlation (roe) between observed scores and error scores is 0, then RXX will equal: a. 0 b. -1 c. 1 d. the value cannot be determined
Ans: C
Learning Objective: 5-4: Identify the four ways to think about reliability in classical test theory.
Cognitive Domain: Analysis
Answer Location: Reliability as the Lack of (Squared) Correlation Between Observed Scores and Error Scores
Difficulty Level: Medium a. the ratio of true score variance b. the standard error of measurement c. the squared correlation error d. the standard correlated error
16. What is another name for the standard deviation of error scores?
Ans: B
Learning Objective: 5-5: Determine how to find the size of measurement error.
Cognitive Domain: Comprehension
Answer Location: Reliability and the Standard Error of Measurement
Difficulty Level: Medium a. true scores b. error scores c. observed scores d. parallel scores
17. The theory of reliability is framed in each of the following except what?
Ans: D
Learning Objective: 5-6: Describe how to estimate the reliability and standard error of measurement for a set of scores.
Cognitive Domain: Analysis
Answer Location: From Theory to Practice: Measurement Models and Their Implications for Estimating Reliability
Difficulty Level: Medium
18. In which of the following test models do we not find that a. parallel b. tau-equivalent c. essentially tau-equivalent d. congeneric Ans: D
Learning Objective: 5-6: Describe how to estimate the reliability and standard error of measurement for a set of scores.
Cognitive Domain: Analysis
Answer Location: From Theory to Practice: Measurement Models and Their Implications for Estimating Reliability
Difficulty Level: Medium a. parallel b. tau-equivalent c. essentially tau-equivalent d. congeneric Ans: A
19. Which test model makes the most assumptions and least likely to apply to real data?
Learning Objective: 5-6: Describe how to estimate the reliability and standard error of measurement for a set of scores.
Cognitive Domain: Comprehension
Answer Location: Overview of Key Assumptions
Difficulty Level: Medium a. parallel b. tau-equivalent c. essentially tau-equivalent d. congeneric Ans: D
20. Which of the following tests is the least restrictive?
Learning Objective: 5-6: Describe how to estimate the reliability and standard error of measurement for a set of scores.
Cognitive Domain: Analysis
Answer Location: Congeneric Tests
Difficulty Level: Medium a. parallel b. tau-equivalent c. essentially tau-equivalent d. congeneric
21. A pair of tests or items are most likely to fit into which model?
Ans: D
Learning Objective: 5-6: Describe how to estimate the reliability and standard error of measurement for a set of scores.
Cognitive Domain: Comprehension
Answer Location: Congeneric Tests
Difficulty Level: Medium
22. Domain sampling theory is an alternative to: a. classical test theory b. standard deviation c. generalizability theory d. psychometric theory
Ans: A
Learning Objective: 5-7: Define domain sampling theory and why it is a common classical test theory.
Cognitive Domain: Comprehension
Answer Location: Domain Sampling Theory
Difficulty Level: Medium a. classical test theory b. domain sampling theory c. generalizability theory d. psychometric theory
23. With ____________, the practical examination of reliability rests on the assumption that it would be possible to create two tests that are at a minimum parallel to each other.
Ans: A
Learning Objective: 5-7: Define domain sampling theory and why it is a common classical test theory.
Cognitive Domain: Comprehension
Answer Location: Domain Sampling Theory
Difficulty Level: Medium a. classical test theory b. domain sampling theory c. generalizability theory d. psychometric theory
24. With ____________, you do not have to make the above assumption, but if you follow the logic of the theory, you will end up with parallel tests by fiat.
Ans: B
Learning Objective: 5-7: Define domain sampling theory and why it is a common classical test theory.
Cognitive Domain: Comprehension
Answer Location: Domain Sampling Theory
Difficulty Level: Medium a. classical test theory b. domain sampling theory c. generalizability theory d. psychometric theory
25. Which theory rests on the assumption that items on any particular test represent a sample from a large indefinite number or domain of potential test items?
Ans: B
Learning Objective: 5-7: Define domain sampling theory and why it is a common classical test theory.
Cognitive Domain: Comprehension
Answer Location: Domain Sampling Theory
Difficulty Level: Medium
True/False
1. Score variance is the sum of true score variance and error variance.
Ans: T
Learning Objective: 5-4: Identify the four ways to think about reliability in classical test theory.
Cognitive Domain: Knowledge
Answer Location: Reliability as Lack of Error Variance
Difficulty Level: Easy
2 When people refer to reliability, they typically are referring to the index of reliability. Ans: F
Learning Objective: 5-4: Identify the four ways to think about reliability in classical test theory.
Cognitive Domain: Comprehension
Answer Location: Reliability as the (Squared) Correlation Between Observed Scores and True Scores
Difficulty Level: Medium
3. If RXX = 1, then sem = 0. Ans: T
Learning Objective: 5-5: Determine how to find the size of measurement error.
Cognitive Domain: Comprehension
Answer Location: Reliability and the Standard Error of Measurement
Difficulty Level: Medium
4. We have no way of knowing people’s true scores on a psychological variable or the error associated with their test responses.
Ans: T
Learning Objective: 5-6: Describe how to estimate the reliability and standard error of measurement for a set of scores.
Cognitive Domain: Analysis
Answer Location: From Theory to Practice: Measurement Models and Their Implications for Estimating Reliability
Difficulty Level: Medium
5. An essentially tau-equivalent test model has alternate forms.
Ans: F
Learning Objective: 5-6: Describe how to estimate the reliability and standard error of measurement for a set of scores.
Cognitive Domain: Comprehension
Answer Location: From Theory to Practice: Measurement Models and Their Implications for Estimating Reliability
Difficulty Level: Medium
6. The correlation between parallel tests is equal to reliability.
Ans: T
Learning Objective: 5-6: Describe how to estimate the reliability and standard error of measurement for a set of scores.
Cognitive Domain: Comprehension
Answer Location: Parallel Tests
Difficulty Level: Medium
7. Two parallel tests can have identical variances but unidentical standard deviations. Ans: F
Learning Objective: 5-6: Describe how to estimate the reliability and standard error of measurement for a set of scores.
Cognitive Domain: Comprehension
Answer Location: Parallel Tests
Difficulty Level: Medium
8. To meet the criteria for tau-equivalent and essentially tau-equivalent tests, two tests need to have the same reliability.
Ans: F
Learning Objective: 5-6: Describe how to estimate the reliability and standard error of measurement for a set of scores.
Cognitive Domain: Comprehension
Answer Location: Tau-Equivalent and Essentially Tau-Equivalent Tests
Difficulty Level: Medium
9. Errors can be correlated across tests.
Ans: T
Learning Objective: 5-6: Describe how to estimate the reliability and standard error of measurement for a set of scores.
Cognitive Domain: Analysis
Answer Location: Tests With Correlated Errors
Difficulty Level: Medium
10. CFA allows test users to estimate the reliability of a test even in cases where error terms are correlated with each other.
Ans: T
Learning Objective: 5-6: Describe how to estimate the reliability and standard error of measurement for a set of scores.
Cognitive Domain: Comprehension
Answer Location: Tests With Correlated Errors
Difficulty Level: Medium
Short Answer
1. What can reliability range from?
Ans: Answer may vary.
Learning Objective: 5-4: Identify the four ways to think about reliability in classical test theory.
Cognitive Domain: Knowledge
Answer Location: Reliability as the Ratio of True Score Variance to Observed Score Variance
Difficulty Level: Easy
2. What can the size of the reliability coefficient indicate in terms of scores?
Ans: Answer may vary.
Learning Objective: 5-4: Identify the four ways to think about reliability in classical test theory.
Cognitive Domain: Analysis
Answer Location: Reliability as the Ratio of True Score Variance to Observed Score Variance
Difficulty Level: Medium
3. What is the reliability range?
Ans: Answer may vary.
Learning Objective: 5-4: Identify the four ways to think about reliability in classical test theory.
Cognitive Domain: Comprehension
Answer Location: Reliability as the Ratio of True Score Variance to Observed Score Variance
Difficulty Level: Medium
4. What does a small degree of error variance indicate?
Ans: Answer may vary.
Learning Objective: 5-4: Identify the four ways to think about reliability in classical test theory.
Cognitive Domain: Analysis
Answer Location: Reliability as Lack of Error Variance
Difficulty Level: Medium
5. What is it called when there is a correlation between observed scores and true scores?
Ans: Answer may vary.
Learning Objective: 5-4: Identify the four ways to think about reliability in classical test theory.
Cognitive Domain: Comprehension
Answer Location: Reliability as the (Squared) Correlation Between Observed Scores and True Scores
Difficulty Level: Medium
6. What do you get when you square the index of reliability?
Ans: Answer may vary.
Learning Objective: 5-4: Identify the four ways to think about reliability in classical test theory.
Cognitive Domain: Comprehension
Answer Location: Reliability as the (Squared) Correlation Between Observed Scores and True Scores
Difficulty Level: Medium
7. What is another name for the standard deviation of error scores?
Ans: Answer may vary.
Learning Objective: 5-5: Determine how to find the size of measurement error.
Cognitive Domain: Knowledge
Answer Location: Reliability and the Standard Error of Measurement
Difficulty Level: Easy
8. What is one of the assumptions that must hold true for a test to be “parallel”?
Ans: Answer may vary.
Learning Objective: 5-6: Describe how to estimate the reliability and standard error of measurement for a set of scores.
Cognitive Domain: Comprehension
Answer Location: Parallel Tests
Difficulty Level: Medium
9. What is one of the assumptions that must hold true for a test to meet the criteria for “tau-equivalent”?
Ans: Answer may vary.
Learning Objective: 5-6: Describe how to estimate the reliability and standard error of measurement for a set of scores.
Cognitive Domain: Comprehension
Answer Location: Tau-Equivalent and Essentially Tau-Equivalent Tests
Difficulty Level: Medium
10. What are the assumptions that must hold true for a test to meet the criteria for “essentially tau-equivalent”?
Ans: Answer may vary.
Learning Objective: 5-6: Describe how to estimate the reliability and standard error of measurement for a set of scores.
Cognitive Domain: Comprehension
Answer Location: Tau-Equivalent and Essentially Tau-Equivalent Tests
Difficulty Level: Medium
