Psychometrics: An Introduction 4th Edition R. Michael Furr ISBN 9781071824085, 1071824082. All Chapt

Instructor Resource Furr, Psychometrics, 4e SAGE Publishing, 2022 Chapter 1: Psychometrics and the Importance of Psychological Measurement Test Bank Multiple Choice 1. Who might be accepted (or not) based on tests of knowledge and achievement? a. graduate students b. clinical psychologists c. human resources director d. university admissions officer Ans: A Learning Objective: 1-1: Summarize the importance of psychological testing.

Cognitive Domain: Analysis Answer Location: Why Psychological Testing Matters to You Difficulty Level: Medium 2. Who of the following is least likely to face acceptance (or not) based on personality tests? a. graduate students b. clinical psychologists c. human resources director d. university admissions officer Ans: A Learning Objective: 1-1: Summarize the importance of psychological testing.

Cognitive Domain: Analysis Answer Location: Why Psychological Testing Matters to You Difficulty Level: Medium 3. Who of the following is least likely to be passed over based on others’ ratings of their psychological characteristics, such as attitude? a. graduate students b. clinical psychologists c. human resources director d. university admissions officer Ans: A Learning Objective: 1-1: Summarize the importance of psychological testing.

Cognitive Domain: Analysis Answer Location: Why Psychological Testing Matters to You Difficulty Level: Medium 4. Which of the following is not one of the key foundations that psychometrics has been built upon? a. the practice of psychological testing b. measurement

Instructor Resource Furr, Psychometrics, 4e SAGE Publishing, 2022 c. psychological characteristics d. the development of statistical concepts Ans: C Learning Objective: 1-1: Summarize the importance of psychological testing.

Cognitive Domain: Analysis Answer Location: Why Psychological Testing Matters to You Difficulty Level: Medium 5. The definition "theoretical psychological characteristics, attributes, processes, or states that cannot be directly observed” matches which term? a. participant reactivity b. hypothetical constructs c. operational definition d. psychometrics Ans: B Learning Objective: 1-2: Differentiate between observable behavior and unobservable psychological attributes.

Cognitive Domain: Knowledge Answer Location: Observable Behavior and Unobservable Psychological Attributes Difficulty Level: Easy 6. "Procedures used to measure theoretical psychological characteristics, attributes, processes, or states that cannot be directly observed” matches which term? a. participant reactivity b. hypothetical constructs c. operational definition d. psychometrics Ans: C Learning Objective: 1-2: Differentiate between observable behavior and unobservable psychological attributes.

Cognitive Domain: Knowledge Answer Location: Observable Behavior and Unobservable Psychological Attributes Difficulty Level: Easy 7. Which of the following is not one of the components of Cronbach’s definition of a test? a. the purpose of the tests is to compare the behaviors of two or more people b. the test must be time-limited, with the goal of seeing respondent accuracy c. tests involve behavioral samples of some kind d. the behavioral samples must be collected in some systematic way Ans: B Learning Objective: 1-3: Define what a psychological test is and some of the key differences and examples of psychological tests.

Cognitive Domain: Comprehension Answer Location: What Is a Psychological Test?

Difficulty Level: Medium

Instructor Resource Furr, Psychometrics, 4e SAGE Publishing, 2022 8. In what way is a quiz a “systematic” procedure? a. The people who take the “test” are different from each other. b. All people take the “test” in a clear and standardized way. c. The “test scorer” will use different procedures/guidelines/criteria to score every test (to the best of his ability, at least) d. Everyone will take it in a different location at (presumably) a different time.

Ans: B Learning Objective: 1-3: Define what a psychological test is and some of the key differences and examples of psychological tests.

Cognitive Domain: Analysis Answer Location: What Is a Psychological Test?

Difficulty Level: Medium 9. Which of the following is a definition of "criterion-referenced tests"? a. tests used to understand how a respondent compares with other respondents (e.g., who has higher or lower levels of the construct being measured) b. time-limited tests, with the goal of seeing how many questions a respondent can answer correctly in a specific amount of time c. tests with cutoff scores, used to sort people into groups (e.g., pass vs no pass) d. tests that are not time-limited, with the expectation that respondents will answer all questions Ans: C Learning Objective: 1-3: Define what a psychological test is and some of the key differences and examples of psychological tests.

Cognitive Domain: Knowledge Answer Location: Types of Tests Difficulty Level: Easy 10. Which of the following is a definition of "norm-referenced tests"? a. tests used to understand how a respondent compares with other respondents (e.g., who has higher or lower levels of the construct being measured) b. time-limited tests, with the goal of seeing how many questions a respondent can answer correctly in a specific amount of time c. tests with cutoff scores, used to sort people into groups (e.g., pass vs no pass) d. tests that are not time-limited, with the expectation that respondents will answer all questions Ans: A Learning Objective: 1-3: Define what a psychological test is and some of the key differences and examples of psychological tests.

Cognitive Domain: Knowledge Answer Location: Types of Tests Difficulty Level: Easy 11. Which of the following is a definition of "speeded tests"?

Instructor Resource Furr, Psychometrics, 4e SAGE Publishing, 2022 a. tests used to understand how a respondent compares with other respondents (e.g., who has higher or lower levels of the construct being measured) b. time-limited tests, with the goal of seeing how many questions a respondent can answer correctly in a specific amount of time c. tests with cutoff scores, used to sort people into groups (e.g., pass vs no pass) d. tests that are not time-limited, with the expectation that respondents will answer all questions Ans: B Learning Objective: 1-3: Define what a psychological test is and some of the key differences and examples of psychological tests.

Cognitive Domain: Knowledge Answer Location: Types of Tests Difficulty Level: Easy 12. Which of the following is a definition of "power tests"? a. tests used to understand how a respondent compares with other respondents (e.g., who has higher or lower levels of the construct being measured) b. time-limited tests, with the goal of seeing how many questions a respondent can answer correctly in a specific amount of time c. tests with cutoff scores, used to sort people into groups (e.g., pass vs no pass) d. tests that are not time-limited, with the expectation that respondents will answer all questions Ans: D Learning Objective: 1-3: Define what a psychological test is and some of the key differences and examples of psychological tests.

Cognitive Domain: Knowledge Answer Location: Types of Tests Difficulty Level: Easy 13. "The science concerned with evaluating the attributes of psychological tests” matches which term? a. participant reactivity b. hypothetical constructs c. operational definition d. psychometrics Ans: D Learning Objective: 1-4: Describe how psychometrics evaluates the attributes of psychological tests.

Cognitive Domain: Knowledge Answer Location: Psychometrics Difficulty Level: Easy 14. Where does the practice of using formal tests originate, some 2,000 to 4,000 years ago? a. the United States b. Russia

Instructor Resource Furr, Psychometrics, 4e SAGE Publishing, 2022 c. China d. England Ans: C Learning Objective: 1-4: Describe how psychometrics evaluates the attributes of psychological tests.

Cognitive Domain: Comprehension Answer Location: A Brief History of Psychometrics Difficulty Level: Medium 15. When did psychological measurement increase, as psychological science emerged and researchers began systemically measuring various qualities and responses of individuals in experimental studies? a. 16th century b. 17th century c. 18th century d. 19th century Ans: D Learning Objective: 1-4: Describe how psychometrics evaluates the attributes of psychological tests.

Cognitive Domain: Comprehension Answer Location: A Brief History of Psychometrics Difficulty Level: Medium 16. Who is considered the founding father of modern psychometrics? a. Galton b. Pearson c. Spearman d. Guilford Ans: A Learning Objective: 1-4: Describe how psychometrics evaluates the attributes of psychological tests.

Cognitive Domain: Comprehension Answer Location: A Brief History of Psychometrics Difficulty Level: Medium 17. "When a respondent realizes that she/he is being assessed, and that realization affects or biases the way that she/he responds to the test” matches which term? a. participant reactivity b. hypothetical constructs c. operational definition d. psychometrics Ans: A Learning Objective: 1-5: Explain the challenges to measurement in psychology.

Cognitive Domain: Knowledge Answer Location: Challenges to Measurement in Psychology Difficulty Level: Easy

Instructor Resource Furr, Psychometrics, 4e SAGE Publishing, 2022 18. Which type of psychologist would most likely wish to track clients’ emotional changes from one therapy session to another? a. abnormal b. clinical c. experimental d. differential Ans: B Learning Objective: 1-5: Explain the challenges to measurement in psychology.

Cognitive Domain: Analysis Answer Location: Challenges to Measurement in Psychology Difficulty Level: Medium 19. The ability to identify and characterize psychological difference is at the heart of: a. psychological testing b. psychological measurement c. psychological characteristics d. statistical concepts Ans: B Learning Objective: 1-6: Evaluate why individual differences are important in psychological measurement.

Cognitive Domain: Comprehension Answer Location: The Importance of Individual Differences Difficulty Level: Medium 20. What is the foundation of all methods used to evaluate tests? a. psychological testing b. psychological measurement c. psychological characteristics d. statistical concepts Ans: B Learning Objective: 1-6: Evaluate why individual differences are important in psychological measurement.

Cognitive Domain: Analysis Answer Location: The Importance of Individual Differences Difficulty Level: Medium 21. Who was primarily interested in how people differ from each other? Such as how some people are taller than others, or smarter than others. a. Galton b. Pearson c. Spearman d. Guilford Ans: A Learning Objective: 1-7: Recall what differential psychology is and the important pioneers of the field.

Instructor Resource Furr, Psychometrics, 4e SAGE Publishing, 2022 Cognitive Domain: Comprehension Answer Location: But Psychometrics Goes Well Beyond “Differential” Psychology Difficulty Level: Medium 22. Which of the following typically contrasts with differential psychology? a. abnormal b. experimental c. biological d. comparative Ans: B Learning Objective: 1-7: Recall what differential psychology is and the important pioneers of the field.

Cognitive Domain: Analysis Answer Location: But Psychometrics Goes Well Beyond “Differential” Psychology Difficulty Level: Medium 23. Which of the following types of psychology usually focused on intelligence, aptitude, and personality, as discussed in the text? a. differential b. experimental c. biological d. comparative Ans: A Learning Objective: 1-7: Recall what differential psychology is and the important pioneers of the field.

Cognitive Domain: Comprehension Answer Location: But Psychometrics Goes Well Beyond “Differential” Psychology Difficulty Level: Medium 24. Which of the following types of psychology focused mainly on the average person instead of the differences among people? a. differential b. experimental c. biological d. comparative Ans: B Learning Objective: 1-7: Recall what differential psychology is and the important pioneers of the field.

Cognitive Domain: Comprehension Answer Location: But Psychometrics Goes Well Beyond “Differential” Psychology Difficulty Level: Medium 25. Which type of psychologist should be concerned with measuring behavior and psychological attributes? a. behavioral psychologists b. experimental psychologists

Instructor Resource Furr, Psychometrics, 4e SAGE Publishing, 2022 c. differential psychologists d. all psychologists Ans: D Learning Objective: 1-7: Recall what differential psychology is and the important pioneers of the field.

Cognitive Domain: Analysis Answer Location: But Psychometrics Goes Well Beyond “Differential” Psychology Difficulty Level: Medium True/False 1. Test users have a responsibility to examine and interpret important information about the meaning and quality of tests they use.

Ans: T Learning Objective: 1-1: Summarize the importance of psychological testing.

Cognitive Domain: Comprehension Answer Location: Why Psychological Testing Matters to You Difficulty Level: Medium 2. An operational definition is procedures used to measure theoretical psychological characteristics, attributes, processes, or states that cannot be directly observed Ans: T Learning Objective: 1-2: Differentiate between observable behavior and unobservable psychological attributes.

Cognitive Domain: Knowledge Answer Location: Observable Behavior and Unobservable Psychological Attributes Difficulty Level: Easy 3. Hypothetical constructs are when a respondent realizes that she/he is being assessed and that realization affects or biases the way that she/he responds to the test.

Ans: F Learning Objective: 1-2: Differentiate between observable behavior and unobservable psychological attributes.

Cognitive Domain: Knowledge Answer Location: Observable Behavior and Unobservable Psychological Attributes Difficulty Level: Easy 4. “Hunger” is an unobservable psychological attribute.

Ans: T Learning Objective: 1-2: Differentiate between observable behavior and unobservable psychological attributes.

Cognitive Domain: Analysis Answer Location: Observable Behavior and Unobservable Psychological Attributes Difficulty Level: Medium

Instructor Resource Furr, Psychometrics, 4e SAGE Publishing, 2022 5. Norm-referenced tests are tests with cutoff scores, used to sort people into groups (e.g., pass vs no pass).

Ans: F Learning Objective: 1-3: Define what a psychological test is and some of the key differences and examples of psychological tests.

Cognitive Domain: Knowledge Answer Location: Types of Tests Difficulty Level: Easy 6. Power tests are tests that are not time-limited, with the expectation that respondents will answer all questions.

Ans: T Learning Objective: 1-3: Define what a psychological test is and some of the key differences and examples of psychological tests.

Cognitive Domain: Knowledge Answer Location: Types of Tests Difficulty Level: Easy 7. Speeded-referenced tests are time-limited tests, with the goal of seeing how many questions a respondent can answer correctly in a specific amount of time.

Ans: T Learning Objective: 1-3: Define what a psychological test is and some of the key differences and examples of psychological tests.

Cognitive Domain: Knowledge Answer Location: Types of Tests Difficulty Level: Easy 8. Criterion-referenced tests are tests used to understand how a respondent compares with other respondents (e.g., who have higher or lower levels of the construct being measured).

Ans: F Learning Objective: 1-3: Define what a psychological test is and some of the key differences and examples of psychological tests.

Cognitive Domain: Knowledge Answer Location: Types of Tests Difficulty Level: Easy 9. Psychometrics is the science concerned with evaluating the attributes of psychological tests.

Ans: T Learning Objective: 1-4: Describe how psychometrics evaluates the attributes of psychological tests.

Cognitive Domain: Knowledge Answer Location: Psychometrics Difficulty Level: Easy

Instructor Resource Furr, Psychometrics, 4e SAGE Publishing, 2022 10. All measures in psychology require that we obtain behavioral samples of some kind.

Ans: T Learning Objective: 1-6: Evaluate why individual differences are important in psychological measurement.

Cognitive Domain: Comprehension Answer Location: The Importance of Individual Differences Difficulty Level: Medium

Short Answer 1. Is the following an unobservable psychological attribute or an observable behavioral attribute?: intelligence.

Ans: Answer may vary.

Learning Objective: 1-2: Differentiate between observable behavior and unobservable psychological attributes.

Cognitive Domain: Analysis Answer Location: Observable Behavior and Unobservable Psychological Attributes Difficulty Level: Medium 2. Is the following an unobservable psychological attribute or an observable behavioral attribute?: percent correct on a psychometric test.

Ans: Answer may vary.

Learning Objective: 1-2: Differentiate between observable behavior and unobservable psychological attributes.

Cognitive Domain: Analysis Answer Location: Observable Behavior and Unobservable Psychological Attributes Difficulty Level: Medium 3. Is the following an unobservable psychological attribute or an observable behavioral attribute?: depression.

Ans: Answer may vary.

Learning Objective: 1-2: Differentiate between observable behavior and unobservable psychological attributes.

Cognitive Domain: Analysis Answer Location: Observable Behavior and Unobservable Psychological Attributes Difficulty Level: Medium 4. Is the following an unobservable psychological attribute or an observable behavioral attribute?: number of words recalled from a list of words.

Ans: Answer may vary.

Learning Objective: 1-2: Differentiate between observable behavior and unobservable psychological attributes.

Cognitive Domain: Analysis

Instructor Resource Furr, Psychometrics, 4e SAGE Publishing, 2022 Answer Location: Observable Behavior and Unobservable Psychological Attributes Difficulty Level: Medium 5. Is the following an unobservable psychological attribute or an observable behavioral attribute?: extroversion.

Ans: Answer may vary.

Learning Objective: 1-2: Differentiate between observable behavior and unobservable psychological attributes.

Cognitive Domain: Analysis Answer Location: Observable Behavior and Unobservable Psychological Attributes Difficulty Level: Medium 6. Is the following an unobservable psychological attribute or an observable behavioral attribute?: aptitude.

Ans: Answer may vary.

Learning Objective: 1-2: Differentiate between observable behavior and unobservable psychological attributes.

Cognitive Domain: Analysis Answer Location: Observable Behavior and Unobservable Psychological Attributes Difficulty Level: Medium 7. Is the following an unobservable psychological attribute or an observable behavioral attribute?: number of food pellets eaten in a 24-hour period.

Ans: Answer may vary.

Learning Objective: 1-2: Differentiate between observable behavior and unobservable psychological attributes.

Cognitive Domain: Analysis Answer Location: Observable Behavior and Unobservable Psychological Attributes Difficulty Level: Medium 8. What is one of the three components of Cronbach’s definition of a test?

Ans: Answer may vary.

Learning Objective: 1-3: Define what a psychological test is and some of the key differences and examples of psychological tests.

Cognitive Domain: Knowledge Answer Location: What Is a Psychological Test?

Difficulty Level: Easy 9. What is the purpose of a test, as implied by Cronbach’s definition of a psychological test?

Ans: Answer may vary.

Learning Objective: 1-3: Define what a psychological test is and some of the key differences and examples of psychological tests.

Cognitive Domain: Comprehension Answer Location: What Is a Psychological Test?

Difficulty Level: Medium

Instructor Resource Furr, Psychometrics, 4e SAGE Publishing, 2022 10. Name one type of person who should be concerned about psychometrics and the quality of psychological testing.

Ans: Answer may vary.

Learning Objective: 1-4: Describe how psychometrics evaluates the attributes of psychological tests.

Cognitive Domain: Comprehension Answer Location: What Is Psychometrics?

Difficulty Level: Medium

Instructor Resource Furr, Psychometrics, 4e SAGE Publishing, 2022 Chapter 2: Scaling Test Bank Multiple Choice 1. According to the text, __________ is the assignment of numerals to objects or events according to rules. a. scaling b. measurement c. quantity d. order Ans: B Learning Objective: 2-1: Summarize the three important numerical properties and the meaning of zero.

Cognitive Domain: Knowledge Answer Location: Introduction Difficulty Level: Easy 2. ___________ concerns the way numerical values are assigned to psychological attributes. a. scaling b. measurement c. quantity d. order Ans: A Learning Objective: 2-1: Summarize the three important numerical properties and the meaning of zero.

Cognitive Domain: Knowledge Answer Location: Introduction Difficulty Level: Easy 3. Which of the following is the simplest psychological measurement? a. order b. quantity c. quality d. identity Ans: D Learning Objective: 2-1: Summarize the three important numerical properties and the meaning of zero.

Cognitive Domain: Comprehension Answer Location: The Property of Identity Difficulty Level: Medium

Instructor Resource Furr, Psychometrics, 4e SAGE Publishing, 2022 4. Which form of psychological measure would you assume would be attributed to an example of children being classified as having behavior problems similar to them with respect to behavior and different from children who are classified as not having behavioral problems? a. order b. quantity c. quality d. identity Ans: D Learning Objective: 2-1: Summarize the three important numerical properties and the meaning of zero.

Cognitive Domain: Comprehension Answer Location: The Property of Identity Difficulty Level: Medium 5. Characteristics such as happy or sad, introverted or extroverted, apply most to which psychological measure? a. order b. quantity c. quality d. identity Ans: D Learning Objective: 2-1: Summarize the three important numerical properties and the meaning of zero.

Cognitive Domain: Analysis Answer Location: The Property of Identity Difficulty Level: Medium 6. Which property would indicate the rank of people relative to each other along some dimension? a. order b. quantity c. quality d. identity Ans: A Learning Objective: 2-1: Summarize the three important numerical properties and the meaning of zero.

Cognitive Domain: Comprehension Answer Location: The Property of Order Difficulty Level: Medium 7. Which property conveys the greatest information? a. order b. quantity c. quality d. identity

Instructor Resource Furr, Psychometrics, 4e SAGE Publishing, 2022 Ans: B Learning Objective: 2-1: Summarize the three important numerical properties and the meaning of zero.

Cognitive Domain: Comprehension Answer Location: The Property of Quantity Difficulty Level: Medium 8. Which of the following is not considered a quantitative value in the context of measurement? a. scalar b. metric c. behavior d. cardinal Ans: C Learning Objective: 2-1: Summarize the three important numerical properties and the meaning of zero.

Cognitive Domain: Comprehension Answer Location: The Property of Quantity Difficulty Level: Medium 9. What is 0 called when it has no existence, such as in reaction time? a. absent zero b. attribute zero c. arbitrary zero d. absolute zero Ans: D Learning Objective: 2-1: Summarize the three important numerical properties and the meaning of zero.

Cognitive Domain: Comprehension Answer Location: The Number 0 Difficulty Level: Medium 10. What is 0 called when it is tied to attributes such as time and temperature? a. absent zero b. attribute zero c. arbitrary zero d. absolute zero Ans: C Learning Objective: 2-1: Summarize the three important numerical properties and the meaning of zero.

Cognitive Domain: Comprehension Answer Location: The Number 0 Difficulty Level: Medium

Instructor Resource Furr, Psychometrics, 4e SAGE Publishing, 2022 11. A group of third-grade children are taking a spelling test, and out of a list of ten spelling words, Miguel did not get any right. What is Miguel’s score an example of in the simplest of interpretations? a. absent zero b. attribute zero c. arbitrary zero d. absolute zero Ans: D Learning Objective: 2-1: Summarize the three important numerical properties and the meaning of zero.

Cognitive Domain: Analysis Answer Location: The Number 0 Difficulty Level: Medium 12. Which of the following is true about additivity? a. Additivity requires unit size to continually increase, whereby a unit increase at one point in the measurement process must be the same as the unit decrease at any other point. b. Additivity requires unit size to continually decrease, whereby a unit increase at one point in the measurement process must be the same as the unit decrease at any other point. c. Additivity requires unit size to remain constant, whereby a unit increase at one point in the measurement process must be the same as the unit increase at any other point. d. Additivity requires unit size to remain flexible, whereby a unit decrease at one point in the measurement process must be the same as the unit increase at any other point.

Ans: C Learning Objective: 2-3: Explain the difference between counting as a form of measurement versus forms of counting that are not forms of measurement.

Cognitive Domain: Analysis Answer Location: Additivity Difficulty Level: Medium 13. Religious orientation (Christian, Jewish, Buddhist, Hindu, Muslim, etc.) is measured on a(n) ________ scale. a. nominal b. ordinal c. interval d. ratio Ans: A Learning Objective: 2-4: Differentiate between four scales of measurement and how they relate to numerical properties.

Cognitive Domain: Analysis Answer Location: Nominal Scales Difficulty Level: Medium 14. What is the most fundamental level of measurement?

Instructor Resource Furr, Psychometrics, 4e SAGE Publishing, 2022 a. nominal b. ordinal c. interval d. ratio Ans: A Learning Objective: 2-4: Differentiate between four scales of measurement and how they relate to numerical properties.

Cognitive Domain: Comprehension Answer Location: Nominal Scales Difficulty Level: Medium 15. Sorting people into two categories based on biological sex—males and females, Group 0 and Group 1, respectively. What scale is this an example of? a. nominal b. ordinal c. interval d. ratio Ans: A Learning Objective: 2-4: Differentiate between four scales of measurement and how they relate to numerical properties.

Cognitive Domain: Analysis Answer Location: Nominal Scales Difficulty Level: Medium 16. Which scale is used to identify groups of people who share a common attribute that is not shared by people in other groups? a. nominal b. ordinal c. interval d. ratio Ans: A Learning Objective: 2-4: Differentiate between four scales of measurement and how they relate to numerical properties.

Cognitive Domain: Comprehension Answer Location: Nominal Scales Difficulty Level: Medium 17. Which of the following is an example of an ordinal scale? a. biological sex b. class rank c. temperature d. distance Ans: B Learning Objective: 2-4: Differentiate between four scales of measurement and how they relate to numerical properties.

Cognitive Domain: Comprehension

Instructor Resource Furr, Psychometrics, 4e SAGE Publishing, 2022 Answer Location: Ordinal Scales Difficulty Level: Medium 18. Which scale of measurement has identity and order but not quantity or absolute zero? a. nominal b. ordinal c. interval d. ratio Ans: B Learning Objective: 2-4: Differentiate between four scales of measurement and how they relate to numerical properties.

Cognitive Domain: Analysis Answer Location: Ordinal Scales Difficulty Level: Medium 19. This scale is based on numbers that represent quantitative differences between people in the amount of the attribute being measured, but it also has an arbitrary zero. a. nominal b. ordinal c. interval d. ratio Ans: C Learning Objective: 2-4: Differentiate between four scales of measurement and how they relate to numerical properties.

Cognitive Domain: Comprehension Answer Location: Interval Scales Difficulty Level: Medium 20. How are the vast majority of intelligence tests, personality tests, achievement tests, developmental tests, and many other psychological assessments categorized in terms of scale? a. nominal b. ordinal c. interval d. ratio Ans: C Learning Objective: 2-4: Differentiate between four scales of measurement and how they relate to numerical properties.

Cognitive Domain: Comprehension Answer Location: Interval Scales Difficulty Level: Medium 21. Which of the following is an example of an interval scale? a. biological sex b. class rank

Instructor Resource Furr, Psychometrics, 4e SAGE Publishing, 2022 c. temperature d. distance Ans: C Learning Objective: 2-4: Differentiate between four scales of measurement and how they relate to numerical properties.

Cognitive Domain: Comprehension Answer Location: Interval Scales Difficulty Level: Medium 22. This scale is based on numbers that represent quantitative differences between people in the amount of the attribute being measured, but it also has an absolute zero. a. nominal b. ordinal c. interval d. ratio Ans: D Learning Objective: 2-4: Differentiate between four scales of measurement and how they relate to numerical properties.

Cognitive Domain: Comprehension Answer Location: Ratio Scales Difficulty Level: Medium 23. Which scale is considered the highest level of measurement? a. nominal b. ordinal c. interval d. ratio Ans: D Learning Objective: 2-4: Differentiate between four scales of measurement and how they relate to numerical properties.

Cognitive Domain: Comprehension Answer Location: Ratio Scales Difficulty Level: Medium 24. Which scale allows additivity and multiplicative interpretations, such as saying 80 miles is "twice as far" as 40 miles? a. nominal b. ordinal c. interval d. ratio Ans: D Learning Objective: 2-4: Differentiate between four scales of measurement and how they relate to numerical properties.

Cognitive Domain: Analysis Answer Location: Ratio Scales Difficulty Level: Medium

Instructor Resource Furr, Psychometrics, 4e SAGE Publishing, 2022 25. Which of the following is an example of a ratio scale? a. biological sex b. class rank c. temperature d. distance Ans: D Learning Objective: 2-4: Differentiate between four scales of measurement and how they relate to numerical properties.

Cognitive Domain: Analysis Answer Location: Ratio Scales Difficulty Level: Medium True/False 1. A numerical value cannot be used to represent your biological sex.

Ans: F Learning Objective: 2-1: Summarize the three important numerical properties and the meaning of zero.

Cognitive Domain: Analysis Answer Location: Fundamental Issues With Numbers Difficulty Level: Medium 2. To establish a category, the people within a category must satisfy the property of identity.

Ans: T Learning Objective: 2-1: Summarize the three important numerical properties and the meaning of zero.

Cognitive Domain: Comprehension Answer Location: The Property of Identity Difficulty Level: Medium 3. When placing people into categories, the categories must be mutually exclusive.

Ans: T Learning Objective: 2-1: Summarize the three important numerical properties and the meaning of zero.

Cognitive Domain: Analysis Answer Location: The Property of Identity Difficulty Level: Medium 4. The property of order conveys more information than the identity or quantity properties.

Ans: F Learning Objective: 2-1: Summarize the three important numerical properties and the meaning of zero.

Instructor Resource Furr, Psychometrics, 4e SAGE Publishing, 2022 Cognitive Domain: Comprehension Answer Location: The Property of Order Difficulty Level: Medium 5. Measuring a day with a temperature of 50°C compared to a day with a temperature of 40°C would most likely be found in the property of order.

Ans: F Learning Objective: 2-1: Summarize the three important numerical properties and the meaning of zero.

Cognitive Domain: Analysis Answer Location: The Property of Quantity Difficulty Level: Medium 6. The number 0 can only be reflected as a state in which an attribute of an object or event has no existence.

Ans: F Learning Objective: 2-1: Summarize the three important numerical properties and the meaning of zero.

Cognitive Domain: Comprehension Answer Location: The Number 0 Difficulty Level: Medium 7. Arithmetical operations should be restricted when a test has a relative 0 point.

Ans: T Learning Objective: 2-1: Summarize the three important numerical properties and the meaning of zero.

Cognitive Domain: Analysis Answer Location: The Number 0 Difficulty Level: Medium 8. Arbitrariness is an important concept in understanding units of measurement.

Ans: T Learning Objective: 2-2: Describe familiar and unfamiliar units of measurement, including the solution to the unfamiliar units of measurement.

Cognitive Domain: Knowledge Answer Location: Units of Measurement Difficulty Level: Easy 9. According to additivity, the size of a measurement unit should change as the conditions of measurement change.

Ans: F Learning Objective: 2-3: Explain the difference between counting as a form of measurement versus forms of counting that are not forms of measurement.

Cognitive Domain: Comprehension Answer Location: Additivity Difficulty Level: Medium

Instructor Resource Furr, Psychometrics, 4e SAGE Publishing, 2022 10. The order principle can be found in the nominal scale but not the ratio scale.

Ans: F Learning Objective: 2-4: Differentiate between four scales of measurement and how they relate to numerical properties.

Cognitive Domain: Analysis Answer Location: Nominal Scales Difficulty Level: Medium

Short Answer 1. In psychological measurement, what do numerals represent?

Ans: Answer may vary.

Learning Objective: 2-1: Summarize the three important numerical properties and the meaning of zero.

Cognitive Domain: Knowledge Answer Location: Fundamental Issues With Numbers Difficulty Level: Easy 2. IQ test is to level of intelligence, as Rosenberg Self-Esteem Inventory is to what?

Ans: Answer may vary.

Learning Objective: 2-1: Summarize the three important numerical properties and the meaning of zero.

Cognitive Domain: Analysis Answer Location: Fundamental Issues With Numbers Difficulty Level: Medium 3. What is one of the numerical properties outlined in this chapter?

Ans: Answer may vary.

Learning Objective: 2-1: Summarize the three important numerical properties and the meaning of zero.

Cognitive Domain: Knowledge Answer Location: Fundamental Issues With Numbers Difficulty Level: Easy 4. What is one of the rules that must be followed when sorting people into categories?

Ans: Answer may vary.

Learning Objective: 2-1: Summarize the three important numerical properties and the meaning of zero.

Cognitive Domain: Comprehension Answer Location: The Property of Identity Difficulty Level: Medium 5. What is a limitation of the order property?

Instructor Resource Furr, Psychometrics, 4e SAGE Publishing, 2022 Ans: Answer may vary.

Learning Objective: 2-1: Summarize the three important numerical properties and the meaning of zero.

Cognitive Domain: Analysis Answer Location: The Property of Order Difficulty Level: Medium 6. What is one way the number 0 can be interpreted?

Ans: Answer may vary.

Learning Objective: 2-1: Summarize the three important numerical properties and the meaning of zero.

Cognitive Domain: Analysis Answer Location: The Number 0 Difficulty Level: Medium 7. What is one form of arbitrariness?

Ans: Answer may vary.

Learning Objective: 2-2: Describe familiar and unfamiliar units of measurement, including the solution to the unfamiliar units of measurement.

Cognitive Domain: Comprehension Answer Location: Units of Measurement Difficulty Level: Medium 8. Which level of measurement involves identity, but not order, quantity, or absolute zero?

Ans: Answer may vary.

Learning Objective: 2-4: Differentiate between four scales of measurement and how they relate to numerical properties.

Cognitive Domain: Application Answer Location: Nominal Scales Difficulty Level: Hard 9. Which level of measurement can be associated with all the following principles: identity, order, quantity, absolute zero?

Ans: Answer may vary.

Learning Objective: 2-4: Differentiate between four scales of measurement and how they relate to numerical properties.

Cognitive Domain: Application Answer Location: Ratio Scales Difficulty Level: Hard 10. What scale would a variable such as hair color fall under (it’s mathematically possible, though not particularly meaningful)?

Ans: Answer may vary.

Learning Objective: 2-5: Discuss the implications for meaningfulness when choosing a scale of measurement.

Instructor Resource Furr, Psychometrics, 4e SAGE Publishing, 2022 Cognitive Domain: Analysis Answer Location: Scales of Measurement: Practical Implications Difficulty Level: Medium

Instructor Resource Furr, Psychometrics, 4e SAGE Publishing, 2022 Chapter 3: Differences, Consistency, and the Meaning of Test Scores Test Bank Multiple Choice 1. A group of respondents took a psychological test. In their distribution of test scores, the variance is 4.56. Which of the following is the most accurate statement about that value? a. It means that the people’s scores on the test are likely connected to scores on another test b. It reflects a small amount of variability (people did not differ greatly from each other, in terms of their test scores) c. It reflects a large amount of variability (i.e., people do differ quite a lot from each other, in terms of their test scores) d. This value is possible, but it is difficult to know whether it reflects a large or small amount of variability.

Ans: D Learning Objective: 3-3: Summarize the importance of central tendency, variability, and distributions in statistics.

Cognitive Domain: Analysis Answer Location: Variability and Distribution of Scores Difficulty Level: Medium 2. What kind of distribution is usually considered valid in statistical procedures? a. exponential b. gamma c. normal d. chi-squared Ans: C Learning Objective: 3-3: Summarize the importance of central tendency, variability, and distributions in statistics.

Cognitive Domain: Comprehension Answer Location: Distribution Shapes and Normal Distributions Difficulty Level: Medium 3. What type of distribution is considered a theoretical ideal? a. exponential b. gamma c. normal d. chi-squared Ans: C Learning Objective: 3-3: Summarize the importance of central tendency, variability, and distributions in statistics.

Instructor Resource Furr, Psychometrics, 4e SAGE Publishing, 2022 Cognitive Domain: Comprehension Answer Location: Distribution Shapes and Normal Distributions Difficulty Level: Medium 4. What is the value of a perfectly symmetrical distribution? a. 0 b. 1 c. -1 d. 2 Ans: A Learning Objective: 3-3: Summarize the importance of central tendency, variability, and distributions in statistics.

Cognitive Domain: Comprehension Answer Location: Distribution Shapes and Normal Distributions Difficulty Level: Medium 5. What is the most likely value of a negatively skewed distribution? a. 0 b. 1 c. -1 d. 2 Ans: C Learning Objective: 3-3: Summarize the importance of central tendency, variability, and distributions in statistics.

Cognitive Domain: Comprehension Answer Location: Distribution Shapes and Normal Distributions Difficulty Level: Medium 6. A group of respondents took two psychological tests, and the correlation between the two distributions of scores is -.39. Which of the following is true? a. People who score relatively high on one of the tests tend to score relatively high on the other test, and people who score relatively low on one of the tests tend to score relatively low on the other test. b. People who score relatively high on one of the tests tend to score relatively low on the other test. c. Scores on the first test have nothing to do with scores on the second test. d. This value is impossible, you cannot get a correlation -.39.

Ans: B Learning Objective: 3-4: Describe how associations are made between variables.

Cognitive Domain: Analysis Answer Location: Correlation Difficulty Level: Medium 7. Which Pearson correlation coefficient realistically shows the strongest relationship between two variables?

Instructor Resource Furr, Psychometrics, 4e SAGE Publishing, 2022 a. 1.20 b. .00 c. .75 d. -.80 Ans: D Learning Objective: 3-4: Describe how associations are made between variables.

Cognitive Domain: Application Answer Location: Correlation Difficulty Level: Hard 8. Say, in a sample of students, there is a zero correlation between height and selfesteem. This means that _________. a. taller people tend to have higher self-esteem than do shorter people b. shorter people tend to have higher self-esteem than do taller people c. none of the students have high self-esteem d. a person with high self-esteem is equally likely to be tall or short Ans: D Learning Objective: 3-4: Describe how associations are made between variables.

Cognitive Domain: Analysis Answer Location: Correlation Difficulty Level: Medium 9. You conducted a study that examines the correlation between IQ and income, and you find a value of r = 0.75. At the end of the study, you find out all the IQ scores were scored 10 points too high. What will the value of r be after you rescore the IQ data, subtracting 10 from each person’s score? a. r will be increased b. r will be decreased c. r will remain the same d. cannot be determined from the information given Ans: C Learning Objective: 3-4: Describe how associations are made between variables.

Cognitive Domain: Analysis Answer Location: Correlation Difficulty Level: Medium 10. Imagine that we examine the predictive accuracy of SAT scores by correlating those scores with college GPA. Imagine we find that the correlation between SAT scores and CGPA is r = .40 for females and r = .20 for males. What is the most accurate interpretation of these findings? a. SAT scores are higher among females than among males b. Both SAT scores and CGPA scores are higher (on average) among females than among males c. SAT scores are more strongly associated with CGPA among females than among males d. The distribution of SAT scores is more skewed among females than among males.

Instructor Resource Furr, Psychometrics, 4e SAGE Publishing, 2022 Ans: C Learning Objective: 3-4: Describe how associations are made between variables.

Cognitive Domain: Analysis Answer Location: Correlation Difficulty Level: Medium 11. The variance: a. is the sum of participants' scores divided by the number of scores b. reflects the degree to which participants' scores differ from each other c. is an index of skewness d. is inferior to the range as an index of central tendency Ans: B Learning Objective: 3-5: Compare and contrast variance and covariance in computing composite scores.

Cognitive Domain: Comprehension Answer Location: Variance and Covariance for “Composite Variables” Difficulty Level: Medium 12. If you and your best friend are answering questions related to neuroticism. If you obtained a score of 24 on the scale and your friend obtained 85 on the scale, what could you interpret? a. You are more neurotic than your friend. b. Your friend is more neurotic than you are. c. Both of you are above the accepted neurotic level. d. It is impossible to make that kind of distinction.

Ans: B Learning Objective: 3-7: Describe how test scores are represented, including z scores, standard scores, and percentile ranks.

Cognitive Domain: Analysis Answer Location: Interpreting Test Scores Difficulty Level: Medium 13. A z score:

a. is the inverse of the mean b. is a measure or error variance c. reflects how far a participant's score falls from the mean d. is a measure of the variance Ans: C Learning Objective: 3-7: Describe how test scores are represented, including z scores, standard scores, and percentile ranks.

Cognitive Domain: Comprehension Answer Location: z Scores (Standard Scores) Difficulty Level: Medium 14. Imagine that I responded to a new self-esteem questionnaire, and my raw score was 153.25. Based on that information, what would you conclude about me?

Instructor Resource Furr, Psychometrics, 4e SAGE Publishing, 2022 a. I have a high level of self-esteem. b. I have a low level of self-esteem. c. Nothing; such a score would be impossible. d. Nothing, because you have no interpretive frame of reference for scores on that questionnaire.

Ans: D Learning Objective: 3-7: Describe how test scores are represented, including z scores, standard scores, and percentile ranks.

Cognitive Domain: Analysis Answer Location: z Scores (Standard Scores) Difficulty Level: Medium 15. Now, imagine that I also responded to a questionnaire measuring “knowledge of psychometrics,” and my raw score is 36 on that test. Based on that information and my self-esteem score, what could you conclude about me? a. My level of self-esteem is greater than the level of my knowledge of psychometrics. b. My level of self-esteem is less than the level of my knowledge about psychometrics. c. Nothing, because you don’t know whether the two measures are comparable in any way. d. Nothing; such a score would be impossible.

Ans: C Learning Objective: 3-7: Describe how test scores are represented, including z scores, standard scores, and percentile ranks.

Cognitive Domain: Analysis Answer Location: z Scores (Standard Scores) Difficulty Level: Medium 16. Say my z score (standard score) on a self-esteem test is .50. How can you interpret that value? a. My self-esteem is 50% of the total possible self-esteem as measured by the test b. My self-esteem score is one-half of a standard deviation above the mean score. c. Compared to the average person, my self-esteem score is relatively low. d. I score higher than 50% of the other people who have taken the test.

Ans: B Learning Objective: 3-7: Describe how test scores are represented, including z scores, standard scores, and percentile ranks.

Cognitive Domain: Analysis Answer Location: z Scores (Standard Scores) Difficulty Level: Medium 17. Imagine that your raw score on a Neuroticism scale was 45, where the average score was 50, with a standard deviation of 5. What is your z score (i.e., standard score)? a. -1 b. .90 c. 1

Instructor Resource Furr, Psychometrics, 4e SAGE Publishing, 2022 d. -5 Ans: A Learning Objective: 3-7: Describe how test scores are represented, including z scores, standard scores, and percentile ranks.

Cognitive Domain: Application Answer Location: z Scores (Standard Scores) Difficulty Level: Hard 18. After taking this exam, you find out that your standard score (z score) is .99. What’s the correct interpretation for this? a. Your score is higher than 99% of your classmates’ scores. b. Your score is almost 1 standard deviation higher than the average class score. c. You earned 99% of the possible points on the test. d. Compared to the average person, your score is relatively low.

Ans: B Learning Objective: 3-7: Describe how test scores are represented, including z scores, standard scores, and percentile ranks.

Cognitive Domain: Analysis Answer Location: z Scores (Standard Scores) Difficulty Level: Medium 19. A developmental psychologist studies the development of reading ability. He studies a sample of 7-year-olds and a sample of 10-year-olds. Not surprisingly, he finds that the 10-year-old group has a higher mean level of reading ability than the 7-year-old group.

He also finds that the 10-year-old group has a larger standard deviation than the 7-yearold sample. What does the difference in standard deviations tell us? a. The average 10-year-old reads better than the average 7-year-old. b. All 10-year-olds read better than all 7-year-olds. c. There are greater differences among the 10-year-olds’ reading abilities than among the 7-year-olds’ reading abilities. d. There are fewer differences among the 10-year-olds’ reading abilities than among the 7-year-olds’ reading abilities.

Ans: C Learning Objective: 3-7: Describe how test scores are represented, including z scores, standard scores, and percentile ranks.

Cognitive Domain: Analysis Answer Location: Converted Standard Scores (Standardized Scores) Difficulty Level: Medium 20. Say my percentile rank on a “knowledge of psychometrics” test is .60. How can you interpret that value (mark all that are correct)? a. My score is 60% of the total possible self-esteem as measured by the test b. My score is .60 of a standard deviation above the mean score. c. Compared to the average person, my self-esteem score is relatively low. d. I score higher than 60% of the other people who have taken the test.

Ans: D

Instructor Resource Furr, Psychometrics, 4e SAGE Publishing, 2022 Learning Objective: 3-7: Describe how test scores are represented, including z scores, standard scores, and percentile ranks.

Cognitive Domain: Analysis Answer Location: Percentile Ranks Difficulty Level: Medium 21. After taking this quiz, let’s say you learn that your percentile rank on the test is .95.

How can you interpret this value? a. You know 95% of the total possible level of “Knowledge of Psychological Testing.” b. Your score is .95 of a standard deviation above the mean. c. Compared to the average person, your score is relatively low. d. You score higher than 95% of the other people who have taken the quiz.

Ans: D Learning Objective: 3-7: Describe how test scores are represented, including z scores, standard scores, and percentile ranks.

Cognitive Domain: Analysis Answer Location: Percentile Ranks Difficulty Level: Medium 22. In what procedure must test developers must make two assumptions: levels of the psychological attribute are normally distributed, and the actual test data that they obtained in their sample are imperfect reflections of the distribution of the construct itself? a. normal distributions b. converted standard scores c. percentile ranks d. normalization transformations Ans: D Learning Objective: 3-7: Describe how test scores are represented, including z scores, standard scores, and percentile ranks.

Cognitive Domain: Comprehension Answer Location: Normalized Scores Difficulty Level: Medium 23. When creating a test, the test creator will sometimes administer the test to a large group of people to develop test norms. That group is called a _____________. a. reference sample b. variance group c. covariance group d. herd Ans: A Learning Objective: 3-8: Discuss how test developers “norm” tests to interpret the scores.

Cognitive Domain: Comprehension Answer Location: Test Norms Difficulty Level: Medium

Instructor Resource Furr, Psychometrics, 4e SAGE Publishing, 2022 24. What is the value of having test norms? a. The test norms reflect the one score that would be expected from a normative (i.e., typical) respondent. b. The test norms reflect the one score that would be expected from a normal (i.e., non-pathological) respondent. c. Any new test-taker can be compared to those test norms as a frame of reference for interpreting her/his score. d. Test norms allow researchers to use a test in their studies, as they allow researchers to compute correlations between the test’s scores and other variables of interest.

Ans: C Learning Objective: 3-8: Discuss how test developers “norm” tests to interpret the scores.

Cognitive Domain: Comprehension Answer Location: Test Norms Difficulty Level: Medium 25. What refers to the use of certain procedures that ensure a representative sample? a. probability sampling b. nonprobability sampling c. correlation coefficient d. normal distribution Ans: A Learning Objective: 3-8: Discuss how test developers “norm” tests to interpret the scores.

Cognitive Domain: Knowledge Answer Location: Representativeness of the Reference Sample Difficulty Level: Easy

True/False 1. The numerator of the variance is the “sum of squares.”

Ans: T Learning Objective: 3-3: Summarize the importance of central tendency, variability, and distributions in statistics.

Cognitive Domain: Comprehension Answer Location: Variability Difficulty Level: Medium 2. The variance is viewed as more intuitive than standard deviation.

Ans: F Learning Objective: 3-3: Summarize the importance of central tendency, variability, and distributions in statistics.

Cognitive Domain: Analysis

Instructor Resource Furr, Psychometrics, 4e SAGE Publishing, 2022 Answer Location: Variability Difficulty Level: Medium 3. Standard deviation can never be less than 0, but variance can be less than 0.

Ans: F Learning Objective: 3-3: Summarize the importance of central tendency, variability, and distributions in statistics.

Cognitive Domain: Analysis Answer Location: Variability Difficulty Level: Medium 4. When a person obtains a relatively high score on one variable and then again on another, the variables have a negative association.

Ans: F Learning Objective: 3-4: Describe how associations are made between variables.

Cognitive Domain: Comprehension Answer Location: Interpreting the Association Between Two Variables Difficulty Level: Medium 5. A scatterplot represents the association between two variables in terms of an upward or downward trend of points.

Ans: T Learning Objective: 3-4: Describe how associations are made between variables.

Cognitive Domain: Knowledge Answer Location: Scatterplots: Visually Representing the Association Between Two Variables Difficulty Level: Easy 6. The variance of a binary item is maximized when half the people provide a positively valenced response and the other have provided a negatively valenced response.

Ans: T Learning Objective: 3-6: Identify what a binary item is and how tests based on binary items are scored.

Cognitive Domain: Comprehension Answer Location: Binary Items Difficulty Level: Medium 7. Z score = 0 is the mean.

Ans: T Learning Objective: 3-7: Describe how test scores are represented, including z scores, standard scores, and percentile ranks.

Cognitive Domain: Comprehension Answer Location: z Scores (Standard Scores) Difficulty Level: Medium 8. Z score < 0 is above the mean.

Instructor Resource Furr, Psychometrics, 4e SAGE Publishing, 2022 Ans: F Learning Objective: 3-7: Describe how test scores are represented, including z scores, standard scores, and percentile ranks.

Cognitive Domain: Comprehension Answer Location: z Scores (Standard Scores) Difficulty Level: Medium 9. Z score < 0 is below the mean.

Ans: T Learning Objective: 3-7: Describe how test scores are represented, including z scores, standard scores, and percentile ranks.

Cognitive Domain: Comprehension Answer Location: z Scores (Standard Scores) Difficulty Level: Medium 10. Z score = 0 is the same as absolute value.

Ans: F Learning Objective: 3-7: Describe how test scores are represented, including z scores, standard scores, and percentile ranks.

Cognitive Domain: Comprehension Answer Location: z Scores (Standard Scores) Difficulty Level: Medium

Short Answer 1. Which is more central to the point of psychological testing and assessment? 1) To detect psychological differences, or 2) to reduce an individual’s entire character to one number?

Ans: Answer may vary.

Learning Objective: 3-1: Describe the implicit and explicit nature of variability found in psychological measurement.

Cognitive Domain: Analysis Answer Location: The Nature of Variability Difficulty Level: Medium 2. What is the term for a set of scores (e.g. a set of test scores)?

Ans: Answer may vary.

Learning Objective: 3-2: Explain why individual differences are fundamental to psychological measurement.

Cognitive Domain: Knowledge Answer Location: Importance of Individual Differences Difficulty Level: Easy 3. What is the conceptual term meaning “differences”?

Instructor Resource Furr, Psychometrics, 4e SAGE Publishing, 2022 Ans: Answer may vary.

Learning Objective: 3-3: Summarize the importance of central tendency, variability, and distributions in statistics.

Cognitive Domain: Knowledge Answer Location: Variability and Distribution of Scores Difficulty Level: Easy 4. What is the term for a statistical value used to quantify the degree of differences?

Ans: Answer may vary.

Learning Objective: 3-3: Summarize the importance of central tendency, variability, and distributions in statistics.

Cognitive Domain: Knowledge Answer Location: Variability and Distribution of Scores Difficulty Level: Easy 5. What is a conceptual term for the typical score in a distribution?

Ans: Answer may vary.

Learning Objective: 3-3: Summarize the importance of central tendency, variability, and distributions in statistics.

Cognitive Domain: Knowledge Answer Location: Central Tendency Difficulty Level: Easy 6. What is a statistical value used to quantitatively represent the typical score in a distribution?

Ans: Answer may vary.

Learning Objective: 3-3: Summarize the importance of central tendency, variability, and distributions in statistics.

Cognitive Domain: Comprehension Answer Location: Central Tendency Difficulty Level: Medium 7. What are the two most commonly used indexes of variability in behavioral research— lying at the heart of psychometric theory in particular?

Ans: Answer may vary.

Learning Objective: 3-3: Summarize the importance of central tendency, variability, and distributions in statistics.

Cognitive Domain: Comprehension Answer Location: Variability Difficulty Level: Medium 8. What is the conceptual term for the consistency between two distributions of scores?

Learning Objective: 3-4: Describe how associations are made between variables.

Ans: Answer may vary.

Cognitive Domain: Knowledge

Instructor Resource Furr, Psychometrics, 4e SAGE Publishing, 2022 Answer Location: Quantifying the Association or Consistency Between Distributions Difficulty Level: Easy 9. What is the term for a statistical value used to quantify the consistency between two distributions of scores?

Ans: Answer may vary.

Learning Objective: 3-4: Describe how associations are made between variables.

Cognitive Domain: Knowledge Answer Location: Covariance Difficulty Level: Easy

Instructor Resource Furr, Psychometrics, 4e SAGE Publishing, 2022 Chapter 4: Test Dimensionality and Factor Analysis Test Bank Multiple Choice 1. What is another way to think about a dimension? That it is a _________. a. test b. item c. trait d. response Ans: C Learning Objective: 4-1: Understand the different types of dimensionality tests and their implications for scoring and psychometric evaluation.

Cognitive Domain: Knowledge Answer Location: Test Dimensionality Difficulty Level: Easy 2. Consider the figure below. Which of the following is the best meaning for the circle?

a. a psychological dimension b. a response to a test item c. the assumption that the trait affects responses to test items d. the assumption that the item affects the trait Ans: A Learning Objective: 4-1: Understand the different types of dimensionality tests and their implications for scoring and psychometric evaluation.

Cognitive Domain: Comprehension Answer Location: Unidimensional Tests Difficulty Level: Medium 3. Consider the figure below. Which of the following is the best meaning for the box?

Instructor Resource Furr, Psychometrics, 4e SAGE Publishing, 2022

a. a psychological dimension b. a response to a test item c. the assumption that the trait affects responses to test items d. the assumption that the item affects the trait Ans: B Learning Objective: 4-1: Understand the different types of dimensionality tests and their implications for scoring and psychometric evaluation.

Cognitive Domain: Comprehension Answer Location: Unidimensional Tests Difficulty Level: Medium 4. Consider the figure below. Which of the following is the best meaning for the arrows’ directionality?

a. a psychological dimension b. a response to a test item c. the assumption that the trait affects responses to test items d. the assumption that the item affects the trait Ans: C Learning Objective: 4-1: Understand the different types of dimensionality tests and their implications for scoring and psychometric evaluation.

Cognitive Domain: Comprehension Answer Location: Unidimensional Tests Difficulty Level: Medium 5. Which kind of test has one and only one score? a. unidimensional b. multidimensional with correlated dimensions c. multidimensional with uncorrelated dimensions d. hyperdimensional with redundant dimensions Ans: A Learning Objective: 4-1: Understand the different types of dimensionality tests and their implications for scoring and psychometric evaluation.

Cognitive Domain: Knowledge Answer Location: Unidimensional Tests Difficulty Level: Easy

Instructor Resource Furr, Psychometrics, 4e SAGE Publishing, 2022 6. What kind of test (in terms of dimensionality) is best represented by the figure below?

a. unidimensional b. multidimensional with correlated dimensions c. multidimensional with uncorrelated dimensions d. hyperdimensional with redundant dimensions Ans: B Learning Objective: 4-1: Understand the different types of dimensionality tests and their implications for scoring and psychometric evaluation.

Cognitive Domain: Comprehension Answer Location: Multidimensional Test With Correlated Dimensions (Tests With Higher-Order Factors) Difficulty Level: Medium 7. When responses to a test’s items reflect more than one psychological dimension, the test is ___________. a. unidimensional b. hyperdimensional c. multidimensional d. psychologically invalid Ans: C Learning Objective: 4-1: Understand the different types of dimensionality tests and their implications for scoring and psychometric evaluation.

Cognitive Domain: Comprehension Answer Location: Multidimensional Tests With Correlated Dimensions (Tests With Higher-Order Factors) Difficulty Level: Medium 8. Recall the BFI (Big Five Inventory, with scores representing Neuroticism, Extraversion, Openness to Experience, etc.). What type of test was that? Recall that we computed 5 and only 5 scores. a. unidimensional b. multidimensional with correlated dimensions c. multidimensional with uncorrelated dimensions d. hyperdimensional with redundant dimensions Ans: C Learning Objective: 4-1: Understand the different types of dimensionality tests and their implications for scoring and psychometric evaluation.

Cognitive Domain: Analysis

Instructor Resource Furr, Psychometrics, 4e SAGE Publishing, 2022 Answer Location: Multidimensional Tests With Uncorrelated Dimensions Difficulty Level: Medium 9. EFA is most directly based on: a. correlations among items b. differences between groups of people c. a conceptual analysis of item meaning d. respondents’ comments about their view on test dimensionality Ans: A Learning Objective: 4-4: Describe factors, factor loadings, and rotations, and how they offer deeper insight into factor analysis.

Cognitive Domain: Comprehension Answer Location: A Deeper Perspective on Factors, Factor Loadings, and Rotation Difficulty Level: Medium 10. Imagine that you are examining a test, and you find that responses to one item are highly correlated with responses on another item. This finding indicates: a. The two items likely reflect the same factor. b. The two items likely reflect different factors. c. Responses to the first item are likely affected by the same psychological attribute that affects responses to the second item. d. There were different groups of people that answered each item.

Ans: A Learning Objective: 4-1: Understand the different types of dimensionality tests and their implications for scoring and psychometric evaluation.

Cognitive Domain: Application Answer Location: Multidimensional Tests With Correlated Dimensions (Tests With Higher-Order Factors) Difficulty Level: Hard 11. Which kind of test is often scored in a way that produces a total score, combined across several subsets, which are often combined? a. unidimensional b. multidimensional with correlated dimensions c. multidimensional with uncorrelated dimensions d. hyperdimensional with redundant dimensions Ans: B Learning Objective: 4-1: Understand the different types of dimensionality tests and their implications for scoring and psychometric evaluation.

Cognitive Domain: Comprehension Answer Location: Multidimensional Tests With Correlated Dimensions (Tests With Higher-Order Factors) Difficulty Level: Medium 12. "Higher-order" and "second-order" structures are most likely associated with which kind of test?

Instructor Resource Furr, Psychometrics, 4e SAGE Publishing, 2022 a. unidimensional b. multidimensional with correlated dimensions c. multidimensional with uncorrelated dimensions d. hyperdimensional with redundant dimensions Ans: B Learning Objective: 4-1: Understand the different types of dimensionality tests and their implications for scoring and psychometric evaluation.

Cognitive Domain: Comprehension Answer Location: Multidimensional Tests With Correlated Dimensions (Tests With Higher-Order Factors) Difficulty Level: Medium 13. In terms of scoring a test, what is a key difference between the two kinds of multidimensional tests? a. For a multidimensional test with correlated dimensions, the subscales can be combined into a total scale score, but this is not true for a multidimensional test with uncorrelated dimensions. b. For a multidimensional test with uncorrelated dimensions, the subscales can be combined into a total scale score, but this is not true for a multidimensional test with correlated dimensions. c. For a multidimensional test with correlated dimensions, each subscale should be examined separately from the others, but this is not true for a multidimensional test with uncorrelated dimensions. d. For a multidimensional test with uncorrelated dimensions, each subscale should be examined separately from the others; however, for a multi-dimensional test with correlated dimensions, the subscales should always be combined and not examined separately.

Ans: A Learning Objective: 4-1: Understand the different types of dimensionality tests and their implications for scoring and psychometric evaluation.

Cognitive Domain: Analysis Answer Location: Multidimensional Tests With Uncorrelated Dimensions Difficulty Level: Medium 14. Which kind of test is combined across several subsets, with dimensions that are not associated with each other (or only weakly associated)? a. unidimensional b. multidimensional with correlated dimensions c. multidimensional with uncorrelated dimensions d. hyperdimensional with redundant dimensions Ans: C Learning Objective: 4-1: Understand the different types of dimensionality tests and their implications for scoring and psychometric evaluation.

Cognitive Domain: Knowledge

Instructor Resource Furr, Psychometrics, 4e SAGE Publishing, 2022 Answer Location: Multidimensional Tests With Uncorrelated Dimensions Difficulty Level: Easy 15. By identifying sets of items that go together, items that are relatively strongly correlated with each other but weakly correlated with other items, what are we able to understand? a. test dimensionality b. factor loadings c. rotations d. scree plots Ans: A Learning Objective: 4-2: Explain how test developers use factor analysis in order to address the core questions in the three dimensionality tests.

Cognitive Domain: Comprehension Answer Location: Factor Analysis: Examining the Dimensionality of a Test Difficulty Level: Medium 16. Which of the following is the statistical tool most commonly used to test dimensionality? a. scree plots b. cluster analysis c. multidimensional scaling d. factor analysis Ans: D Learning Objective: 4-2: Explain how test developers use factor analysis in order to address the core questions in the three dimensionality tests.

Cognitive Domain: Comprehension Answer Location: Factor Analysis: Examining the Dimensionality of a Test Difficulty Level: Medium 17. "Exploratory" and "confirmatory" are two types of what way to examine the dimensionality of a test? a. scree plots b. cluster analysis c. multidimensional scaling d. factor analysis Ans: D Learning Objective: 4-2: Explain how test developers use factor analysis in order to address the core questions in the three dimensionality tests.

Cognitive Domain: Comprehension Answer Location: Factor Analysis: Examining the Dimensionality of a Test Difficulty Level: Medium 18. In an EFA, what does a “Factor” represent? a. a psychological trait or dimension b. responses to an item

Instructor Resource Furr, Psychometrics, 4e SAGE Publishing, 2022 c. clusters of participants d. something that two items do NOT have in common with each other Ans: A Learning Objective: 4-3: Summarize the process of conducting an exploratory factor analysis.

Cognitive Domain: Comprehension Answer Location: Conducting and Interpreting an Exploratory Factor Analysis Difficulty Level: Medium 19. When conducting an Exploratory Factor Analysis and attempting to understand the psychological meaning of each factor, researchers usually rely upon: a. tests of statistical significance b. reasoned judgment, based on factor loadings c. the “flattening-point” of a scree test d. inferring what non-loading tests share in common Ans: B Learning Objective: 4-3: Summarize the process of conducting an exploratory factor analysis.

Cognitive Domain: Analysis Answer Location: Conducting and Interpreting an Exploratory Factor Analysis Difficulty Level: Medium 20. Imagine that you conducted an EFA of a 16-item test, and you obtained the scree plot below. Based on this, how many factors does the test seem to have?

7 6 5 4 3 2 1 0 1

2

3

4

5

6

7

8

9 10 11 12 13 14 15 16

a. 7 b. 16 c. 3 d. 1 Ans: C Learning Objective: 4-3: Summarize the process of conducting an exploratory factor analysis.

Cognitive Domain: Application Answer Location: Conducting and Interpreting an Exploratory Factor Analysis Difficulty Level: Hard

Instructor Resource Furr, Psychometrics, 4e SAGE Publishing, 2022 21. You examine the dimensionality of a nine-item test intended to assess relationship satisfaction. After doing an EFA, you get the following set of factor loadings. Which is the most appropriate statement below?

Item

F1

F2

1

I don't like being intimate with my partner.

.958

.003

2

I don't feel I can be .840 close to my partner.

.015

3

I am nervous about getting close to my partner.

.800

-.123

4

I am nervous about opening up to my partner.

.798

.132

5

I feel comfortable .750 about being close to my partner.

-.078

6

I am very happy with -.143 my partner.

.898

7

I feel reassured by my partner.

.111

.765

8

I enjoy my relationship with my partner.

.092

.703

9

I don't worry about my relationship with my partner.

-.003

.698

a. The on-factor loadings are too high (note, “on-factor” here means an item’s largest factor loading). b. The on-factor loadings are generally lower than we’d like. c. There’s a problem: none of the factor loadings are significant at p <.05. d. There’s likely a mistake—an item loading on F1 should probably have a negative loading on that factor.

Ans: D Learning Objective: 4-3: Summarize the process of conducting an exploratory factor analysis.

Cognitive Domain: Application

Instructor Resource Furr, Psychometrics, 4e SAGE Publishing, 2022 Answer Location: Conducting and Interpreting an Exploratory Factor Analysis Difficulty Level: Hard 22. Which of the following are oblique rotations? a. varimax and promax b. quartimax and oblimin c. varimax and equamax d. promax and simplimax Ans: D Learning Objective: 4-3: Summarize the process of conducting an exploratory factor analysis.

Cognitive Domain: Application Answer Location: Conducting and Interpreting an Exploratory Factor Analysis Difficulty Level: Hard 23. Imagine that you conduct an EFA of responses to a set of test items, and you find a two-factor structure, with an inter-factor correlation of .68. Based on these results, what would you conclude about the dimensionality of your test? a. unidimensional b. multidimensional with correlated dimensions c. multidimensional with uncorrelated dimensions d. hyperdimensional with redundant dimensions Ans: B Learning Objective: 4-4: Describe factors, factor loadings, and rotations, and how they offer deeper insight into factor analysis.

Cognitive Domain: Analysis Answer Location: Conducting and Interpreting an Exploratory Factor Analysis Difficulty Level: Medium 24. Imagine that you conduct an EFA of responses to a set of test items, and you find a two-factor structure, with an inter-factor correlation of .00. Based on these results, what would you conclude about the dimensionality of your test? a. unidimensional b. multidimensional with correlated dimensions c. multidimensional with uncorrelated dimensions d. hyperdimensional with redundant dimensions Ans: C Learning Objective: 4-4: Describe factors, factor loadings, and rotations, and how they offer deeper insight into factor analysis.

Cognitive Domain: Analysis Answer Location: Conducting and Interpreting an Exploratory Factor Analysis Difficulty Level: Medium 25. The factors extracted in factor analysis are made more clear and interpretable by:

Instructor Resource Furr, Psychometrics, 4e SAGE Publishing, 2022 a. converting raw scores to z scores before analysis b. adding variables that have low correlations with other variables c. applying a square root transformation to the raw data before analysis d. statistically rotating factors Ans: D Learning Objective: 4-4: Describe factors, factor loadings, and rotations, and how they offer deeper insight into factor analysis.

Cognitive Domain: Analysis Answer Location: A Deeper Perspective on Factors, Factor Loadings, and Rotation Difficulty Level: Medium True/False 1. It is psychometrically problematic if a test includes some items that reflect one dimension and other items that reflect another dimension. For example, it’s problematic if a test includes some items reflecting depression and other items reflecting anxiety.

Ans: F Learning Objective: 4-1: Understand the different types of dimensionality tests and their implications for scoring and psychometric evaluation.

Cognitive Domain: Analysis Answer Location: Test Dimensionality Difficulty Level: Medium 2. It is psychometrically ideal for each item (on a test) to reflect only one dimension, and problematic if an item is affected by more than one dimension. For example, it’s problematic if responses to Item 1 on a test are affected by both depression and anxiety.

Ans: T Learning Objective: 4-1: Understand the different types of dimensionality tests and their implications for scoring and psychometric evaluation.

Cognitive Domain: Analysis Answer Location: Test Dimensionality Difficulty Level: Medium 3. No psychological test should produce more than a single score for each respondent.

Ans: F Learning Objective: 4-1: Understand the different types of dimensionality tests and their implications for scoring and psychometric evaluation.

Cognitive Domain: Comprehension Answer Location: Test Dimensionality Difficulty Level: Medium 4. A score in a psychological test should represent a single psychological feature or attribute.

Ans: T

Instructor Resource Furr, Psychometrics, 4e SAGE Publishing, 2022 Learning Objective: 4-1: Understand the different types of dimensionality tests and their implications for scoring and psychometric evaluation.

Cognitive Domain: Knowledge Answer Location: Test Dimensionality Difficulty Level: Easy 5. For a unidimensional test, we would have one score for each respondent.

Ans: T Learning Objective: 4-1: Understand the different types of dimensionality tests and their implications for scoring and psychometric evaluation.

Cognitive Domain: Knowledge Answer Location: Unidimensional Tests Difficulty Level: Easy 6. When evaluating the psychometric quality of a test, we should evaluate the psychometric quality for each scale score on the test.

Ans: T Learning Objective: 4-1: Understand the different types of dimensionality tests and their implications for scoring and psychometric evaluation.

Cognitive Domain: Comprehension Answer Location: Multidimensional Tests With Correlated Dimensions (Tests With Higher-Order Factors) Difficulty Level: Medium 7. For a multidimensional test, we would have one score for each respondent.

Ans: F Learning Objective: 4-1: Understand the different types of dimensionality tests and their implications for scoring and psychometric evaluation.

Cognitive Domain: Knowledge Answer Location: Multidimensional Tests With Correlated Dimensions (Tests With Higher-Order Factors) Difficulty Level: Easy 8. It is theoretically possible for a test to produce some scale/dimension scores that have good psychometric quality and other scale/dimension scores that have poor psychometric quality.

Ans: T Learning Objective: 4-1: Understand the different types of dimensionality tests and their implications for scoring and psychometric evaluation.

Cognitive Domain: Analysis Answer Location: Multidimensional Tests With Correlated Dimensions (Tests With Higher-Order Factors) Difficulty Level: Medium 9. Each subtest in a multidimensional test is, itself, unidimensional.

Ans: T

Instructor Resource Furr, Psychometrics, 4e SAGE Publishing, 2022 Learning Objective: 4-1: Understand the different types of dimensionality tests and their implications for scoring and psychometric evaluation.

Cognitive Domain: Comprehension Answer Location: Answer Location: Multidimensional Tests With Correlated Dimensions (Tests With Higher-Order Factors) Difficulty Level: Medium 10. When scoring a multidimensional test that has uncorrelated dimensions, each score should reflect only a single dimension.

Ans: T Learning Objective: 4-1: Understand the different types of dimensionality tests and their implications for scoring and psychometric evaluation.

Cognitive Domain: Knowledge Answer Location: Multidimensional Tests With Uncorrelated Dimensions Difficulty Level: Easy Short Answer 1. What is one of the three key questions regarding test dimensionality?

Ans: Answer may vary.

Learning Objective: 4-1: Understand the different types of dimensionality tests and their implications for scoring and psychometric evaluation.

Cognitive Domain: Application Answer Location: Three Dimensionality Questions: What They Are and Why They Matter Difficulty Level: Hard 2. What type of psychological test includes items that reflect only a single attribute of a person, a single psychological dimension?

Ans: Answer may vary.

Learning Objective: 4-1: Understand the different types of dimensionality tests and their implications for scoring and psychometric evaluation.

Cognitive Domain: Knowledge Answer Location: Unidimensional Tests Difficulty Level: Easy 3. What kind of test would you classify intelligence tests, like the Wechsler Intelligence Scale for Children?

Ans: Answer may vary.

Learning Objective: 4-1: Understand the different types of dimensionality tests and their implications for scoring and psychometric evaluation.

Cognitive Domain: Analysis Answer Location: Answer Location: Multidimensional Tests With Correlated Dimensions (Tests With Higher-Order Factors) Difficulty Level: Medium

Instructor Resource Furr, Psychometrics, 4e SAGE Publishing, 2022 4. Most traditional IQ tests are built upon the idea that there is an ability, called “g” or “general intelligence,” that affects people’s scores on a range of cognitive tasks. This is why those tests produce a single “IQ score” that we’re all familiar with.

A few years ago, researchers conducted an EFA of scores on 12 cognitive tasks (N = 44,600 participants). They state that their analysis revealed a three-factor structure among these tasks. This apparent finding is clearly at odds with the traditional view that one single factor (i.e., “general intelligence”) accounts for much of the covariation among diverse sets of cognitive tasks. Based on this finding, the researchers concluded that the idea that there is a single general factor of intelligence is “a myth” and is simply incorrect.

This is a provocative claim—if it’s true, then we should dramatically rethink our view of intelligence and intelligence testing. In an online supplement to the article, the researcher reported the 12 eigenvalues from their EFA. Using these, we can sketch out the following scree plot: 3.5 3 2.5 2 1.5 1 0.5 0 1

2

3

4

5

6

7

8

9

10

11

12

Looking at the scree plot, how many factors would you say there are?

Ans: Answer may vary.

Learning Objective: 4-3: Summarize the process of conducting an exploratory factor analysis.

Cognitive Domain: Analysis Answer Location: Conducting and Interpreting an Exploratory Factor Analysis Difficulty Level: Medium 5. Statistical values that typically range between -1 and +1 and reflect the degree/direction of connection between an item and a factor are best labeled as what term?

Ans: Answer may vary.

Learning Objective: 4-3: Summarize the process of conducting an exploratory factor analysis.

Cognitive Domain: Knowledge Answer Location: Conducting and Interpreting an Exploratory Factor Analysis Difficulty Level: Easy

Instructor Resource Furr, Psychometrics, 4e SAGE Publishing, 2022 6. Statistical values that are often used to understand the number of factors when conducting an EFA are best labeled as what term?

Ans: Answer may vary.

Learning Objective: 4-3: Summarize the process of conducting an exploratory factor analysis.

Cognitive Domain: Knowledge Answer Location: Conducting and Interpreting an Exploratory Factor Analysis Difficulty Level: Easy 7. A figure that presents the magnitudes of eigenvalues is best labeled as what term?

Ans: Answer may vary.

Learning Objective: 4-3: Summarize the process of conducting an exploratory factor analysis.

Cognitive Domain: Knowledge Answer Location: Conducting and Interpreting an Exploratory Factor Analysis Difficulty Level: Easy 8. When the results of a factor analysis indicate that each item is strongly connected to one and only one factor, it is called what?

Ans: Answer may vary.

Learning Objective: 4-3: Summarize the process of conducting an exploratory factor analysis.

Cognitive Domain: Knowledge Answer Location: Conducting and Interpreting an Exploratory Factor Analysis Difficulty Level: Easy 9. The particular statistical/mathematical procedure that you might use when beginning an EFA is best labeled as what term?

Ans: Answer may vary.

Learning Objective: 4-3: Summarize the process of conducting an exploratory factor analysis.

Cognitive Domain: Knowledge Answer Location: Conducting and Interpreting an Exploratory Factor Analysis Difficulty Level: Easy 10. The procedure by which you might clarify the meaning of factors, by achieving simple structure, is best labeled as what term?

Ans: Answer may vary.

Learning Objective: 4-4: Describe factors, factor loadings, and rotations, and how they offer deeper insight into factor analysis.

Cognitive Domain: Knowledge Answer Location: A Deeper Perspective on Factors, Factor Loadings, and Rotation Difficulty Level: Easy

Instructor Resource Furr, Psychometrics, 4e SAGE Publishing, 2022 Chapter 5: Reliability Test Bank Multiple Choice 1. Conceptually, what does reliability refer to? a. the degree to which differences in test scores are consistent with differences in true scores b. the degree to which we correctly interpret the meaning of test scores c. the degree to which a test is unbiased and works equally well for all groups of respondents d. the raw amount of true differences among respondents Ans: A Learning Objective: 5-1: Describe reliability according to classical test theory.

Cognitive Domain: Knowledge Answer Location: Overview of Reliability and Classical Test Theory Difficulty Level: Easy 2. There are two key assumptions of Classical Test Theory. What is one of them? a. true scores are determined additively by Observed Score and Measurement Error b. measurement error is nonrandom/systematic c. measurement error is random/unsystematic d. the measure in question is a valid representation of the underlying latent variable Ans: C Learning Objective: 5-2: Differentiate between observed scores, true scores, and measurement error.

Cognitive Domain: Comprehension Answer Location: Observed Scores, True Scores, and Measurement Error Difficulty Level: Medium 3. As an implication of one of the assumptions of classical test theory, what is the correlation between true scores on a test and error scores on that test? a. 0 b. 1 c. .5 d. unknown, it depends on the test Ans: A Learning Objective: 5-2: Differentiate between observed scores, true scores, and measurement error.

Cognitive Domain: Analysis Answer Location: Observed Scores, True Scores, and Measurement Error Difficulty Level: Medium

Instructor Resource Furr, Psychometrics, 4e SAGE Publishing, 2022 4. In terms of individuals’ observed score, measurement error will tend to make: a. each individual’s score smaller than it should be b. each individual’s score larger than it should be c. the observed scores imprecise estimates of the true scores d. the observed scores precise, true scores Ans: C Learning Objective: 5-2: Differentiate between observed scores, true scores, and measurement error.

Cognitive Domain: Comprehension Answer Location: Observed Scores, True Scores, and Measurement Error Difficulty Level: Medium 5. Let’s say that Amy’s true score for a self-esteem test was 45, and Barry’s true score was 40. If the test scores have good reliability, what is the difference that we should see between Amy’s and Barry’s observed test scores? a. Amy’s observed score will be exactly equal to Barry’s. b. Amy’s observed score will be 5 points less than Barry’s. c. Amy’s observed score will be 5 points higher than Barry’s. d. Amy’s observed score will be 10 points higher than Barry’s.

Ans: C Learning Objective: 5-2: Differentiate between observed scores, true scores, and measurement error.

Cognitive Domain: Analysis Answer Location: Observed Scores, True Scores, and Measurement Error Difficulty Level: Medium 6. Imagine you were told that a new test had a reliability of .87. How should this be interpreted? a. This is a good level of reliability. b. This is a poor level of reliability. c. It depends, depending on the test and its purpose, this could good or poor. d. This level of reliability is theoretically impossible.

Ans: A Learning Objective: 5-3: Summarize how reliability depends on the variance in observed scores, true scores, and measurement error.

Cognitive Domain: Analysis Answer Location: Variances in Observed Scores, True Scores, and Error Scores Difficulty Level: Medium 7. Let’s say that you administered a test to a set of respondents. You then improved the test in a way that would ultimately improve its reliability. You then administered the revised test to the same set of original respondents. According to CTT, which of the following would happen? a. increase true score variance b. decrease error variance

Instructor Resource Furr, Psychometrics, 4e SAGE Publishing, 2022 c. increase the correlation between true scores and error scores d. increase the average error score Ans: B Learning Objective: 5-3: Summarize how reliability depends on the variance in observed scores, true scores, and measurement error.

Cognitive Domain: Analysis Answer Location: Variances in Observed Scores, True Scores, and Error Scores Difficulty Level: Medium 8. Say that I develop a test, and I try to convince you that it is a measure of intelligence.

You then gather responses to the test, estimate reliability, and find that those test scores have high (estimated) reliability. What can we conclude about those scores, assuming that they do indeed have high reliability? a. the test is unbiased and works equally well for all groups of respondents b. responses to the test items were not strongly affected by measurement error c. the test’s (observed) scores are uncorrelated with other scores that they should be uncorrelated with d. The test is a good (precise) measure of intelligence Ans: B Learning Objective: 5-3: Summarize how reliability depends on the variance in observed scores, true scores, and measurement error.

Cognitive Domain: Analysis Answer Location: Variances in Observed Scores, True Scores, and Error Scores Difficulty Level: Medium 9. More technically, reliability can be defined as: a. the ratio of error score variance to observed score variance b. the (squared) correlation between observed scores and true scores c. the correlation between true scores and error scores d. the correlation between true scores on one test and true scores on another test Ans: B Learning Objective: 5-4: Identify the four ways to think about reliability in classical test theory.

Cognitive Domain: Knowledge Answer Location: Four Ways to Think of Reliability Difficulty Level: Easy 10. A true score variance of 0 means a reliability of _____. a. -1 b. 0 c. 1 d. the value cannot be determined with this information Ans: B Learning Objective: 5-4: Identify the four ways to think about reliability in classical test theory.

Cognitive Domain: Comprehension

Instructor Resource Furr, Psychometrics, 4e SAGE Publishing, 2022 Answer Location: Reliability as the Ratio of True Score Variance to Observed Score Variance Difficulty Level: Medium 11. What is perfect reliability? a. -1 b. 0 c. 1 d. there is no such thing Ans: C Learning Objective: 5-4: Identify the four ways to think about reliability in classical test theory.

Cognitive Domain: Comprehension Answer Location: Reliability as the Ratio of True Score Variance to Observed Score Variance Difficulty Level: Medium 12. Which of the following is a reliability score you would most be satisfied with? a. .40 b. .55 c. .70 d. .85 Ans: D Learning Objective: 5-4: Identify the four ways to think about reliability in classical test theory.

Cognitive Domain: Application Answer Location: Reliability as the Ratio of True Score Variance to Observed Score Variance Difficulty Level: Hard 13. What term represents the degree to which error affects different people in different ways? a. true score b. reliability c. error variance d. parallel test Ans: C Learning Objective: 5-4: Identify the four ways to think about reliability in classical test theory.

Cognitive Domain: Knowledge Answer Location: Reliability as Lack of Error Variance Difficulty Level: Easy 14. What is the index of reliability? a. the coefficient of reliability b. the error variance

Instructor Resource Furr, Psychometrics, 4e SAGE Publishing, 2022 c. the standard error of measurement d. classical test theory Ans: A Learning Objective: 5-4: Identify the four ways to think about reliability in classical test theory.

Cognitive Domain: Knowledge Answer Location: Reliability as the (Squared) Correlation Between Observed Scores and True Scores Difficulty Level: Easy 15. If the correlation (roe) between observed scores and error scores is 0, then RXX will equal: a. 0 b. -1 c. 1 d. the value cannot be determined Ans: C Learning Objective: 5-4: Identify the four ways to think about reliability in classical test theory.

Cognitive Domain: Analysis Answer Location: Reliability as the Lack of (Squared) Correlation Between Observed Scores and Error Scores Difficulty Level: Medium 16. What is another name for the standard deviation of error scores? a. the ratio of true score variance b. the standard error of measurement c. the squared correlation error d. the standard correlated error Ans: B Learning Objective: 5-5: Determine how to find the size of measurement error.

Cognitive Domain: Comprehension Answer Location: Reliability and the Standard Error of Measurement Difficulty Level: Medium 17. The theory of reliability is framed in each of the following except what? a. true scores b. error scores c. observed scores d. parallel scores Ans: D Learning Objective: 5-6: Describe how to estimate the reliability and standard error of measurement for a set of scores.

Cognitive Domain: Analysis

Instructor Resource Furr, Psychometrics, 4e SAGE Publishing, 2022 Answer Location: From Theory to Practice: Measurement Models and Their Implications for Estimating Reliability Difficulty Level: Medium 18. In which of the following test models do we not find that st21 = st22 ? a. parallel b. tau-equivalent c. essentially tau-equivalent d. congeneric Ans: D Learning Objective: 5-6: Describe how to estimate the reliability and standard error of measurement for a set of scores.

Cognitive Domain: Analysis Answer Location: From Theory to Practice: Measurement Models and Their Implications for Estimating Reliability Difficulty Level: Medium 19. Which test model makes the most assumptions and least likely to apply to real data? a. parallel b. tau-equivalent c. essentially tau-equivalent d. congeneric Ans: A Learning Objective: 5-6: Describe how to estimate the reliability and standard error of measurement for a set of scores.

Cognitive Domain: Comprehension Answer Location: Overview of Key Assumptions Difficulty Level: Medium 20. Which of the following tests is the least restrictive? a. parallel b. tau-equivalent c. essentially tau-equivalent d. congeneric Ans: D Learning Objective: 5-6: Describe how to estimate the reliability and standard error of measurement for a set of scores.

Cognitive Domain: Analysis Answer Location: Congeneric Tests Difficulty Level: Medium 21. A pair of tests or items are most likely to fit into which model? a. parallel b. tau-equivalent c. essentially tau-equivalent

Instructor Resource Furr, Psychometrics, 4e SAGE Publishing, 2022 d. congeneric Ans: D Learning Objective: 5-6: Describe how to estimate the reliability and standard error of measurement for a set of scores.

Cognitive Domain: Comprehension Answer Location: Congeneric Tests Difficulty Level: Medium 22. Domain sampling theory is an alternative to: a. classical test theory b. standard deviation c. generalizability theory d. psychometric theory Ans: A Learning Objective: 5-7: Define domain sampling theory and why it is a common classical test theory.

Cognitive Domain: Comprehension Answer Location: Domain Sampling Theory Difficulty Level: Medium 23. With ____________, the practical examination of reliability rests on the assumption that it would be possible to create two tests that are at a minimum parallel to each other. a. classical test theory b. domain sampling theory c. generalizability theory d. psychometric theory Ans: A Learning Objective: 5-7: Define domain sampling theory and why it is a common classical test theory.

Cognitive Domain: Comprehension Answer Location: Domain Sampling Theory Difficulty Level: Medium 24. With ____________, you do not have to make the above assumption, but if you follow the logic of the theory, you will end up with parallel tests by fiat. a. classical test theory b. domain sampling theory c. generalizability theory d. psychometric theory Ans: B Learning Objective: 5-7: Define domain sampling theory and why it is a common classical test theory.

Cognitive Domain: Comprehension Answer Location: Domain Sampling Theory Difficulty Level: Medium

Instructor Resource Furr, Psychometrics, 4e SAGE Publishing, 2022 25. Which theory rests on the assumption that items on any particular test represent a sample from a large indefinite number or domain of potential test items? a. classical test theory b. domain sampling theory c. generalizability theory d. psychometric theory Ans: B Learning Objective: 5-7: Define domain sampling theory and why it is a common classical test theory.

Cognitive Domain: Comprehension Answer Location: Domain Sampling Theory Difficulty Level: Medium True/False 1. Score variance is the sum of true score variance and error variance.

Ans: T Learning Objective: 5-4: Identify the four ways to think about reliability in classical test theory.

Cognitive Domain: Knowledge Answer Location: Reliability as Lack of Error Variance Difficulty Level: Easy 2. When people refer to reliability, they typically are referring to the index of reliability.

Ans: F Learning Objective: 5-4: Identify the four ways to think about reliability in classical test theory.

Cognitive Domain: Comprehension Answer Location: Reliability as the (Squared) Correlation Between Observed Scores and True Scores Difficulty Level: Medium 3. If RXX = 1, then sem = 0.

Ans: T Learning Objective: 5-5: Determine how to find the size of measurement error.

Cognitive Domain: Comprehension Answer Location: Reliability and the Standard Error of Measurement Difficulty Level: Medium 4. We have no way of knowing people’s true scores on a psychological variable or the error associated with their test responses.

Ans: T Learning Objective: 5-6: Describe how to estimate the reliability and standard error of measurement for a set of scores.

Cognitive Domain: Analysis

Instructor Resource Furr, Psychometrics, 4e SAGE Publishing, 2022 Answer Location: From Theory to Practice: Measurement Models and Their Implications for Estimating Reliability Difficulty Level: Medium 5. An essentially tau-equivalent test model has alternate forms.

Ans: F Learning Objective: 5-6: Describe how to estimate the reliability and standard error of measurement for a set of scores.

Cognitive Domain: Comprehension Answer Location: From Theory to Practice: Measurement Models and Their Implications for Estimating Reliability Difficulty Level: Medium 6. The correlation between parallel tests is equal to reliability.

Ans: T Learning Objective: 5-6: Describe how to estimate the reliability and standard error of measurement for a set of scores.

Cognitive Domain: Comprehension Answer Location: Parallel Tests Difficulty Level: Medium 7. Two parallel tests can have identical variances but unidentical standard deviations.

Ans: F Learning Objective: 5-6: Describe how to estimate the reliability and standard error of measurement for a set of scores.

Cognitive Domain: Comprehension Answer Location: Parallel Tests Difficulty Level: Medium 8. To meet the criteria for tau-equivalent and essentially tau-equivalent tests, two tests need to have the same reliability.

Ans: F Learning Objective: 5-6: Describe how to estimate the reliability and standard error of measurement for a set of scores.

Cognitive Domain: Comprehension Answer Location: Tau-Equivalent and Essentially Tau-Equivalent Tests Difficulty Level: Medium 9. Errors can be correlated across tests.

Ans: T Learning Objective: 5-6: Describe how to estimate the reliability and standard error of measurement for a set of scores.

Cognitive Domain: Analysis Answer Location: Tests With Correlated Errors Difficulty Level: Medium

Instructor Resource Furr, Psychometrics, 4e SAGE Publishing, 2022 10. CFA allows test users to estimate the reliability of a test even in cases where error terms are correlated with each other.

Ans: T Learning Objective: 5-6: Describe how to estimate the reliability and standard error of measurement for a set of scores.

Cognitive Domain: Comprehension Answer Location: Tests With Correlated Errors Difficulty Level: Medium

Short Answer 1. What can reliability range from?

Ans: Answer may vary.

Learning Objective: 5-4: Identify the four ways to think about reliability in classical test theory.

Cognitive Domain: Knowledge Answer Location: Reliability as the Ratio of True Score Variance to Observed Score Variance Difficulty Level: Easy 2. What can the size of the reliability coefficient indicate in terms of scores?

Ans: Answer may vary.

Learning Objective: 5-4: Identify the four ways to think about reliability in classical test theory.

Cognitive Domain: Analysis Answer Location: Reliability as the Ratio of True Score Variance to Observed Score Variance Difficulty Level: Medium 3. What is the reliability range?

Ans: Answer may vary.

Learning Objective: 5-4: Identify the four ways to think about reliability in classical test theory.

Cognitive Domain: Comprehension Answer Location: Reliability as the Ratio of True Score Variance to Observed Score Variance Difficulty Level: Medium 4. What does a small degree of error variance indicate?

Ans: Answer may vary.

Learning Objective: 5-4: Identify the four ways to think about reliability in classical test theory.

Cognitive Domain: Analysis

Instructor Resource Furr, Psychometrics, 4e SAGE Publishing, 2022 Answer Location: Reliability as Lack of Error Variance Difficulty Level: Medium 5. What is it called when there is a correlation between observed scores and true scores?

Ans: Answer may vary.

Learning Objective: 5-4: Identify the four ways to think about reliability in classical test theory.

Cognitive Domain: Comprehension Answer Location: Reliability as the (Squared) Correlation Between Observed Scores and True Scores Difficulty Level: Medium 6. What do you get when you square the index of reliability?

Ans: Answer may vary.

Learning Objective: 5-4: Identify the four ways to think about reliability in classical test theory.

Cognitive Domain: Comprehension Answer Location: Reliability as the (Squared) Correlation Between Observed Scores and True Scores Difficulty Level: Medium 7. What is another name for the standard deviation of error scores?

Ans: Answer may vary.

Learning Objective: 5-5: Determine how to find the size of measurement error.

Cognitive Domain: Knowledge Answer Location: Reliability and the Standard Error of Measurement Difficulty Level: Easy 8. What is one of the assumptions that must hold true for a test to be “parallel”?

Ans: Answer may vary.

Learning Objective: 5-6: Describe how to estimate the reliability and standard error of measurement for a set of scores.

Cognitive Domain: Comprehension Answer Location: Parallel Tests Difficulty Level: Medium 9. What is one of the assumptions that must hold true for a test to meet the criteria for “tau-equivalent”?

Ans: Answer may vary.

Learning Objective: 5-6: Describe how to estimate the reliability and standard error of measurement for a set of scores.

Cognitive Domain: Comprehension Answer Location: Tau-Equivalent and Essentially Tau-Equivalent Tests Difficulty Level: Medium

Instructor Resource Furr, Psychometrics, 4e SAGE Publishing, 2022 10. What are the assumptions that must hold true for a test to meet the criteria for “essentially tau-equivalent”?

Ans: Answer may vary.

Learning Objective: 5-6: Describe how to estimate the reliability and standard error of measurement for a set of scores.

Cognitive Domain: Comprehension Answer Location: Tau-Equivalent and Essentially Tau-Equivalent Tests Difficulty Level: Medium

Instructor Resource Furr, Psychometrics, 4e SAGE Publishing, 2022 Chapter 6: Empirical Estimates of Reliability Test Bank Multiple Choice 1. From the “alternate forms” approach to reliability, how do you estimate reliability? a. Create two versions of the same test, administer both forms to a single group of participants, and correlate the scores from the two forms. The correlation is your estimate of reliability. b. Create two versions of the same test, administer both forms to a single group of participants, and correlate the scores from the two forms. Multiply the correlation by two (to reflect the two forms), and that is your estimate of reliability. c. Create two versions of the same test, administer both forms to a single group of participants, and compute the mean score on each version. The difference between the means reflects the reliability of the test. d. Create several alternate versions of the same test and determine which one is the best. Estimate the reliability of that test, and it’s the best estimate for the entire test.

Ans: A Learning Objective: 6-1: Summarize how alternate forms reliability is used for estimating the reliability of test scores.

Cognitive Domain: Comprehension Answer Location: Alternate Forms Reliability Difficulty Level: Medium 2. For the alternate forms to produce an accurate estimate of reliability, some things have to be true. What must be true for this approach to produce an accurate estimate of reliability? a. the versions must include the same items b. there must be carryover effects (in which error scores from one version are correlated with error scores from the other version) c. there must be *no* carryover effects (in which error scores from one version are correlated with error scores from the other version) d. error scores on each version must be specific Ans: C Learning Objective: 6-1: Summarize how alternate forms reliability is used for estimating the reliability of test scores.

Cognitive Domain: Comprehension Answer Location: Alternate Forms Reliability Difficulty Level: Medium 3. If error scores are positively correlated across the versions of a test (e.g., due to carryover effects), then the alternate forms approach will produce an estimate of reliability that is:

Instructor Resource Furr, Psychometrics, 4e SAGE Publishing, 2022 a. accurate b. too low (compared to the true reliability) c. too high (compared to the true reliability) d. unknowable Ans: C Learning Objective: 6-1: Summarize how alternate forms reliability is used for estimating the reliability of test scores.

Cognitive Domain: Comprehension Answer Location: Alternate Forms Reliability Difficulty Level: Medium 4. Which of the following is unlikely to be a problem that might limit the usefulness of the alternative forms approach to estimating reliability? a. it is difficult to create different test “forms” that have identical psychometric properties b. you never know for sure whether different test forms have identical psychometric properties c. it is difficult to have one set of participants take more than one “test” at a time d. you never know for sure whether different test forms are measuring the same true scores (i.e., whether they are measuring the same construct).

Ans: C Learning Objective: 6-1: Summarize how alternate forms reliability is used for estimating the reliability of test scores.

Cognitive Domain: Analysis Answer Location: Alternate Forms Reliability Difficulty Level: Medium 5. From the “test-retest forms” approach to reliability, how do you estimate reliability? a. Create two versions of the same test, administer both forms to a single group of participants, and correlate the scores from the two forms. The correlation is your estimate of reliability. b. Create one version of a test, administer it to a single group of participants at two different times, and correlate the scores from the two testing times. The correlation is your estimate of reliability. c. Create one version of a test, administer it to a single group of participants at two different times, and correlate the scores from the two testing times. Multiply the correlation by two (to reflect the two testing occasions), and that’s your estimate of reliability. d. Create one version of a test, administer it twice to a single group of participants, and compute the mean score on each testing occasion. The difference between the means reflects the reliability of the test.

Ans: B Learning Objective: 6-2: Describe the test-retest method of reliability and how it compares to alternate forms reliability.

Cognitive Domain: Comprehension

Instructor Resource Furr, Psychometrics, 4e SAGE Publishing, 2022 Answer Location: Test-Retest Reliability Difficulty Level: Medium 6. For the test-retest approach to produce an accurate estimate of reliability, some things have to be true. Which must hold true for this approach to produce an accurate estimate of reliability? a. respondents’ true scores must change between testing occasions b. scores from the testing occasions must have the same amount of error variance c. the test items must be switched from one occasion to another d. error scores from one testing occasion must be correlated with error scores from another testing occasion.

Ans: B Learning Objective: 6-2: Describe the test-retest method of reliability and how it compares to alternate forms reliability.

Cognitive Domain: Analysis Answer Location: Test-Retest Reliability Difficulty Level: Medium 7. Imagine that you are measuring the following characteristics. For which one would the test-retest approach probably not be a good method for estimating reliability? a. general intelligence b. extraversion (as a personality trait) c. height d. happiness (as a mood state) Ans: D Learning Objective: 6-2: Describe the test-retest method of reliability and how it compares to alternate forms reliability.

Cognitive Domain: Analysis Answer Location: Test-Retest Reliability Difficulty Level: Medium 8. Internal consistency approaches are the most common way in which people estimate reliability. Why is that? Which factor differentiates internal consistency approaches from alternate forms and test-retest approaches? a. The internal consistency approaches require only one sample of respondents. b. The internal consistency approaches require two versions of a test. c. The internal consistency approaches require only one occasion of testing. d. The internal consistency approaches are appropriate for tests that have more than one item.

Ans: C Learning Objective: 6-3: Discuss internal consistency reliability and how it is a practical alternative to alternate forms reliability.

Cognitive Domain: Analysis Answer Location: Internal Consistency Reliability Difficulty Level: Medium

Instructor Resource Furr, Psychometrics, 4e SAGE Publishing, 2022 9. What is a challenge that limits the utility of the split-half approach? a. There are multiple ways that you could split a sample of respondents in half, and each way could produce a different estimate of reliability. b. A test must have at least 20 items for this approach to produce an accurate estimate of reliability. c. For most tests, there are multiple ways that you could split the test in half, and each way could produce a different estimate of reliability. d. It’s not very useful to estimate the reliability of a “half-test”, because you want to know the reliability of the full test.

Ans: C Learning Objective: 6-3: Discuss internal consistency reliability and how it is a practical alternative to alternate forms reliability.

Cognitive Domain: Comprehension Answer Location: Internal Consistency Reliability Difficulty Level: Medium 10. Imagine that you have an eight-item test, and you want to estimate its reliability via split-half. For each respondent, you create a score for each four-item half-test. You then correlate the two sets of “half test” scores and find that it is .40. What is the estimated reliability of the entire test likely to be? a. less than .40 b. .40 c. greater than .40 d. no way to tell, based on the information given Ans: C Learning Objective: 6-3: Discuss internal consistency reliability and how it is a practical alternative to alternate forms reliability.

Cognitive Domain: Application Answer Location: Internal Consistency Reliability Difficulty Level: Hard 11. Imagine that you have an eight-item test, and you divide it into halves. For each respondent, you create a score for each half-test. You then correlate the two sets of “half test” scores and find that it is .40. From the split-half approach, what is the estimated reliability of the full test? a. .20 b. .40 c. .57 d. 1.12 Ans: C Learning Objective: 6-3: Discuss internal consistency reliability and how it is a practical alternative to alternate forms reliability.

Cognitive Domain: Application Answer Location: Internal Consistency Reliability Difficulty Level: Hard

Instructor Resource Furr, Psychometrics, 4e SAGE Publishing, 2022 12. Which of the following values reflects “consistency” among the parts of a test? a. variance of the total test scores b. mean of the total test scores c. number of items d. sum of all the inter-item covariances Ans: D Learning Objective: 6-3: Discuss internal consistency reliability and how it is a practical alternative to alternate forms reliability.

Cognitive Domain: Comprehension Answer Location: Internal Consistency Reliability Difficulty Level: Medium 13. What is it called when a test is divided into two parts? a. raw alpha b. split-half reliability c. alternative forms of reliability d. test-retest reliability Ans: B Learning Objective: 6-3: Discuss internal consistency reliability and how it is a practical alternative to alternate forms reliability.

Cognitive Domain: Knowledge Answer Location: Split-Half Estimates of Reliability Difficulty Level: Easy 14. In which approach must we enter the correlation into a special formula to obtain the estimate of reliability? a. raw alpha b. split-half reliability c. alternative forms of reliability d. test-retest reliability Ans: B Learning Objective: 6-3: Discuss internal consistency reliability and how it is a practical alternative to alternate forms reliability.

Cognitive Domain: Comprehension Answer Location: Split-Half Estimates of Reliability Difficulty Level: Medium 15. Which of the following is an internal consistency estimate of reliability? a. raw alpha b. split-half reliability c. alternative forms of reliability d. test-retest reliability Ans: B Learning Objective: 6-3: Discuss internal consistency reliability and how it is a practical alternative to alternate forms reliability.

Instructor Resource Furr, Psychometrics, 4e SAGE Publishing, 2022 Cognitive Domain: Knowledge Answer Location: Split-Half Estimates of Reliability Difficulty Level: Easy 16. What does it mean if two items have a positive covariance or correlation (e.g. if the responses to one item positively co-vary/correlated with responses to the other item)? a. The people who had relatively “elevated” responses to one item tended to also have relatively “low” responses to the other item. b. Individual differences in the responses to one item tend to be consistent with individual differences in the responses to the other item. c. Individual differences in the responses to one item tend to be inconsistent with individual differences in the responses to the other item. d. The two items seem to have nothing in common, in terms of tapping into the same psychological characteristic.

Ans: B Learning Objective: 6-3: Discuss internal consistency reliability and how it is a practical alternative to alternate forms reliability.

Cognitive Domain: Analysis Answer Location: “Raw” Coefficient Alpha Difficulty Level: Medium 17. You conduct a reliability analysis of a set of test scores in SPSS and get the results below. Which of the following is the most accurate interpretation of the .866 and .887 values?

a. In the sample of respondents, we are 95% sure that the raw alpha value is between .866 and .877. b. In the population from which the sample of respondents was drawn, we are 95% sure that the raw alpha value is between .866 and .877. c. We are 86.6% to 88.7% sure that the reliability is .877 in the sample of respondents. d. We are 86.6% to 88.7% sure that the reliability is .877 in the population from which the sample of respondents was drawn.

Ans: B Learning Objective: 6-3: Discuss internal consistency reliability and how it is a practical alternative to alternate forms reliability.

Cognitive Domain: Application Answer Location: “Raw” Coefficient Alpha Difficulty Level: Hard

Instructor Resource Furr, Psychometrics, 4e SAGE Publishing, 2022 18. In which of the following circumstances would you strongly consider standardizing the responses to items before aggregating (i.e., summing or averaging) into a “total score” for a test? a. when there are relatively weak correlations among the items b. when the correlations among the items are too strong c. when some items have much greater variance than do other items d. when some items have much higher means than do other items Ans: C Learning Objective: 6-3: Discuss internal consistency reliability and how it is a practical alternative to alternate forms reliability.

Cognitive Domain: Analysis Answer Location: “Standardized” Coefficient Alpha Difficulty Level: Medium 19. All “internal consistency” methods of estimating reliability are based upon “consistency among the parts of a test.” For the alpha method (raw or standardized), how many “parts of a test” are there? a. number of “parts” = 1 b. number of “parts” = 2 c. number of “parts” = the number of items on the test d. number of “parts” = the number of factors revealed by an exploratory factor analysis Ans: C Learning Objective: 6-3: Discuss internal consistency reliability and how it is a practical alternative to alternate forms reliability.

Cognitive Domain: Application Answer Location: On the Assumptions Underlying Alpha and Omega, the Relative Applicability of Those Indices, and Their Limitations Difficulty Level: Hard 20. What might you be able to increase the reliability of a test? a. shorten the test by removing items that are consistent with the other items on the test b. lengthen the test by adding items that are relatively inconsistent with the other items on the test c. removing items that detract from internal consistency and replace them with items that enhance internal consistency d. removing items that increase internal consistency and replace them with items that reduce internal consistency Ans: C Learning Objective: 6-3: Discuss internal consistency reliability and how it is a practical alternative to alternate forms reliability.

Cognitive Domain: Analysis Answer Location: On the Assumptions Underlying Alpha and Omega, the Relative Applicability of Those Indices, and Their Limitations Difficulty Level: Medium

Instructor Resource Furr, Psychometrics, 4e SAGE Publishing, 2022 21. Which is the most frequently used approach to estimating reliability? a. alternate forms b. test-retest c. split-half d. alpha Ans: D Learning Objective: 6-3: Discuss internal consistency reliability and how it is a practical alternative to alternate forms reliability.

Cognitive Domain: Analysis Answer Location: On the Assumptions Underlying Alpha and Omega, the Relative Applicability of Those Indices, and Their Limitations Difficulty Level: Medium 22. What term is based on the idea that reliability is a ratio of signal and noise? a. alpha b. omega c. variability d. split-half Ans: B Learning Objective: 6-3: Discuss internal consistency reliability and how it is a practical alternative to alternate forms reliability.

Cognitive Domain: Knowledge Answer Location: Omega Difficulty Level: Easy 23. What is a solution to the problem of two test scores with very different variabilities? a. find a new test b. standardize the test scores c. try test-retest reliability d. use a new sample set Ans: B Learning Objective: 6-5: Identify the tools to address psychometric challenges that arise with difference scores.

Cognitive Domain: Analysis Answer Location: The Problem of Unequal Variability Difficulty Level: Medium 24. ___________ scores are most meaningful if two test scores have some psychological attribute in common. a. difference b. true c. z d. raw Ans: A Learning Objective: 6-5: Identify the tools to address psychometric challenges that arise with difference scores.

Instructor Resource Furr, Psychometrics, 4e SAGE Publishing, 2022 Cognitive Domain: Knowledge Answer Location: The Problem of Unequal Variability Difficulty Level: Easy 25. Which type of scores can have poor psychometric quality and lead to questionable psychological conclusions, especially as they may suffer from high intercorrelations between component tests, poor reliability of the component tests, or unequal variances in the component tests? a. difference b. true c. z d. raw Ans: A Learning Objective: 6-5: Identify the tools to address psychometric challenges that arise with difference scores.

Cognitive Domain: Analysis Answer Location: Difference Scores: Summary and Caution Difficulty Level: Medium True/False 1. Reliability cannot be estimated for state-like constructs such as emotions or moods, which can change rapidly over time.

Ans: F Learning Objective: 6-2: Describe the test-retest method of reliability and how it compares to alternate forms reliability.

Cognitive Domain: Analysis Answer Location: Test-Retest Reliability Difficulty Level: Medium 2. For an accurate estimate of reliability, two halves of the split-half approach must be parallel tests.

Ans: T Learning Objective: 6-3: Discuss internal consistency reliability and how it is a practical alternative to alternate forms reliability.

Cognitive Domain: Analysis Answer Location: Split-Half Estimates of Reliability Difficulty Level: Medium 3. Split-half reliability is easily obtained from statistical software.

Ans: T Learning Objective: 6-3: Discuss internal consistency reliability and how it is a practical alternative to alternate forms reliability.

Cognitive Domain: Comprehension

Instructor Resource Furr, Psychometrics, 4e SAGE Publishing, 2022 Answer Location: Split-Half Estimates of Reliability Difficulty Level: Medium 4. The alpha approach produces an accurate estimate of reliability in more circumstances than does the omega approach.

Ans: F Learning Objective: 6-3: Discuss internal consistency reliability and how it is a practical alternative to alternate forms reliability.

Cognitive Domain: Comprehension Answer Location: On the Assumptions Underlying Alpha and Omega, the Relative Applicability of Those Indices, and Their Limitations Difficulty Level: Medium 5. If we compute alpha for a set of test scores, we can be sure that this value accurately reflects the reliability of the test scores.

Ans: F Learning Objective: 6-3: Discuss internal consistency reliability and how it is a practical alternative to alternate forms reliability.

Cognitive Domain: Analysis Answer Location: On the Assumptions Underlying Alpha and Omega, the Relative Applicability of Those Indices, and Their Limitations Difficulty Level: Medium 6. We can estimate reliability, but we can never really know for sure what the reliability of a set of test scores is.

Ans: T Learning Objective: 6-3: Discuss internal consistency reliability and how it is a practical alternative to alternate forms reliability.

Cognitive Domain: Comprehension Answer Location: On the Assumptions Underlying Alpha and Omega, the Relative Applicability of Those Indices, and Their Limitations Difficulty Level: Medium 7. It is probably not a good idea to treat an internal consistency estimate of reliability (e.g., alpha) as a measure of the conceptual homogeneity or dimensionality of the test.

Ans: T Learning Objective: 6-3: Discuss internal consistency reliability and how it is a practical alternative to alternate forms reliability.

Cognitive Domain: Comprehension Answer Location: Internal Consistency Versus Dimensionality Difficulty Level: Medium 8. A reliability estimate of .63 is considered on the high end of acceptable.

Ans: F Learning Objective: 6-3: Discuss internal consistency reliability and how it is a practical alternative to alternate forms reliability.

Instructor Resource Furr, Psychometrics, 4e SAGE Publishing, 2022 Cognitive Domain: Analysis Answer Location: Factors Affecting the Reliability of Test Scores Difficulty Level: Medium 9. The greater the variability among respondents with respect to the psychological attribute being measured, the larger the reliability coefficient.

Ans: T Learning Objective: 6-4: Describe how heterogeneity affects the size of reliability coefficients.

Cognitive Domain: Comprehension Answer Location: Sample Homogeneity and Reliability Generalization Difficulty Level: Medium 10. Anything that increases st2 relative to se2 will decrease the RXX. Ans: F Learning Objective: 6-4: Describe how heterogeneity affects the size of reliability coefficients.

Cognitive Domain: Analysis Answer Location: Sample Homogeneity and Reliability Generalization Difficulty Level: Medium

Short Answer 1. What is it called when a test is divided into two parts?

Ans: Answer may vary.

Learning Objective: 6-3: Discuss internal consistency reliability and how it is a practical alternative to alternate forms reliability.

Cognitive Domain: Knowledge Answer Location: Split-Half Estimates of Reliability Difficulty Level: Easy 2. What is another name for the internal consistency estimate of reliability?

Ans: Answer may vary.

Learning Objective: 6-3: Discuss internal consistency reliability and how it is a practical alternative to alternate forms reliability.

Cognitive Domain: Knowledge Answer Location: Split-Half Estimates of Reliability Difficulty Level: Easy 3. What is the most widely used measure of psychopathology?

Ans: Answer may vary.

Learning Objective: 6-3: Discuss internal consistency reliability and how it is a practical alternative to alternate forms reliability.

Cognitive Domain: Comprehension

Instructor Resource Furr, Psychometrics, 4e SAGE Publishing, 2022 Answer Location: Raw Alpha for Binary Items: KR20 Difficulty Level: Medium 4. What does KR20 stand for?

Ans: Answer may vary.

Learning Objective: 6-3: Discuss internal consistency reliability and how it is a practical alternative to alternate forms reliability.

Cognitive Domain: Comprehension Answer Location: Raw Alpha for Binary Items: KR20 Difficulty Level: Medium 5. What is one advantage Omega has over Alpha?

Ans: Answer may vary.

Learning Objective: 6-3: Discuss internal consistency reliability and how it is a practical alternative to alternate forms reliability.

Cognitive Domain: Analysis Answer Location: Omega Difficulty Level: Medium 6. What is one factor affecting an internal consistency reliability estimate?

Ans: Answer may vary.

Learning Objective: 6-3: Discuss internal consistency reliability and how it is a practical alternative to alternate forms reliability.

Cognitive Domain: Analysis Answer Location: Factors Affecting the Reliability of Test Scores Difficulty Level: Medium 7. What is one tool that can be used to address psychometric challenges that may arise with difference scores?

Ans: Answer may vary.

Learning Objective: 6-5: Identify the tools to address psychometric challenges that arise with difference scores.

Cognitive Domain: Comprehension Answer Location: Reliability of Different Scores Difficulty Level: Medium 8. What would you use to study change or differences?

Ans: Answer may vary.

Learning Objective: 6-5: Identify the tools to address psychometric challenges that arise with difference scores.

Cognitive Domain: Comprehension Answer Location: Defining Difference Scores Difficulty Level: Medium 9. What would the following information be useful in estimating?

Instructor Resource Furr, Psychometrics, 4e SAGE Publishing, 2022 (1) the estimated reliability of each set of test scores used to compute the difference scores (RXX and RYY), (2) the variability of the tests’ observed scores ( sX2 o , sY2o , sX o , and sYo ), and (3) the correlation between the observed test scores ( rX oYo ) : Ans: Answer may vary.

Learning Objective: 6-5: Identify the tools to address psychometric challenges that arise with difference scores.

Cognitive Domain: Analysis Answer Location: Estimating the Reliability of Difference Scores Difficulty Level: Medium 10. What is one factor that determines whether a set of difference scores has good reliability?

Ans: Answer may vary.

Learning Objective: 6-5: Identify the tools to address psychometric challenges that arise with difference scores.

Cognitive Domain: Comprehension Answer Location: Factors Affecting the Reliability of Difference Scores Difficulty Level: Medium

Instructor Resource Furr, Psychometrics, 4e SAGE Publishing, 2022 Chapter 7: The Importance of Reliability Test Bank Multiple Choice 1. What is the best definition of a “point estimate” of a true score? a. the range of values within which we strongly believe the person’s true score is likely to be located b. our best guess about a test taker’s actual standing on a particular psychological attribute c. our best guess of the actual reliability of a set of test scores d. the point at which an individual’s true score is greater than the amount of error affecting her/his observed score Ans: B Learning Objective: 7-1: Describe how reliability affects the confidence, accuracy, and precision with which an individual’s true score is estimated.

Cognitive Domain: Knowledge Answer Location: Applied Behavioral Practice: Evaluation of an Individual’s Test Score Difficulty Level: Easy 2. When using an “adjusted true score estimate,” how does an individual’s estimated true scores compare to their observed (unadjusted) scores? a. The estimated true score will generally be lower than the individual’s observed score. b. The estimated true score will generally be higher than the individual’s observed score. c. The estimated true score will be less extreme (i.e., closer to the mean of observed scores) than the individual’s observed score. d. The estimated true score will be more extreme (i.e., further away from the mean of observed scores) than the individual’s observed score.

Ans: C Learning Objective: 7-1: Describe how reliability affects the confidence, accuracy, and precision with which an individual’s true score is estimated.

Cognitive Domain: Comprehension Answer Location: Point Estimates of True Scores Difficulty Level: Medium 3. Say that a confidence interval around a true score was 950 to 1050. Which is the best interpretation of these values? a. We are 95% confident that the child’s observed score falls between 950 and 1050. b. We are 95% confident that the child’s true score lies between 950 and 1050. c. The true score is likely to be statistically significant if it is in the range of 950 to 1050. d. The true score is unlikely to be statistically significant if it is in the range of 950 to 1050.

Instructor Resource Furr, Psychometrics, 4e SAGE Publishing, 2022 Ans: B Learning Objective: 7-1: Describe how reliability affects the confidence, accuracy, and precision with which an individual’s true score is estimated.

Cognitive Domain: Analysis Answer Location: Point Estimates of True Scores Difficulty Level: Medium 4. Which is the best (i.e., more precise) confidence interval? a. 950 to 1050 b. 990 to 1010 c. 910 to 1010 d. 990 to 1050 Ans: B Learning Objective: 7-1: Describe how reliability affects the confidence, accuracy, and precision with which an individual’s true score is estimated.

Cognitive Domain: Comprehension Answer Location: Confidence Intervals Difficulty Level: Medium 5. What is the link between test reliability and the confidence interval around an individual’s score? a. There is no systematic association between reliability and the CI (though the CI is heavily affected by validity). b. Better reliability leads to a narrower CI. c. Better reliability leads to a wider CI. d. Better reliability leads to a CI that’s more likely to be statistically significant.

Ans: B Learning Objective: 7-1: Describe how reliability affects the confidence, accuracy, and precision with which an individual’s true score is estimated.

Cognitive Domain: Analysis Answer Location: Confidence Intervals Difficulty Level: Medium 6. According to true score theory, ________ scores are distributed normally around true scores. a. observed b. applied c. standard d. normalized Ans: A Learning Objective: 7-1: Describe how reliability affects the confidence, accuracy, and precision with which an individual’s true score is estimated.

Cognitive Domain: Knowledge Answer Location: Debate and Alternatives Difficulty Level: Easy

Instructor Resource Furr, Psychometrics, 4e SAGE Publishing, 2022 7. Poor __________ produces scores that are imprecise reflections of individuals’ true psychological traits and skills. a. estimates b. scores c. reliability d. variance Ans: C Learning Objective: 7-1: Describe how reliability affects the confidence, accuracy, and precision with which an individual’s true score is estimated.

Cognitive Domain: Knowledge Answer Location: Summary (Applied Behavioral Practice: Evolution of an Individual’s Test Score) Difficulty Level: Easy 8. Which term means the effect that measurement error (unreliability) has on the observed score effect size? a. attenuation b. Cohen’s D c. correlation d. validity Ans: A Learning Objective: 7-2: Summarize how reliability affects observed effect sizes and statistical significance obtained in research.

Cognitive Domain: Knowledge Answer Location: Measurement Error (Low Reliability) Attenuates the Observed Associations Between Measures Difficulty Level: Easy 9. Imagine you were a psychology researcher and you hypothesized that variable X is correlated with variable Y. You do a study to test that hypothesis—you measure both variables, and you compute the correlation between your two measures. Which result would provide the strongest legitimate support for your hypothesis? a. r = 0 b. r = .05 c. r = .20 d. r = .30 Ans: D Learning Objective: 7-2: Summarize how reliability affects observed effect sizes and statistical significance obtained in research.

Cognitive Domain: Application Answer Location: Reliability, Effect Sizes, and Statistical Significance Difficulty Level: Hard 10. Imagine further that, in your study, your analysis reveals that the correlation is statistically significant. What does statistical significance mean? a. The correlation signifies an important finding with serious practical consequences.

Instructor Resource Furr, Psychometrics, 4e SAGE Publishing, 2022 b. The correlation indicates a large and robust relationship. c. The correlation is unlikely to have occurred by chance. d. There is no way to make a correlation in your analysis.

Ans: C Learning Objective: 7-2: Summarize how reliability affects observed effect sizes and statistical significance obtained in research.

Cognitive Domain: Analysis Answer Location: Reliability, Effect Sizes, and Statistical Significance Difficulty Level: Medium 11. In a research context, low reliability will: a. make the observed effect sizes (e.g., correlations among measures) weaker than they should be b. make the observed effect sizes (e.g., correlations among measures) stronger than they should be c. make the observed effect sizes (e.g., correlations among measures) either weaker or stronger than they should be d. not influence the observed effects, but it will influence the “true” effects Ans: A Learning Objective: 7-2: Summarize how reliability affects observed effect sizes and statistical significance obtained in research.

Cognitive Domain: Analysis Answer Location: Reliability, Effect Sizes, and Statistical Significance Difficulty Level: Medium 12. What is the effect of low reliability on statistical significance? Low reliability will: a. increase the likelihood of getting a “significant” result b. decrease the likelihood of getting a “significant” result c. affect significance in different ways for different effect sizes (making correlations less likely to be significant, but other effect sizes more likely to be significant) d. have no systematic effect on statistical significance Ans: B Learning Objective: 7-2: Summarize how reliability affects observed effect sizes and statistical significance obtained in research.

Cognitive Domain: Analysis Answer Location: Reliability, Effect Sizes, and Statistical Significance Difficulty Level: Medium 13. Which term means a statistical value representing the result of a study as a matter of degree (e.g., the degree of association between variables or the degree of difference between groups)? a. effect size b. true score effect size c. observed score effect size d. statistical significance Ans: A

Instructor Resource Furr, Psychometrics, 4e SAGE Publishing, 2022 Learning Objective: 7-2: Summarize how reliability affects observed effect sizes and statistical significance obtained in research.

Cognitive Domain: Knowledge Answer Location: Reliability, Effect Sizes, and Statistical Significance Difficulty Level: Easy 14. Which term means a value representing the result of a study as a dichotomous outcome? a. effect size b. true score effect size c. observed score effect size d. statistical significance Ans: D Learning Objective: 7-2: Summarize how reliability affects observed effect sizes and statistical significance obtained in research.

Cognitive Domain: Knowledge Answer Location: Reliability, Effect Sizes, and Statistical Significance Difficulty Level: Easy 15. Which term means the result of a study at a psychological level (i.e., that would be obtained if the perfect measurement was possible)? a. effect size b. true score effect size c. observed score effect size d. statistical significance Ans: B Learning Objective: 7-2: Summarize how reliability affects observed effect sizes and statistical significance obtained in research.

Cognitive Domain: Knowledge Answer Location: Reliability, Effect Sizes, and Statistical Significance Difficulty Level: Easy 16. Which term means the results of a study that is obtained by the research (using imperfect measures)? a. effect size b. true score effect size c. observed score effect size d. statistical significance Ans: C Learning Objective: 7-2: Summarize how reliability affects observed effect sizes and statistical significance obtained in research.

Cognitive Domain: Knowledge Answer Location: Reliability, Effect Sizes, and Statistical Significance Difficulty Level: Easy 17. Which term means an effect size representing the difference between two groups?

Instructor Resource Furr, Psychometrics, 4e SAGE Publishing, 2022 a. attenuation b. Cohen’s D c. correlation d. validity Ans: B Learning Objective: 7-2: Summarize how reliability affects observed effect sizes and statistical significance obtained in research.

Cognitive Domain: Knowledge Answer Location: Reliability, Effect Sizes, and Statistical Significance Difficulty Level: Easy 18. Which term means an effect size representing the degree of association between variables? a. attenuation b. Cohen’s D c. correlation d. validity Ans: C Learning Objective: 7-2: Summarize how reliability affects observed effect sizes and statistical significance obtained in research.

Cognitive Domain: Knowledge Answer Location: Reliability, Effect Sizes, and Statistical Significance Difficulty Level: Easy 19. ____________ affects the observed effect sizes and statistical significance obtained in research. a. test construction b. item variance c. reliability d. difference scores Ans: C Learning Objective: 7-2: Summarize how reliability affects observed effect sizes and statistical significance obtained in research.

Cognitive Domain: Knowledge Answer Location: Summary (Behavioral Research) Difficulty Level: Easy 20. Which is the correct definition for "item discrimination"? a. a general term meaning the degree to which an item differentiates people who score high on the total test from those who score low on the total test b. the reliability estimate that you would obtain if an item were dropped from the test (i.e., the reliability estimated you’d get for a test comprising all the other items) c. the statistical association between an item and the total test score d. the statistical association between an item and the total test score (where the total test score excludes the item in question) Ans: A

Instructor Resource Furr, Psychometrics, 4e SAGE Publishing, 2022 Learning Objective: 7-3: Discuss the key ways reliability is evaluated and considered in research and practice in behavioral science.

Cognitive Domain: Knowledge Answer Location: Test Construction and Refinement Difficulty Level: Easy 21. Which is the correct definition for "item-total correlation"? a. a general term meaning the degree to which an item differentiates people who score high on the total test from those who score low on the total test b. the reliability estimate that you would obtain if an item were dropped from the test (i.e., the reliability estimated you’d get for a test comprising all the other items) c. the statistical association between an item and the total test score d. the statistical association between an item and the total test score (where the total test score excludes the item in question) Ans: C Learning Objective: 7-3: Discuss the key ways reliability is evaluated and considered in research and practice in behavioral science.

Cognitive Domain: Knowledge Answer Location: Item Discrimination and Other Information Regarding Internal Consistency Difficulty Level: Easy 22. Which is the correct definition for "alpha if item deleted"? a. a general term meaning the degree to which an item differentiates people who score high on the total test from those who score low on the total test b. the reliability estimate that you would obtain if an item were dropped from the test (i.e., the reliability estimated you’d get for a test comprising all the other items) c. the statistical association between an item and the total test score d. the statistical association between an item and the total test score (where the total test score excludes the item in question) Ans: B Learning Objective: 7-3: Discuss the key ways reliability is evaluated and considered in research and practice in behavioral science.

Cognitive Domain: Knowledge Answer Location: Item Discrimination and Other Information Regarding Internal Consistency Difficulty Level: Easy 23. Which is the correct definition for "correlated item-total correlation"? a. a general term meaning the degree to which an item differentiates people who score high on the total test from those who score low on the total test b. the reliability estimate that you would obtain if an item were dropped from the test (i.e., the reliability estimated you’d get for a test comprising all the other items) c. the statistical association between an item and the total test score d. the statistical association between an item and the total test score (where the total test score excludes the item in question)

Instructor Resource Furr, Psychometrics, 4e SAGE Publishing, 2022 Ans: D Learning Objective: 7-3: Discuss the key ways reliability is evaluated and considered in research and practice in behavioral science.

Cognitive Domain: Knowledge Answer Location: Item Discrimination and Other Information Regarding Internal Consistency Difficulty Level: Easy 24. An item’s _________ and __________ may be related to the consistency between that item and the other items on a test. a. interval; effect size b. alpha; omega c. test score; difference score d. mean; variance Ans: D Learning Objective: 7-3: Discuss the key ways reliability is evaluated and considered in research and practice in behavioral science.

Cognitive Domain: Comprehension Answer Location: Item Difficulty (Mean) and Item Variance Difficulty Level: Medium 25. Items with limited _________ are less likely to have good correlational characteristics than are items with substantial ____________. (Both blanks are the same). a. effect sizes b. test scores c. variances d. difference scores Ans: C Learning Objective: 7-3: Discuss the key ways reliability is evaluated and considered in research and practice in behavioral science.

Cognitive Domain: Comprehension Answer Location: Item Difficulty (Mean) and Item Variance Difficulty Level: Medium True/False 1. The quality and meaningfulness of any research hinges on the quality of the measurement procedures used in that research.

Ans: T Learning Objective: 7-2: Summarize how reliability affects observed effect sizes and statistical significance obtained in research.

Cognitive Domain: Analysis Answer Location: Behavioral Research Difficulty Level: Medium

Instructor Resource Furr, Psychometrics, 4e SAGE Publishing, 2022 2. A fundamental goal of research is to discover how important variables are related to each other.

Ans: T Learning Objective: 7-2: Summarize how reliability affects observed effect sizes and statistical significance obtained in research.

Cognitive Domain: Analysis Answer Location: Reliability, True Associations, and Observed Associations Difficulty Level: Medium 3. CTT implies that the correlation between two measures is determined by the correlation between psychological constructs and by the reliability of the measures.

Ans: T Learning Objective: 7-2: Summarize how reliability affects observed effect sizes and statistical significance obtained in research.

Cognitive Domain: Comprehension Answer Location: Reliability, True Association, and Observed Associations Difficulty Level: Medium 4. Cohens d is the effect that measurement error (unreliability) has on the observed score effect size.

Ans: F Learning Objective: 7-2: Summarize how reliability affects observed effect sizes and statistical significance obtained in research.

Cognitive Domain: Knowledge Answer Location: Measurement Error (Low Reliability) Attenuates the Observed Associations Between Measures Difficulty Level: Easy 5. True score effect size is a statistical value representing the result of a study as a matter of degree (e.g., the degree of association between variables or the degree of difference between groups).

Ans: F Learning Objective: 7-2: Summarize how reliability affects observed effect sizes and statistical significance obtained in research.

Cognitive Domain: Knowledge Answer Location: Reliability, Effect Size, and Statistical Significance Difficulty Level: Easy 6. Statistical significance is a statistical value representing the result of a study as a dichotomous outcome.

Ans: T Learning Objective: 7-2: Summarize how reliability affects observed effect sizes and statistical significance obtained in research.

Cognitive Domain: Knowledge

Instructor Resource Furr, Psychometrics, 4e SAGE Publishing, 2022 Answer Location: Reliability, Effect Size, and Statistical Significance Difficulty Level: Easy 7. True score effect size is the result of a study at a psychological level (i.e., that would be obtained if the perfect measurement was possible).

Ans: T Learning Objective: 7-2: Summarize how reliability affects observed effect sizes and statistical significance obtained in research.

Cognitive Domain: Knowledge Answer Location: Reliability, Effect Size, and Statistical Significance Difficulty Level: Easy 8. Effect size is the results of a study that is obtained by the research (using imperfect measures).

Ans: F Learning Objective: 7-2: Summarize how reliability affects observed effect sizes and statistical significance obtained in research.

Cognitive Domain: Knowledge Answer Location: Reliability, Effect Size, and Statistical Significance Difficulty Level: Easy 9. Correlation is an effect size representing the difference between two groups.

Ans: F Learning Objective: 7-2: Summarize how reliability affects observed effect sizes and statistical significance obtained in research.

Cognitive Domain: Knowledge Answer Location: Reliability, Effect Size, and Statistical Significance Difficulty Level: Easy 10. Correlation is an effect size representing the degree of association between variables.

Ans: T Learning Objective: 7-2: Summarize how reliability affects observed effect sizes and statistical significance obtained in research.

Cognitive Domain: Knowledge Answer Location: Reliability, Effect Size, and Statistical Significance Difficulty Level: Easy Short Answer 1. How are confidence intervals computed for various degrees of confidence?

Ans: Answer may vary.

Learning Objective: 7-1: Describe how reliability affects the confidence, accuracy, and precision with which an individual’s true score is estimated.

Cognitive Domain: Comprehension

Instructor Resource Furr, Psychometrics, 4e SAGE Publishing, 2022 Answer Location: Debate and Alternatives Difficulty Level: Medium 2. In terms of psychometrics, what is the most common way of quantifying the association between variables?

Ans: Answer may vary.

Learning Objective: 7-2: Summarize how reliability affects observed effect sizes and statistical significance obtained in research.

Cognitive Domain: Comprehension Answer Location: Reliability, True Association, and Observed Associations Difficulty Level: Medium 3. According to CTT, the correlation between observed scores of two measures is determined by two factors. What is one of them?

Ans: Answer may vary.

Learning Objective: 7-2: Summarize how reliability affects observed effect sizes and statistical significance obtained in research.

Cognitive Domain: Comprehension Answer Location: Reliability, True Association, and Observed Associations Difficulty Level: Medium 4. In terms of observed scores, the correlation between scores from two measures is given by what equation?

Ans: Answer may vary.

Learning Objective: 7-2: Summarize how reliability affects observed effect sizes and statistical significance obtained in research.

Cognitive Domain: Comprehension Answer Location: Reliability, True Association, and Observed Associations Difficulty Level: Medium 5. There are at least three important implications of considering reliability when drawing psychological conclusions from research. What is one of them?

Ans: Answer may vary.

Learning Objective: 7-2: Summarize how reliability affects observed effect sizes and statistical significance obtained in research.

Cognitive Domain: Comprehension Answer Location: Implications for Conducting and Interpreting Behavioral Research Difficulty Level: Medium 6. There are reasons that a researcher may use a low-reliability measure instead of spending the time attempting to develop a highly reliable measure when drawing psychological conclusions from research. What is one reason a researcher may use a low-reliability measure?

Ans: Answer may vary.

Learning Objective: 7-2: Summarize how reliability affects observed effect sizes and statistical significance obtained in research.

Instructor Resource Furr, Psychometrics, 4e SAGE Publishing, 2022 Cognitive Domain: Comprehension Answer Location: Implications for Conducting and Interpreting Behavioral Research Difficulty Level: Medium 7. What is the general term meaning the degree to which an item differentiates people who score high on the total test from those who score low on the total test?

Ans: Answer may vary.

Learning Objective: 7-3: Discuss the key ways reliability is evaluated and considered in research and practice in behavioral science.

Cognitive Domain: Knowledge Answer Location: Test Construction and Refinement Difficulty Level: Easy 8. What is the term for a statistical association between an item and the total test score?

Ans: Answer may vary.

Learning Objective: 7-3: Discuss the key ways reliability is evaluated and considered in research and practice in behavioral science.

Cognitive Domain: Knowledge Answer Location: Item Discrimination and Other Information Regarding Internal Consistency Difficulty Level: Easy 9. What is the term for a statistical association between an item and the total test score (where the total test score excludes the item in question)?

Ans: Answer may vary.

Learning Objective: 7-3: Discuss the key ways reliability is evaluated and considered in research and practice in behavioral science.

Cognitive Domain: Knowledge Answer Location: Item Discrimination and Other Information Regarding Internal Consistency Difficulty Level: Easy 10. What is the term for the reliability estimate that you would obtain if an item were dropped from the test (i.e., the reliability estimated you’d get for a test comprising all the other items)?

Ans: Answer may vary.

Learning Objective: 7-3: Discuss the key ways reliability is evaluated and considered in research and practice in behavioral science.

Cognitive Domain: Knowledge Answer Location: Item Discrimination and Other Information Regarding Internal Consistency Difficulty Level: Easy

Instructor Resource Furr, Psychometrics, 4e SAGE Publishing, 2022 Chapter 8: Validity: Conceptual Basis Test Bank Multiple Choice 1. Which of the following is an accurate statement about validity, based on the definition endorsed by AERA, APA, and NCME? a. Validity is about the interpretations and use of test scores. b. Validity is about the precision with which a test’s scores reflect any particular construct. c. Validity is an all-or-none issue, a test is either valid (and thus psychometrically appropriate for use) or not. d. Validity is supported by a test user’s experience and personal opinions on the relevant psychological phenomenon.

Ans: A Learning Objective: 8-1: Describe how the concept of validity has evolved over time.

Cognitive Domain: Comprehension Answer Location: What Is Validity?

Difficulty Level: Medium 2. Imagine that you develop your own psychological test, and you want to write about validity when describing your analysis of the test. Which of the following statements is phrased in a way that is most consistent with the contemporary definition of validity? a. “My test is valid.” b. “My test is moderately valid.” c. “My test is valid as a measure of construct X.” d. “My test is moderately valid as a measure of construct X.” Ans: D Learning Objective: 8-1: Describe how the concept of validity has evolved over time.

Cognitive Domain: Application Answer Location: What Is Validity?

Difficulty Level: Hard 3. From the traditional perspective, which of the following is not a main type of validity? a. construct b. content c. criterion d. internal Ans: D Learning Objective: 8-1: Describe how the concept of validity has evolved over time.

Cognitive Domain: Analysis Answer Location: What Is Validity?

Difficulty Level: Medium

Instructor Resource Furr, Psychometrics, 4e SAGE Publishing, 2022 4. Which of the following is the most straightforward example of a lack of construct validity? a. A researcher uses a scale that a colleague designed to measure depression. The scale’s scores measure depression but do so very imprecisely (with a great deal of random measurement error). This imprecision harms her ability to detect meaningful psychological effects in her data b. The admissions staff at a university requires applicants to participate in an admissions interview. In this interview process, interviewers rate the applicant’s “capacity for academic achievement.” These interview-based ratings do accept this capacity but do so more precisely for females than for males. This leads the staff to make more well-informed admissions decisions for female applicants than for male applicants. c. A clinical psychologist designs a new scale that he believes to reflect psychopathy, and he uses in his practice and interprets as such when working with clients. In fact, the scale does not measure psychopathy, it instead actually reflects histrionic disorder. This misinterpretation leads to ineffective and potentially harmful work with his clients. d. A lack of construct validity cannot be constructed in one, straightforward example.

Ans: C Learning Objective: 8-1: Describe how the concept of validity has evolved over time.

Cognitive Domain: Analysis Answer Location: What Is Validity?

Difficulty Level: Medium 5. Which of the following is the best for evaluating the construct validity of a test or assessment? a. Evaluate whether test scores are correlated with things that they should be correlated with. b. Evaluate whether test scores are consistent from one time to a later time. c. Evaluate whether the different items on a test are consistent with each other. d. Evaluate whether test scores do not differ (on average) across groups of respondents.

Ans: A Learning Objective: 8-1: Describe how the concept of validity has evolved over time.

Cognitive Domain: Comprehension Answer Location: What Is Validity?

Difficulty Level: Medium 6. What is perhaps the most crucial issue in a test’s psychometric quality? a. raw scores b. a diverse sample c. reliability d. validity

Instructor Resource Furr, Psychometrics, 4e SAGE Publishing, 2022 Ans: D Learning Objective: 8-2: Discuss the role and importance of validity in psychological research and practice.

Cognitive Domain: Analysis Answer Location: The Importance of Validity Difficulty Level: Medium 7. Loren develops a new test of mathematical ability. She views math ability as a twofactor construct—one factor representing calculus ability and one representing noncalculus ability. Her new test includes various types of math items (i.e., calculus, algebra, and geometry). Based on her theory of the “math ability” construct, she hypothesizes that there will be two weakly correlated dimensions to her test—a) calculus ability, and b) non-calculus math ability, with all algebra and geometry items loading equally strongly on the non-calc factor. After collecting responses to her test from many students, she finds that the test seems to have a three-dimensional structure (algebra ability, geometry ability, calculus ability) with weakly correlated dimensions.

This finding suggests: a. a lack of internal structure validity b. a good level of internal structure validity c. a lack of content validity d. good content validity Ans: A Learning Objective: 8-3: Identify how an expert rating of test content is an important facet of validity.

Cognitive Domain: Analysis Answer Location: Validity Evidence: Test Content Difficulty Level: Medium 8. Micha wants to study moral relativism. He designs a new scale to detect who does (and does not) endorse a relativistic moral philosophy. He writes an initial set of items.

He then asks experts (professors and students in moral philosophy and moral psychology) to read all of his items and to rate the degree to which each item reflects moral relativism (as opposed to any other construct). Micha is attempting to evaluate __________________. a. content validity b. face validity c. internal structure validity d. conceptual validity Ans: A Learning Objective: 8-3: Identify how an expert rating of test content is an important facet of validity.

Cognitive Domain: Analysis Answer Location: Validity Evidence: Test Content Difficulty Level: Medium

Instructor Resource Furr, Psychometrics, 4e SAGE Publishing, 2022 9. Micha also wants to study moral tolerance, so he develops a new set of items to reflect that construct (to detect who is and is not morally tolerant). He sees moral tolerance as a psychological construct/dimension that is separate from moral relativism (e.g., relativists could either be tolerant or intolerant). He recruits a sample of respondents to answer his moral relativism items and his moral tolerance items. He conducts a factor analysis of the responses to all her items, expecting to find a twofactor structure. Micha is attempting to evaluate __________________ validity. a. content b. face c. internal structure d. conceptual Ans: C Learning Objective: 8-3: Identify how an expert rating of test content is an important facet of validity.

Cognitive Domain: Analysis Answer Location: Validity Evidence: Test Content Difficulty Level: Medium 10. Construct underrepresentation occurs when _________________________. a. you’re evaluating the validity of a test, and you find that the sample of research respondents does not have the construct of interest b. you’re evaluating the validity of a test, and you find that the sample of research respondents does not represent the full range of trait/ability levels of the construct c. a test does not cover the entire range of content that is relevant to its intended construct d. the construct that’s intended to be measured by a test is not well-established as an important psychological attribute worthy of measurement.

Ans: C Learning Objective: 8-3: Identify how an expert rating of test content is an important facet of validity.

Cognitive Domain: Comprehension Answer Location: Threats to Content Validity Difficulty Level: Medium 11. Construct-irrelevant content occurs when _________________________. a. you’re evaluating the validity of a test, and you find that the sample of research respondents does not have the construct of interest b. you’re evaluating the validity of a test, and you find that the sample of research respondents does not represent the full range of trait/ability levels of the construct c. a test does not cover the entire range of content that is relevant to its intended construct d. a test covers material that is not part of the construct that the test is intended to measure Ans: D Learning Objective: 8-3: Identify how an expert rating of test content is an important facet of validity.

Instructor Resource Furr, Psychometrics, 4e SAGE Publishing, 2022 Cognitive Domain: Comprehension Answer Location: Threats to Content Validity Difficulty Level: Medium 12. Let’s say that this class quiz is intended to assess two constructs that we’ll call “knowledge of content validity” and “knowledge of internal structure validity.” Let’s say that the quiz includes an item about discriminant validity. The presence of that item represents: a. content validity b. construct underrepresentation c. construct-irrelevant content d. structural invalidity Ans: C Learning Objective: 8-3: Identify how an expert rating of test content is an important facet of validity.

Cognitive Domain: Analysis Answer Location: Threats to Content Validity Difficulty Level: Medium 13. Robert develops a 12-item test of political attitudes and hypothesizes that it reflects three dimensions—conservative/liberal economic attitudes, conservative/liberal social attitudes, and conservative/liberal foreign policy attitudes. He conducts an EFA of responses to his items and evaluates the internal structure validity of his scale. Which of the following scree plots would be the most supportive of his scale as reflecting the proposed construct dimensionality?

Instructor Resource Furr, Psychometrics, 4e SAGE Publishing, 2022 7

7

A

6

B

6

5

5

4

4

3

3

2

2

1

1

0

0 1

2

3

4

5

6

7

7

8

9

10 11 12

C

6

1

2

3

4

5

6

7

5

4

4

3

3

2

2

1

1

0

8

9

10 11 12

7

8

9

10 11 12

D

6

5

7

0 1

2

3

4

5

6

7

8

9

10 11 12

1

2

3

4

5

6

a. chart A b. chart B c. chart C d. chart D Ans: C Learning Objective: 8-4: Summarize how factor analysis is used to evaluate the internal structure of psychological tests.

Cognitive Domain: Analysis Answer Location: Factor-Analytic Evidence Difficulty Level: Medium 14. Mike creates an exam for his Psychological Testing class, and he includes questions about reliability. Some of these questions are intended to assess whether students understand the conceptual and mathematical connections between various concepts (e.g., reliability, true score variance, etc). As part of the exam, he requires students to explain how they arrived at their answers (e.g., to show their computational work, or to explain whether an answer is based upon some basic assumptions of CTT).

When grading the test, Mike gives “full credit” only when a correct answer is accompanied by an explanation that reflects students arrived at their answers by recognizing, using, and interpreting the connections correctly. By using this scoring strategy, Mike is most directly attempting to ensure that the exam has: a. response process validity b. convergent validity

Instructor Resource Furr, Psychometrics, 4e SAGE Publishing, 2022 c. concurrent validity d. discriminant validity Ans: A Learning Objective: 8-5: Describe the response process as a type of validity evidence and how some procedures are relatively direct or indirect.

Cognitive Domain: Analysis Answer Location: Direct Evidence Difficulty Level: Medium 15. One of the primary ways in which researchers have evaluated the validity of the SAT is by examining the connection between SAT scores and college GPA (under the hypothesis that SAT scores should be correlated with GPA). This procedure reflects an attempt to evaluate: a. criterion validity b. concurrent validity c. predictive validity d. discriminant validity Ans: C Learning Objective: 8-6: Discuss validity evidence’s association between test scores and other variances, including convergent, discriminant, criterion, concurrent, and predictive evidence.

Cognitive Domain: Comprehension Answer Location: Validity Evidence: Associations With Other Variables Difficulty Level: Medium 16. A doctoral student wants to develop a psychotherapeutic treatment for Dependent Personality Disorder (DPD). As part of his dissertation, he develops a new test to measure DPD, but he does not evaluate its psychometric quality very carefully.

He subsequently uses the test in his dissertation. He finds that its scores are indeed affected (i.e., reduced) by a new psychotherapeutic treatment that he has developed.

He, therefore, concludes that the new psychotherapeutic treatment is effective at reducing DPD, and he recommends it as a clinical option for DPD. However, one of his dissertation committee members points to the items on his new measure, and she notes (correctly) that many of them look like they reflect anxiety more so than DPD. Thus, his new measure (which supposedly reflects only levels of DPD) might reflect Generalized Anxiety Disorder (GAD) along with DPD (i.e., the scores likely reflect some messy blend of the two). Based on this, she thus suggests that, unfortunately, his new measure might lack ______________ validity.

Unfortunately, it is thus not clear what his new psychotherapeutic treatment does. Yes, the treatment seems to affect the scores on his new measure. But does that mean that the treatment is a good treatment of DPD or anxiety? Alas, it's not clear, due to the measurement problem that his committee member noticed: a. discriminant validity

Instructor Resource Furr, Psychometrics, 4e SAGE Publishing, 2022 b. criterion validity c. predictive validity d. consequential validity Ans: A Learning Objective: 8-6: Discuss validity evidence’s association between test scores and other variances, including convergent, discriminant, criterion, concurrent, and predictive evidence.

Cognitive Domain: Analysis Answer Location: Validity Evidence: Associations With Other Variables Difficulty Level: Medium 17. Which of the following results would typically be taken as evidence of discriminant validity? a. large positive correlation b. correlation of about zero c. large negative correlation d. two-factor internal structure Ans: B Learning Objective: 8-6: Discuss validity evidence’s association between test scores and other variances, including convergent, discriminant, criterion, concurrent, and predictive evidence.

Cognitive Domain: Comprehension Answer Location: Validity Evidence: Associations With Other Variables Difficulty Level: Medium 18. Suppose a new measure of extraversion possesses a negligible correlation with intelligence (small and non-significant). This would be support for ____________ validity. a. convergent b. discriminant c. content d. concurrent Ans: B Learning Objective: 8-6: Discuss validity evidence’s association between test scores and other variances, including convergent, discriminant, criterion, concurrent, and predictive evidence.

Cognitive Domain: Comprehension Answer Location: Validity Evidence: Associations With Other Variables Difficulty Level: Medium 19. Emma creates new a self-report measure of assertiveness and she recruits participants to complete the test. She also asks the participants’ roommates to describe how assertive the participants are. Emma finds that those participants who got high assertiveness scores on her new test were also the ones that were described as very assertive by their roommates. This finding provides ____________________.

Instructor Resource Furr, Psychometrics, 4e SAGE Publishing, 2022 a. evidence of good internal consistency reliability b. good convergent evidence of validity c. evidence of discriminant validity d. evidence of poor internal consistency reliability Ans: B Learning Objective: 8-6: Discuss validity evidence’s association between test scores and other variances, including convergent, discriminant, criterion, concurrent, and predictive evidence.

Cognitive Domain: Comprehension Answer Location: Convergent Evidence Difficulty Level: Medium 20. What is the degree to which test scores are uncorrelated with tests of unrelated constructs? a. convergent evidence b. discriminant evidence c. direct evidence d. indirect evidence Ans: B Learning Objective: 8-6: Discuss validity evidence’s association between test scores and other variances, including convergent, discriminant, criterion, concurrent, and predictive evidence.

Cognitive Domain: Knowledge Answer Location: Discriminant Evidence Difficulty Level: Easy 21. Which of the following is the least likely to be linked to the associations between variables? a. criterion validity b. concurrent validity c. predictive validity d. discriminant validity Ans: D Learning Objective: 8-6: Discuss validity evidence’s association between test scores and other variances, including convergent, discriminant, criterion, concurrent, and predictive evidence.

Cognitive Domain: Analysis Answer Location: Criterion, Concurrent, and Predictive Evidence Difficulty Level: Medium 22. Which of the following best captures the difference between reliability and validity? a. Reliability is about a test itself, whereas validity is about test scores. b. Validity is about a test itself, whereas reliability is about test scores. c. We can evaluate reliability without regard to the meaning of test scores, whereas validity is primarily about the meaning of test scores.

Instructor Resource Furr, Psychometrics, 4e SAGE Publishing, 2022 d. Reliability concerns the use of a test, whereas validity concerns the meaning of a test.

Ans: C Learning Objective: 8-7: Recall the three key issues regarding consequential validity, including the intended and actual consequences of test use.

Cognitive Domain: Comprehension Answer Location: Validity Evidence: Consequences of Testing Difficulty Level: Medium 23. The book described observations made by Shepard (1997) regarding the use of the Medical College Admission Test (MCAT). The MCAT became so central to the medical school admission process, that undergraduate colleges’ “premed” programs began focusing their classes/training only on the narrow types of material covered by the MCAT (biology, chemistry, etc) and ignoring other material that could be crucial in medical training and practice. This shows how a test can shape the broader context in which it is used.

This is an example of: a. intended systemic effects b. unintended differential impact on groups c. unintended systemic effects d. construct Bias Ans: C Learning Objective: 8-7: Recall the three key issues regarding consequential validity, including the intended and actual consequences of test use.

Cognitive Domain: Analysis Answer Location: Evidence Regarding Unintended Systemic Effects Difficulty Level: Medium 24. Many states in the US require students to take “End of Grade” standardized tests in various subjects, including math. If you read the technical manual for North Carolina’s End of Grade math tests, you will find a description of the test development process.

That process involved educators and experts who evaluated potential test items. They read each potential item and determined whether it matched the “standards” (or material that the test should cover, such as geometry, addition, and multiplication). This process of “expert evaluation” of the test items illustrates an attempt to maximize the EoG tests’: a. content validity b. convergent/Discriminant validity c. response process validity d. consequential validity Ans: D Learning Objective: 8-7: Recall the three key issues regarding consequential validity, including the intended and actual consequences of test use.

Cognitive Domain: Analysis Answer Location: Validity Evidence: Consequences of Testing Difficulty Level: Medium

Instructor Resource Furr, Psychometrics, 4e SAGE Publishing, 2022 25. What is the "hallmark of test validity"? a. criterion b. content c. predictive d. construct Ans: A Learning Objective: 8-8: Summarize alternative perspectives on validity.

Cognitive Domain: Comprehension Answer Location: Other Perspectives on Validity Difficulty Level: Medium True/False 1. Internal structure validity is the degree to which a test covers less material (e.g., items, concepts) than it should if it was intended to reflect a particular construct.

Ans: F Learning Objective: 8-3: Identify how an expert rating of test content is an important facet of validity.

Cognitive Domain: Knowledge Answer Location: Validity Evidence: Test Content Difficulty Level: Easy 2. Construct-irrelevant content is the degree to which a test covers material (e.g., items, concepts) then it should not cover if it was intended to reflect a particular construct.

Ans: T Learning Objective: 8-3: Identify how an expert rating of test content is an important facet of validity.

Cognitive Domain: Knowledge Answer Location: Threats to Construct Validity Difficulty Level: Easy 3. Validity can, in general, be seen as a concern regarding the meaning/interpretation of test scores.

Ans: F Learning Objective: 8-4: Summarize how factor analysis is used to evaluate the internal structure of psychological tests.

Cognitive Domain: Comprehension Answer Location: Factor-Analytic Evidence Difficulty Level: Medium 4. Response process validity is the degree to which the psychological mechanisms that drive individuals’ responses match the mechanisms that should drive their responses.

Ans: T

Instructor Resource Furr, Psychometrics, 4e SAGE Publishing, 2022 Learning Objective: 8-5: Describe the response process as a type of validity evidence and how some procedures are relatively direct or indirect.

Cognitive Domain: Knowledge Answer Location: Validity Evidence: Response Processes Difficulty Level: Easy 5. Convergent validity is the degree to which a test’s scores are correlated (either positive or negatively) with other measures that they should be correlated with.

Ans: T Learning Objective: 8-6: Discuss validity evidence’s association between test scores and other variances, including convergent, discriminant, criterion, concurrent, and predictive evidence.

Cognitive Domain: Knowledge Answer Location: Validity Evidence: Associations With Other Variables Difficulty Level: Easy 6. Discriminant validity is the degree to which a test’s scores are related to a particularly important outcome variable that they should be correlated with.

Ans: F Learning Objective: 8-6: Discuss validity evidence’s association between test scores and other variances, including convergent, discriminant, criterion, concurrent, and predictive evidence.

Cognitive Domain: Knowledge Answer Location: Validity Evidence: Associations With Other Variables Difficulty Level: Easy 7. Construct validity is the degree to which test scores can be interpreted as reflecting a particular psychological characteristic.

Ans: T Learning Objective: 8-6: Discuss validity evidence’s association between test scores and other variances, including convergent, discriminant, criterion, concurrent, and predictive evidence.

Cognitive Domain: Knowledge Answer Location: Validity Evidence: Associations With Other Variables Difficulty Level: Easy 8. Face validity is the degree to which test scores can be interpreted as reflecting a particular psychological characteristic.

Ans: F Learning Objective: 8-6: Discuss validity evidence’s association between test scores and other variances, including convergent, discriminant, criterion, concurrent, and predictive evidence.

Cognitive Domain: Knowledge Answer Location: Validity Evidence: Associations With Other Variables Difficulty Level: Easy

Instructor Resource Furr, Psychometrics, 4e SAGE Publishing, 2022 9. Consequential validity is the degree to which a measure’s actual relationships with other measures correspond to the relationships that the test should have with those other measures.

Ans: F Learning Objective: 8-7: Recall the three key issues regarding consequential validity, including the intended and actual consequences of test use.

Cognitive Domain: Knowledge Answer Location: Validity Evidence: Consequences of Testing Difficulty Level: Easy 10. Consequential validity is the degree to which the actual effects of using a measure correspond with the effects that should be seen.

Ans: T Learning Objective: 8-7: Recall the three key issues regarding consequential validity, including the intended and actual consequences of test use.

Cognitive Domain: Knowledge Answer Location: Validity Evidence: Consequences of Testing Difficulty Level: Easy Short Answer 1. What is one of the key points about the definition of validity?

Ans: Answer may vary.

Learning Objective: 8-1: Describe how the concept of validity has evolved over time.

Cognitive Domain: Comprehension Answer Location: What Is Validity?

Difficulty Level: Medium 2. What term is the degree to which a test covers material (e.g., items, concepts) that it should cover (if it is to reflect a particular psychological construct)?

Ans: Answer may vary.

Learning Objective: 8-3: Identify how an expert rating of test content is an important facet of validity.

Cognitive Domain: Knowledge Answer Location: Validity Evidence: Test Content Difficulty Level: Easy 3. What term is the degree to which a test has a dimensionality that it should have (if it is to reflect a particular psychological construct)?

Ans: Answer may vary.

Learning Objective: 8-3: Identify how an expert rating of test content is an important facet of validity.

Cognitive Domain: Knowledge Answer Location: Validity Evidence: Test Content Difficulty Level: Easy

Instructor Resource Furr, Psychometrics, 4e SAGE Publishing, 2022 4. How is evidence of content validity typically obtained?

Ans: Answer may vary.

Learning Objective: 8-3: Identify how an expert rating of test content is an important facet of validity.

Cognitive Domain: Comprehension Answer Location: Expert Rating Evidence Difficulty Level: Medium 5. What term means the degree to which a test appears to be related to a specific construct in the judgment of non-experts?

Ans: Answer may vary.

Learning Objective: 8-3: Identify how an expert rating of test content is an important facet of validity.

Cognitive Domain: Knowledge Answer Location: Content Validity Versus Face Validity Difficulty Level: Easy 6. If MSEI scores are validly interpreted and responses exhibit a structure that is consistent with a multidimensional conceptual definition of the construct, should the items appear as separate clusters or one large cluster?

Ans: Answer may vary.

Learning Objective: 8-4: Summarize how factor analysis is used to evaluate the internal structure of psychological tests.

Cognitive Domain: Analysis Answer Location: Validity Evidence: Internal Structure of the Test Difficulty Level: Medium 7. What is a form of indirect evidence?

Ans: Answer may vary.

Learning Objective: 8-5: Describe the response process as a type of validity evidence and how some procedures are relatively direct or indirect.

Cognitive Domain: Comprehension Answer Location: Indirect Evidence Difficulty Level: Medium 8. What term means the degree to which a test’s scores are related to a particularly important outcome variable that is measured at the same time that the test itself is completed?

Ans: Answer may vary.

Learning Objective: 8-6: Discuss validity evidence’s association between test scores and other variances, including convergent, discriminant, criterion, concurrent, and predictive evidence.

Cognitive Domain: Knowledge Answer Location: Validity Evidence: Associations With Other Variables Difficulty Level: Easy

Instructor Resource Furr, Psychometrics, 4e SAGE Publishing, 2022 9. What term means the degree to which a test’s scores are related to a particularly important outcome variable that is measured at a later time (than when the test itself is completed)?

Ans: Answer may vary.

Learning Objective: 8-6: Discuss validity evidence’s association between test scores and other variances, including convergent, discriminant, criterion, concurrent, and predictive evidence.

Cognitive Domain: Knowledge Answer Location: Validity Evidence: Associations With Other Variables Difficulty Level: Easy 10. What term means the degree to which a test’s scores are uncorrelated (either positive or negatively) with other measures that they should not be correlated with?

Ans: Answer may vary.

Learning Objective: 8-6: Discuss validity evidence’s association between test scores and other variances, including convergent, discriminant, criterion, concurrent, and predictive evidence.

Cognitive Domain: Knowledge Answer Location: Validity Evidence: Associations With Other Variables Difficulty Level: Easy

Instructor Resource Furr, Psychometrics, 4e SAGE Publishing, 2022 Chapter 9: Estimating and Evaluating Convergent and Discriminant Validity Evidence Test Bank Multiple Choice 1. Why is a construct’s nomological network important in terms of construct validity? a. It leads test-developers to form hypotheses about the other measures that a test should (and should not) be correlated with, thus guiding the design and evaluation of validity studies b. It tells test-developers which construct would be most appropriate for a pre-specified method of assessment, thus guiding test evaluation c. It tells researchers which “informants” from participants’ social networks would be most likely to provide valid ratings of the participants, guiding the design of validity studies. d. It tells researchers which content/items should be included in a measure, thereby guiding the evaluation of content validity.

Ans: A Learning Objective: 9-1: Summarize how researchers use their understanding of a construct’s nomological network when examining construct validity.

Cognitive Domain: Analysis Answer Location: A Construct’s Nomological Network Difficulty Level: Medium 2. A construct’s ____________________ is the set of constructs, behaviors, and/or properties that are (theorized to be) associated with a construct. It dictates a particular pattern of associations among measures of those constructs, behaviors, and properties. a. convergent validity b. discriminant validity c. nomological network d. multitrait-multimethod matrix Ans: C Learning Objective: 9-1: Summarize how researchers use their understanding of a construct’s nomological network when examining construct validity.

Cognitive Domain: Knowledge Answer Location: A Construct’s Nomological Network Difficulty Level: Easy 3. When evaluating her new measure of Schizoid PD, Sarah finds that its scores are correlated with clinicians’ diagnoses of an entirely different PD (Obsessive-Compulsive) at r =.80, p <.05. This strong positive correlation would be evidence of: a. poor discriminant validity b. good convergent validity

Instructor Resource Furr, Psychometrics, 4e SAGE Publishing, 2022 c. good structural validity d. construct bias Ans: A Learning Objective: 9-2: Describe the methods used to evaluate the degree to which measures show convergent and discriminate associations.

Cognitive Domain: Analysis Answer Location: Methods for Evaluating Convergent and Discriminant Validity Difficulty Level: Medium 4. Mayes and Ganster (1983) conducted an MTMM analysis of two methods of measuring four psychological needs: • Two methods—the Manifest Needs Questionnaire (MNQ) and the Personality Research Form (PRF) • Each method included scales measuring four different psychological needs achievement, autonomy, affiliation, and power (i.e., these are hypothesized to be relatively independent needs).

Below is the key Table from their article. The left-hand part of the table includes the scale labels, along with their means, standard deviations, and alpha values. In the righthand part of the table is an MTMM matrix.

From the perspective of an MTMM, the correlations in the “unboxed diagonal” above (i.e., .55,.38, .48, and .74) reflect what? a. content validity b. convergent validity c. discriminant validity d. predictive validity Ans: B Learning Objective: 9-2: Describe the methods used to evaluate the degree to which measures show convergent and discriminate associations.

Cognitive Domain: Analysis Answer Location: Methods for Evaluating Convergent and Discriminant Validity Difficulty Level: Medium

Instructor Resource Furr, Psychometrics, 4e SAGE Publishing, 2022 5. ____________ studies are intended to evaluate the predictive utility of a test’s scores across a range of settings, times, situations, and so on. It is a form of meta-analysis; it combines the results of several smaller individual studies into one large analysis a. nomological network b. multitrait-multimethod matrix c. validity generalization d. focused examinations Ans: C Learning Objective: 9-2: Describe the methods used to evaluate the degree to which measures show convergent and discriminate associations.

Cognitive Domain: Knowledge Answer Location: Focused Associations Difficulty Level: Easy 6. As part of creating a new self-report measure of self-esteem, the test developer conducts a validity study. She administers her new scale to a sample of respondents, who also complete 10 additional self-report scales. These scales are measures of constructs that she thinks will show varying levels of association with self-esteem— some should be positively correlated, some negatively correlated, and some uncorrelated. After getting the respondents’ data, she computes the actual correlations between her new scale and all the other scales. She then eyeballs them somewhat *unsystematically* and declares that they provide evidence for the construct validity of her scale. She is using which kind of approach, as discussed in the book? a. convergent validity b. sets of correlations c. multitrait-multimethod matrix d. QCV Ans: B Learning Objective: 9-2: Describe the methods used to evaluate the degree to which measures show convergent and discriminate associations.

Cognitive Domain: Analysis Answer Location: Sets of Correlations Difficulty Level: Medium 7. The use of an MTMM matrix is based on which realization? a. Self-report questionnaires are (believed to be) prone to error from respondents’ response biases. b. The magnitude of a validity correlation is affected by both: a) the actual association between constructs, and b) similarity or difference in the methods by which those constructs are measured. c. The correlation between a test and an important criterion might vary from one sample or context to another. d. If a test developer unsystematically eyeballs a set of convergent and discriminant validity correlations, then she or he might interpret the results in subjective and inaccurate ways.

Instructor Resource Furr, Psychometrics, 4e SAGE Publishing, 2022 Ans: B Learning Objective: 9-2: Describe the methods used to evaluate the degree to which measures show convergent and discriminate associations.

Cognitive Domain: Comprehension Answer Location: Multitrait-Multimethod Matrices Difficulty Level: Medium 8. Based upon the logic of the MTMM matrix, which of the following correlations would you expect to be the largest? a. The correlation between self-esteem scores from two self-report scales b. The correlation between self-esteem scores from a self-report measure of selfesteem, and self-esteem scores from an “informant report” method of measurement c. The correlation between self-esteem scores and extraversion scores, where both are measured by self-report methods d. The correlation between self-esteem scores from a self-report measure of selfesteem, and extraversion scores from an “informant report” method of measurement Ans: A Learning Objective: 9-2: Describe the methods used to evaluate the degree to which measures show convergent and discriminate associations.

Cognitive Domain: Analysis Answer Location: Multitrait-Multimethod Matrices Difficulty Level: Medium 9. In the parlance of the MTMM matrix, which of the following is the correct label for the correlation between self-esteem scores from a self-report measure of self-esteem, and extraversion scores from an “informant report” method of measurement? a. monotrait-monomethod correlation b. monotrait-heteromethod correlation c. heterotrait-monomethod correlation d. heterortrait-heteromethod correlation Ans: D Learning Objective: 9-2: Describe the methods used to evaluate the degree to which measures show convergent and discriminate associations.

Cognitive Domain: Analysis Answer Location: Multitrait-Multimethod Matrices Difficulty Level: Medium 10. As part of creating a new self-report measure of self-esteem, the test developer conducts a validity study. She administers her new scale to a sample of respondents, who also complete 10 additional self-report scales. These scales are measures of constructs that she thinks will show varying levels of association with self-esteem— some should be positively correlated, some negatively correlated, and some uncorrelated. Based on these expectations, she generates a clear set of hypothesized or “predicted” correlations. After getting the respondents’ data, she computes the actual correlations between her new scale and all the other scales. She then uses statistical

Instructor Resource Furr, Psychometrics, 4e SAGE Publishing, 2022 procedures to systematically compare the actual correlations to her predicted correlations. She is using which kind of approach, as discussed in the book? a. convergent validity b. sets of correlations c. multitrait-multimethod matrix d. QCV Ans: D Learning Objective: 9-2: Describe the methods used to evaluate the degree to which measures show convergent and discriminate associations.

Cognitive Domain: Analysis Answer Location: Quantifying Construct Validity Difficulty Level: Medium 11. What problem is the QCV procedure intended to avoid? a. Self-report questionnaires are (believed to be) prone to error from respondents’ response biases. b. The magnitude of a validity correlation is affected by both: a) the actual association between constructs, and b) similarity or difference in the methods by which those constructs are measured. c. The correlation between a test and an important criterion might vary from one sample or context to another. d. If a test developer unsystematically eyeballs a set of convergent and discriminant validity correlations, then she or he might interpret the results in subjective and inaccurate ways.

Ans: D Learning Objective: 9-2: Describe the methods used to evaluate the degree to which measures show convergent and discriminate associations.

Cognitive Domain: Comprehension Answer Location: Quantifying Construct Validity Difficulty Level: Medium 12. Say that the test developer is designing a study to evaluate the convergent validity of her new self-report measure of extraversion. She is considering two studies. In study A, she would administer her new measure, along with a self-report measure of happiness. In study B, she would administer her new measure to a sample of participants, and she would observe each participant’s behavior in an in-lab social interaction. Based on those behavioral observations, she would then rate each participant’s apparent happiness. In both studies, she would correlate her scale’s scores with scores on happiness (whichever way it had been measured). Setting other factors aside from a moment (e.g., random measurement error), what would she expect about a difference between her findings from these two studies? a. The validity correlation would be higher in Study A than in Study B b. The validity correlation would be higher in Study B than in Study A c. The validity correlation would be identical across the studies d. The validity correlation could not be determined for Study A or B.

Instructor Resource Furr, Psychometrics, 4e SAGE Publishing, 2022 Ans: A Learning Objective: 9-2: Describe the methods used to evaluate the degree to which measures show convergent and discriminate associations.

Cognitive Domain: Analysis Answer Location: Quantifying Construct Validity Difficulty Level: Medium 13. Say that the test developer is designing a study to evaluate the convergent validity of her new self-report measure of extraversion. She is considering two studies. In study A, she would administer her new measure to a sample of participants, and she would observe each participant’s behavior in one in-lab social interaction. Based on those behavioral observations, she would then rate each participant’s apparent happiness. In study B, she would administer her new measure to a sample of participants, and she would observe each participant’s behavior in five separate in-lab social interactions.

Based on those behavioral observations, she would then rate each participant’s apparent happiness within each of the five interactions. She would then aggregate (e.g., average) the five happiness ratings for each participant, to get an “overall happiness” score for each participant. In both studies, she would correlate her scale’s scores with scores on happiness (whichever way it had been measured). What would she expect about a difference between her findings from these two studies? a. The validity correlation would be higher in Study A than in Study B b. The validity correlation would be higher in Study B than in Study A c. The validity correlation would be identical across the studies d. The validity correlation could not be determined for Study A or B. Ans: B Learning Objective: 9-2: Describe the methods used to evaluate the degree to which measures show convergent and discriminate associations.

Cognitive Domain: Analysis Answer Location: Quantifying Construct Validity Difficulty Level: Medium 14. Courtney is examining the validity of a new scale she created. She runs a QCV analysis and obtains a large and significant effect (e.g., rcontrast and ralerting > .80). Which is the most accurate interpretation of these results? a. The pattern of actual convergent/discriminant validity correlations closely matches the pattern of hypothesized correlations. b. The pattern of actual convergent/discriminant validity correlations does not match the pattern of hypothesized correlations. c. The monotrait heteromethod correlations are robustly greater than the heterotrait monomethod correlations d. The monotrait heteromethod correlations are robustly smaller than the heterotrait monomethod correlations Ans: A Learning Objective: 9-2: Describe the methods used to evaluate the degree to which measures show convergent and discriminate associations.

Cognitive Domain: Analysis

Instructor Resource Furr, Psychometrics, 4e SAGE Publishing, 2022 Answer Location: Quantifying Construct Validity Difficulty Level: Medium 15. Sarah develops a new measure of Schizoid Personality Disorder, and she examines its validity (as a measure of Schizoid PD). She administers her new measure to a sample of participants. In addition, she also asks the participants to complete existing measures of six other personality disorders (Histrionic, Borderline, etc., see below).

Based on her understanding of Schizoid PD and the other PDs, she generates a set of “predicted corrections”. She believes that, if scores on her new measure are indeed validly interpretable in terms of Schizoid PD, then her new scale will be correlated with scores on the Histrionic PD measure at r= .60, correlated with scores on the Borderline PD measure at r = .30, etc. The table below presents the six predicted correlations.

It also presents two potential sets of actual correlations that she might observe (based on her participants’ actual responses to the measures).

Histrionic Borderline Dependent Antisocial Narcissistic Paranoid Predicted .60 .30 .10 .00 .00 −.10 Actual (A) Actual (B)

.02

-.11

.72

.29

.25

.50

.50

.40

.15

.01

-.05

-.20

Which type of validity evidence does this question illustrate? a. content validity evidence b. convergent/discriminant validity evidence c. response process validity evidence d. consequential validity evidence Ans: B Learning Objective: 9-2: Describe the methods used to evaluate the degree to which measures show convergent and discriminate associations.

Cognitive Domain: Analysis Answer Location: Quantifying Construct Validity Difficulty Level: Medium 16. Say that, in psychological reality, extraversion and happiness are positively linked— relatively extraverted people tend to be relatively happy, whereas relatively introverted people tend to be less happy. Let’s say that the “true” correlation between extraversion and happiness is r = .40. Finally, let’s say that the test-developer measures both constructs in a sample of participants, where the reliabilities of the two measures are .75 and .60. According to the assumptions of classical test theory, what will the correlation between the measures be? a. The correlation will be positive and greater than .40 b. The correlation will be positive but less than .40

Instructor Resource Furr, Psychometrics, 4e SAGE Publishing, 2022 c. The correlation will be zero. d. The correlation will be negative Ans: B Learning Objective: 9-3: Explain what factors affect the size of a validity correlation and their effects.

Cognitive Domain: Application Answer Location: Random Measurement Error and Reliability Difficulty Level: Hard 17. Say that, in a large population of university students, extraversion and happiness are positively linked—relatively extraverted people tend to be relatively happy, whereas relatively introverted people tend to be less happy. Let’s say that the correlation between extraversion and happiness is r = .40 in the population of students. Finally, let’s say that the test-developer measures both constructs among students who are recruited from a University’s campus counseling center (and thus tend to be relatively low on happiness, on average). Setting other factors aside for a moment (e.g., measurement error), what will the correlation between the measures be within this sub-sample of the population? a. The correlation will be positive and greater than .40 b. The correlation will be positive but less than .40 c. The correlation will be zero d. The correlation will be negative Ans: B Learning Objective: 9-3: Explain what factors affect the size of a validity correlation and their effects.

Cognitive Domain: Application Answer Location: Random Measurement Error and Reliability Difficulty Level: Hard 18. All else being equal, validity coefficients based on correlations between variables measured at different times are likely to be _________ coefficients based on correlations between variables measured at a single point in time. a. larger than b. smaller than c. unreliable compared to d. immeasurable compared to Ans: B Learning Objective: 9-3: Explain what factors affect the size of a validity correlation and their effects.

Cognitive Domain: Analysis Answer Location: Time Difficulty Level: Medium 19. Test developers, evaluators, and users must decide whether __________ are large enough to provide compelling evidence of convergence or if they are small enough to assure discriminant validity.

Instructor Resource Furr, Psychometrics, 4e SAGE Publishing, 2022 a. validity coefficients b. relative proportions c. restricted ranges d. single events Ans: A Learning Objective: 9-4: Discuss how test developers, evaluators, and users interpret validity coefficients.

Cognitive Domain: Comprehension Answer Location: Interpreting a Validity Coefficient Difficulty Level: Medium 20. Two criticisms based on this approach are that it is technically incorrect in some cases and that variance itself is based on a nonintuitive metric. a. discriminant validity b. focused associations c. Taylor-Russell tables d. squared correlation Ans: D Learning Objective: 9-4: Discuss how test developers, evaluators, and users interpret validity coefficients.

Cognitive Domain: Comprehension Answer Location: Squared Correlations and “Variance Explained” Difficulty Level: Medium 21. Within a population of university students, variables X and Y are uncorrelated with each other (r = .00). A researcher recruits a sample of 100 students from the university, measures variable X and variable Y, and computes the correlation. Which of the following is the *least* likely result that she would get in her sample?: a. r = -.10 b. r = .10 c. r = .60 d. r = -.70 Ans: D Learning Objective: 9-4: Discuss how test developers, evaluators, and users interpret validity coefficients.

Cognitive Domain: Application Answer Location: Estimating Practical Effects: Binomial Effect Size Display, TaylorRussell Tables, Utility Analysis, and Sensitivity/Specificity Difficulty Level: Hard 22. Procedures such as the BESD and the Taylor-Russell tables can be used to do what? a. Help researchers gauge the confidence in using a sample’s information to make conclusions about a population of individuals

Instructor Resource Furr, Psychometrics, 4e SAGE Publishing, 2022 b. Help researchers gauge the effects of method variance on validity correlations (or other effect sizes) c. Help test-users understand the range of values within which their “true” trait levels likely lie d. Help researchers translate an effect size into its practical implications Ans: D Learning Objective: 9-4: Discuss how test developers, evaluators, and users interpret validity coefficients.

Cognitive Domain: Comprehension Answer Location: Estimating Practical Effects: Binomial Effect Size Display, TaylorRussell Tables, Utility Analysis, and Sensitivity/Specificity Difficulty Level: Medium 23. Say that the test developer wants to use her newly developed brief test of intelligence to assess intellectual disability (ID). To determine whether the test’s scores lead to the correct identification of individuals who have ID, she compares her “brief test” scores to the results of an in-depth cognitive assessment (seen as the gold standard of diagnosing ID). Let’s say that she conducts a study in which participants are assessed via her new test and an in-depth interview. Comparing the scores, she finds a “sensitivity” of .90. What does this mean? a. Of the individuals who the test identifies as having ID, 90% truly have ID. b. 90% of the individuals who truly have ID are correctly identified (by the new test) as having ID c. 90% of the individuals who truly do NOT have ID are correctly identified (by the test) as not having ID d. It reflects the ability of the test to identify people who do not have ID Ans: B Learning Objective: 9-4: Discuss how test developers, evaluators, and users interpret validity coefficients.

Cognitive Domain: Analysis Answer Location: Estimating Practical Effects: Binomial Effect Size Display, TaylorRussell Tables, Utility Analysis, and Sensitivity/Specificity Difficulty Level: Medium 24. Psychologists have long discussed what “counts” as small, medium, and large correlations. What are Hemphill (2003)’s guidelines for this in psychology? a. small < .50, medium .50 to .80, large > .80 b. small < .30, medium .30 to .70, large > .70 c. small < .20, medium .20 to .30, large > .30 d. small < .05, medium .05 to .15, large > .15 Ans: C Learning Objective: 9-4: Discuss how test developers, evaluators, and users interpret validity coefficients.

Cognitive Domain: Analysis

Instructor Resource Furr, Psychometrics, 4e SAGE Publishing, 2022 Answer Location: Guidelines or Norms for a Field Difficulty Level: Medium 25. Inferential statistics are used to: a. help test-administrators gauge the confidence that they should have when using an individual’s test score to estimate something about the individual’s psychological characteristics b. help test-users gauge the confidence that they should have when making a real-life decision (e.g., hiring) based upon an individual’s test score c. help researchers gauge the confidence that they should have when using a sample of participants to learn about a population d. help researchers gauge the confidence that they should have when using a population to learn about a sample of participants Ans: C Learning Objective: 9-4: Discuss how test developers, evaluators, and users interpret validity coefficients.

Cognitive Domain: Comprehension Answer Location: Statistical Significance Difficulty Level: Medium True/False 1. A large association between constructs will not decrease the size of a validity coefficient between a test and some outcome (assuming all else is equal).

Ans: T Learning Objective: 9-3: Explain what factors affect the size of a validity correlation and their effects.

Cognitive Domain: Analysis Answer Location: Factors Affecting a Validity Coefficient Difficulty Level: Medium 2. If two constructs are strongly associated with each other, then measures of those constructs will probably be highly correlated with each other.

Ans: T Learning Objective: 9-3: Explain what factors affect the size of a validity correlation and their effects.

Cognitive Domain: Analysis Answer Location: Associations Between Constructs Difficulty Level: Medium 3. There are clear, simple guidelines about detecting range restriction.

Ans: F Learning Objective: 9-3: Explain what factors affect the size of a validity correlation and their effects.

Cognitive Domain: Comprehension

Instructor Resource Furr, Psychometrics, 4e SAGE Publishing, 2022 Answer Location: Restricted Range Difficulty Level: Medium 4. Dissimilarly shaped distributions will not decrease the size of a validity coefficient between a test and some outcome (assuming all else is equal).

Ans: F Learning Objective: 9-3: Explain what factors affect the size of a validity correlation and their effects.

Cognitive Domain: Analysis Answer Location: Skew and Relative Proportions Difficulty Level: Medium 5. Validity is sometimes evaluated by examining the correlation between a test given at one point in time and a criterion variable measured at a later point in time.

Ans: T Learning Objective: 9-3: Explain what factors affect the size of a validity correlation and their effects.

Cognitive Domain: Comprehension Answer Location: Time Difficulty Level: Medium 6. Validity coefficients can be affected by an observation of a single event alone.

Ans: F Learning Objective: 9-3: Explain what factors affect the size of a validity correlation and their effects.

Cognitive Domain: Comprehension Answer Location: Predictions of Single Events Difficulty Level: Medium 7. There is difficulty in predicting single events.

Ans: T Learning Objective: 9-3: Explain what factors affect the size of a validity correlation and their effects.

Cognitive Domain: Comprehension Answer Location: Predictions of Single Events Difficulty Level: Medium 8. Validity coefficients must be interpreted.

Ans: T Learning Objective: 9-4: Discuss how test developers, evaluators, and users interpret validity coefficients.

Cognitive Domain: Comprehension Answer Location: Interpreting a Validity Coefficient Difficulty Level: Medium 9. Interpreting a squared correlation is an uncommon practice.

Instructor Resource Furr, Psychometrics, 4e SAGE Publishing, 2022 Ans: F Learning Objective: 9-4: Discuss how test developers, evaluators, and users interpret validity coefficients.

Cognitive Domain: Comprehension Answer Location: Squared Correlations and “Variance Explained” Difficulty Level: Medium 10. Let’s say that the convergent validity correlation between a newly developed intelligence test and the Wechsler Adult Intelligence Scale (WAIS) is .50, the following is likely true: This is a small correlation by Cohen’s widely cited benchmarks for psychology.

Ans: F Learning Objective: 9-4: Discuss how test developers, evaluators, and users interpret validity coefficients.

Cognitive Domain: Analysis Answer Location: Statistical Significance Difficulty Level: Medium

Short Answer 1. Sarah develops a new measure of Schizoid Personality Disorder, and she examines its validity (as a measure of Schizoid PD). She administers her new measure to a sample of participants. In addition, she also asks the participants to complete existing measures of six other personality disorders (Histrionic, Borderline, etc., see below).

Based on her understanding of Schizoid PD and the other PDs, she generates a set of “predicted corrections”. She believes that, if scores on her new measure are indeed validly interpretable in terms of Schizoid PD, then her new scale will be correlated with scores on the Histrionic PD measure at r= .60, correlated with scores on the Borderline PD measure at r = .30, etc. The table below presents the six predicted correlations.

It also presents two potential sets of actual correlations that she might observe (based on her participants’ actual responses to the measures).

Histrionic Borderline Dependent Antisocial Narcissistic Paranoid Predicted .60 .30 .10 .00 .00 −.10 Actual (A) Actual (B)

.02

-.11

.72

.29

.25

.50

.50

.40

.15

.01

-.05

-.20

Which of the two sets of potential actual correlations would provide greater evidence of validity, set A or B?

Instructor Resource Furr, Psychometrics, 4e SAGE Publishing, 2022 Ans: Answer may vary.

Learning Objective: 9-2: Describe the methods used to evaluate the degree to which measures show convergent and discriminate associations.

Cognitive Domain: Analysis Answer Location: Focused Associations Difficulty Level: Medium 2. What is an example of a restricted range?

Ans: Answer may vary.

Learning Objective: 9-3: Explain what factors affect the size of a validity correlation and their effects.

Cognitive Domain: Comprehension Answer Location: Restricted Range Difficulty Level: Medium 3. What is a benefit of self-report data?

Ans: Answer may vary.

Learning Objective: 9-3: Explain what factors affect the size of a validity correlation and their effects.

Cognitive Domain: Comprehension Answer Location: Method Variance Difficulty Level: Medium 4. When two variables are measured through different methods of assessment, are they more strongly or poorly correlated with each other, compared to when two variables are measured through the same method?

Ans: Answer may vary.

Learning Objective: 9-3: Explain what factors affect the size of a validity correlation and their effects.

Cognitive Domain: Analysis Answer Location: Method Variance Difficulty Level: Medium 5. Based on the previous question and your validity correlation of .56, which of the tables below (A or B) correctly reflects the BESD “translation” of that correlation?

Test score

A

B

Below average Above average

Criterion performance Below Above Average average 78 22 22

78

Criterion performance

Test score

Instructor Resource Furr, Psychometrics, 4e SAGE Publishing, 2022

Below average Above average

Below Average 56

Above average 44

44

56

Ans: Answer may vary.

Learning Objective: 9-4: Discuss how test developers, evaluators, and users interpret validity coefficients.

Cognitive Domain: Analysis Answer Location: Estimating Practical Effects: Binomial Effect Size Display, TaylorRussell Tables, Utility Analysis, and Sensitivity/Specificity Difficulty Level: Medium 6. What is a factor that can affect validity coefficients?

Ans: Answer may vary.

Learning Objective: 9-4: Discuss how test developers, evaluators, and users interpret validity coefficients.

Cognitive Domain: Comprehension Answer Location: Predictions of Single Events Difficulty Level: Medium 7. You develop a new measure, and you do a study to evaluate its criterion validity. In your study, you obtain a validity correlation of r = .56. According to Cohen’s well-known guidelines, is this value small or large?

Ans: Answer may vary.

Learning Objective: 9-4: Discuss how test developers, evaluators, and users interpret validity coefficients.

Cognitive Domain: Analysis Answer Location: Guidelines or Norms for a Field Difficulty Level: Medium 8. For her MA thesis, a student develops a new measure of moral relativism. In attempting to validate this measure, she correlates it with a variety of other measures— some of which she believes should be correlated with her measure and some of which she believes should be uncorrelated with her measure. She finds that her measure is statistically significant (p <.05) correlated with all the other measures. Her advisor (who has not taken a course in psychometrics) initially expresses concern that this is a problem for the discriminant validity of her new measure. The student (who has taken a course in psychometrics) explains, however, that before being so pessimistic, they should consider the sample size in her study. Which of the following could be a legitimate explanation of her finding that all correlations were statistically significant, A or B?

Instructor Resource Furr, Psychometrics, 4e SAGE Publishing, 2022 a. Since they conducted their study (and got correlations) based upon a very large sample of participants (N = 10,000 participants), their correlations could be statistically significant even if they were very small in magnitude b. Since they conducted their study (and got correlations) based upon a very small sample of participants (N = 20 participants), their correlations could be statistically significant even if they were very small in magnitude Ans: Answer may vary.

Learning Objective: 9-4: Discuss how test developers, evaluators, and users interpret validity coefficients.

Cognitive Domain: Analysis Answer Location: Statistical Significance Difficulty Level: Medium 9. Imagine that the test developer conducts a validity study for her new measure of extraversion but finds a convergent validity correlation (between extraversion and happiness) that is quite a bit lower than she hypothesized. What explanation might account for this finding?

Ans: Answer may vary.

Learning Objective: 9-4: Discuss how test developers, evaluators, and users interpret validity coefficients.

Cognitive Domain: Comprehension Answer Location: Statistical Significance Difficulty Level: Medium 10. Let’s say that the convergent validity correlation between a newly developed intelligence test and the Wechsler Adult Intelligence Scale (WAIS) is found NOT to be statistically significant in a sample of participants. What could be a plausible explanation for such a finding? Either 1) The new intelligence scale is too strongly correlated with the WAIS, or 2) The sample of participants was too small to detect the association.

Ans: Answer may vary.

Learning Objective: 9-4: Discuss how test developers, evaluators, and users interpret validity coefficients.

Cognitive Domain: Analysis Answer Location: Statistical Significance Difficulty Level: Medium

Instructor Resource Furr, Psychometrics, 4e SAGE Publishing, 2022 Chapter 10: Response Biases Test Bank Multiple Choice 1. As students in Introductory Psychology class, Marie and Martin participate in a research study. They complete a survey about “attitude towards alcohol use.” Let’s say that they actually have the same level of that attitude—on a scale of 1 (totally antialcohol) to 10 (totally enthusiastic about alcohol), they are both “truly” 1. However, when completing the survey, they provide slightly different answers. Marie is willing to use the scale minimum (of 1), whereas Martin shies away from making such an intense response (e.g., he considers the fact that he does go to parties with alcohol, so he doesn’t feel “justified” in making the lowest possible response). So, although they do have the same attitude, they end up with different scores on the survey. This situation is an example of which response bias? a. acquiescence bias b. extremity bias c. social desirability bias (impression management) d. social desirability bias (self-deception) Ans: B Learning Objective: 10-1: Summarize the six different response biases and how they can potentially compromise the quality of psychological measurement.

Cognitive Domain: Analysis Answer Location: Acquiescence Bias (“Yea-Saying and Nay-Saying”) Difficulty Level: Medium 2. Marie applies for graduate school in Clinical Psychology, and she is asked to take a personality test as part of the admissions process. One of the test’s items states “I am persistent and a hard worker”, and she has to respond “true” or “false” about herself.

Marie knows that, while she’s very smart, she’s not usually a very hard worker and doesn’t persist when things get tough. However, she also suspects that her chances of admission will increase if she responds “true” to the item, which she does. Her response reflects which type of response bias?: a. acquiescence bias b. extremity bias c. social desirability bias (impression management) d. social desirability bias (self-deception) Ans: C Learning Objective: 10-1: Summarize the six different response biases and how they can potentially compromise the quality of psychological measurement.

Cognitive Domain: Analysis Answer Location: Acquiescence Bias (“Yea-Saying and Nay-Saying”) Difficulty Level: Medium

Instructor Resource Furr, Psychometrics, 4e SAGE Publishing, 2022 3. ________________ is defined as consistently endorsing (or consistently rejecting) items without much regard for their content a. acquiescence bias b. extremity bias c. social desirability bias (impression management) d. social desirability bias (self-deception) Ans: A Learning Objective: 10-1: Summarize the six different response biases and how they can potentially compromise the quality of psychological measurement.

Cognitive Domain: Knowledge Answer Location: Acquiescence Bias (“Yea-Saying and Nay-Saying”) Difficulty Level: Easy 4. Which of the following could potentially affect response to tests of knowledge, ability, achievement, or aptitude (i.e., tests in which there are presumably right or wrong answers)? a. acquiescence bias b. extremity bias c. social desirability bias (impression management) d. guessing Ans: D Learning Objective: 10-1: Summarize the six different response biases and how they can potentially compromise the quality of psychological measurement.

Cognitive Domain: Comprehension Answer Location: Acquiescence Bias (“Yea-Saying and Nay-Saying”) Difficulty Level: Medium 5. Consider a very brief three-item measure of loneliness: I spend more time by myself than I’d like: Disagree____ Agree _____ I often feel lonely Disagree____ Agree _____ I wish that I had more connection to people Disagree____ Agree _____ For this measure, a respondent gets 1 “point” for each item with which she/he agrees.

The respondent’s loneliness score is then computed simply by summing these points.

From the perspective of acquiescence bias, a concern about this measure is: a. A respondent could get a relatively high score on loneliness by selecting the more intense or extreme options (regardless of whether she/he truly feels lonely). b. A respondent’s score might be a poor reflection of her/his true loneliness because the test blends easy and hard, potentially confusing the respondent’s “agreement” choices. c. An “obvious” scale like this would be easy to manipulate, in terms of appearing lonely (or not) if one wanted to. d. A respondent could get a relatively high score on loneliness just by wanting to agree with items of all types (regardless of whether she/he truly feels lonely).

Ans: D

Instructor Resource Furr, Psychometrics, 4e SAGE Publishing, 2022 Learning Objective: 10-1: Summarize the six different response biases and how they can potentially compromise the quality of psychological measurement.

Cognitive Domain: Application Answer Location: Acquiescence Bias (“Yea-Saying and Nay-Saying”) Difficulty Level: Hard 6. Consider a legal context in which an individual sues an insurance company for compensation. The individual claims that, due to an accident, he is suffering from memory impairments. As part of the lawsuit, the insurance company hires a clinical psychologist to administer a standardized memory test. Which of the following biases is most likely to be a concern (for the insurance company) in this context? a. acquiescence bias b. extremity bias c. malingering d. guessing Ans: C Learning Objective: 10-1: Summarize the six different response biases and how they can potentially compromise the quality of psychological measurement.

Cognitive Domain: Analysis Answer Location: Acquiescence Bias (“Yea-Saying and Nay-Saying”) Difficulty Level: Medium 7. Which of the following is not a valid statement about the actual occurrence and effects of various response biases? a. Acquiescence bias is most likely to occur when test-takers do not easily understand a test’s items b. Research shows that, although it’s a theoretical possibility, extremity bias does *not* seem to be a problem in practice (and thus there’s little reason to be concerned about it) c. Psychologists are debating whether the so-called social desirability response “bias” at least partly reflects meaningful personality characteristics (and is thus not “bias”). d. There is evidence that some lawyers may encourage their clients to “fake bad” on psychological assessments in some circumstances Ans: B Learning Objective: 10-1: Summarize the six different response biases and how they can potentially compromise the quality of psychological measurement.

Cognitive Domain: Analysis Answer Location: Acquiescence Bias (“Yea-Saying and Nay-Saying”) Difficulty Level: Medium 8. The use of “balanced” scales is intended to minimize the effect of which type of response bias? a. acquiescence bias b. extremity bias c. malingering d. guessing

Instructor Resource Furr, Psychometrics, 4e SAGE Publishing, 2022 Ans: A Learning Objective: 10-1: Summarize the six different response biases and how they can potentially compromise the quality of psychological measurement.

Cognitive Domain: Comprehension Answer Location: Acquiescence Bias (“Yea-Saying and Nay-Saying”) Difficulty Level: Medium 9. If an individual has a high score on the MMPI F scale, the test administrator might suspect (thought would not know for sure) that the individuals were responding with a ___________________. a. acquiescence bias b. extremity bias c. malingering d. guessing Ans: C Learning Objective: 10-1: Summarize the six different response biases and how they can potentially compromise the quality of psychological measurement.

Cognitive Domain: Comprehension Answer Location: Acquiescence Bias (“Yea-Saying and Nay-Saying”) Difficulty Level: Medium 10. The BIDR is intended to reveal ___________ bias, while the Dot Counting Task is intended to reveal ______________________ bias. a. malingering, acquiescence b. acquiescence, malingering c. social desirability, malingering d. malingering, social desirability Ans: C Learning Objective: 10-1: Summarize the six different response biases and how they can potentially compromise the quality of psychological measurement.

Cognitive Domain: Comprehension Answer Location: Acquiescence Bias (“Yea-Saying and Nay-Saying”) Difficulty Level: Medium 11. Which of the following is an example of extremity? a. A respondent could get a relatively high score on loneliness by selecting the more intense or extreme options (regardless of whether she/he truly feels lonely). b. A respondent’s score might be a poor reflection of her/his true loneliness because the test blends easy and hard, potentially confusing the respondent’s “agreement” choices. c. An “obvious” scale like this would be easy to manipulate, in terms of appearing lonely (or not) if one wanted to. d. A respondent could get a relatively high score on loneliness just by wanting to agree with items of all types (regardless of whether she/he truly feels lonely).

Ans: A

Instructor Resource Furr, Psychometrics, 4e SAGE Publishing, 2022 Learning Objective: 10-1: Summarize the six different response biases and how they can potentially compromise the quality of psychological measurement.

Cognitive Domain: Comprehension Answer Location: Extreme and Moderate Responding Difficulty Level: Medium 12. Which of the following is the definition of social desirability? a. Responding to items randomly or with very little attention to the content of the items b. Consistently overusing or under-using “extreme” response options, regardless of the respondent’s standing on the relevant psychological construct c. Responding in a way that exaggerates one’s positive qualities d. Guessing at the correct answer to an item (relevant only for tests of ability, knowledge, skill, etc.)

Ans: C Learning Objective: 10-1: Summarize the six different response biases and how they can potentially compromise the quality of psychological measurement.

Cognitive Domain: Knowledge Answer Location: Social Desirability (“Faking Good”) Difficulty Level: Easy 13. A term related to the definition of malingering is: a. intentional b. unconscious c. not related to gain d. psychosomatic symptoms Ans: A Learning Objective: 10-1: Summarize the six different response biases and how they can potentially compromise the quality of psychological measurement.

Cognitive Domain: Knowledge Answer Location: Malingering (“Faking Bad”) Difficulty Level: Easy 14. Which of the following is the definition of random/careless responding? a. Responding to items randomly or with very little attention to the content of the items b. Consistently overusing or under-using “extreme” response options, regardless of the respondent’s standing on the relevant psychological construct c. Responding in a way that exaggerates one’s positive qualities d. Guessing at the correct answer to an item (relevant only for tests of ability, knowledge, skill, etc.)

Ans: A Learning Objective: 10-1: Summarize the six different response biases and how they can potentially compromise the quality of psychological measurement.

Cognitive Domain: Knowledge Answer Location: Careless or Random Responding Difficulty Level: Easy

Instructor Resource Furr, Psychometrics, 4e SAGE Publishing, 2022 15. Which of the following is the definition of guessing? a. Responding to items randomly or with very little attention to the content of the items b. Consistently overusing or under-using “extreme” response options, regardless of the respondent’s standing on the relevant psychological construct c. Responding in a way that exaggerates one’s positive qualities d. Guessing at the correct answer to an item (relevant only for tests of ability, knowledge, skill, etc.)

Ans: D Learning Objective: 10-1: Summarize the six different response biases and how they can potentially compromise the quality of psychological measurement.

Cognitive Domain: Knowledge Answer Location: Guessing Difficulty Level: Easy 16. Psychologists have developed many strategies to help deal with response biases in general. These strategies are intended to accomplish several specific goals. Which of the following is NOT one of those goals? a. prevent or minimize the existence of bias b. minimize the effects of bias c. detect bias and intervene d. eliminate the source of bias Ans: D Learning Objective: 10-2: Discuss the ways psychologists attempt to minimize the existence and effects of bias.

Cognitive Domain: Analysis Answer Location: Methods for Coping With Response Biases Difficulty Level: Medium 17. In some situations (e.g., research), the purpose of measurement is not to shape decisions or knowledge about specific individuals. In such situations, test-takers might remain anonymous. What is the potential effect of such anonymity? a. makes respondents less likely to answer with an extremity bias b. makes respondents more likely to answer with an extremity bias c. makes respondents more willing to admit to socially undesirable qualities or behaviors d. makes respondents feel that they can respond certainly and/or strategically Ans: C Learning Objective: 10-2: Discuss the ways psychologists attempt to minimize the existence and effects of bias.

Cognitive Domain: Analysis Answer Location: Minimizing the Existence of Bias by Managing the Testing Context Difficulty Level: Medium 18. Imagine that you were asked to complete a personality inventory, and one of the items is:

Instructor Resource Furr, Psychometrics, 4e SAGE Publishing, 2022 Which of the following is more characteristic of you? ___ Creative ___ Well-adjusted What type of item is that and which response bias is it intended to deal with? a. balanced; acquiescence bias b. balanced; social desirability bias c. forced-choice; social desirability bias d. forced-choice; extremity bias Ans: C Learning Objective: 10-2: Discuss the ways psychologists attempt to minimize the existence and effects of bias.

Cognitive Domain: Analysis Answer Location: Minimizing the Existence of Bias by Managing the Testing Context Difficulty Level: Medium 19. When writing good items for a psychological measure, test developers often avoid items that might be confusing for respondents (e.g., items with double negatives). In terms of response biases, why would we avoid such items? a. such items could prevent us from detecting response biases b. such items could prevent us from intervening (e.g., through statistical control) to deal with response biases that might have occurred c. such items could increase respondent frustration and lead to careless or unmotivated responding d. such items would not allow for test developers to employ multiple strategies to identify items that will probably suffer from bias Ans: C Learning Objective: 10-2: Discuss the ways psychologists attempt to minimize the existence and effects of bias.

Cognitive Domain: Analysis Answer Location: Minimizing the Existence of Bias by Managing the Testing Context Difficulty Level: Medium 20. Which of the following is not a way test developers might be able to nullify or reduce some biases? a. complex items b. forced-choice format c. randomized response d. neutral items Ans: A Learning Objective: 10-2: Discuss the ways psychologists attempt to minimize the existence and effects of bias.

Cognitive Domain: Analysis Answer Location: Minimizing the Existence of Bias by Managing Test Content Difficulty Level: Medium

Instructor Resource Furr, Psychometrics, 4e SAGE Publishing, 2022 21. What is a balanced scale? a. a questionnaire or test that has both easy and hard items b. a questionnaire or test that splits its items among more than one psychological characteristic c. a questionnaire or test that has both extreme and moderate items d. a questionnaire or test that has both “positively keyed” and “negatively keyed” items Ans: D Learning Objective: 10-2: Discuss the ways psychologists attempt to minimize the existence and effects of bias.

Cognitive Domain: Knowledge Answer Location: Minimizing the Effects of Bias by Managing Test Content or Scoring Difficulty Level: Easy 22. What is a “validity scale”? a. Sets of items that are embedded within a large inventory and are intended to be particularly good items that were written to reflect a measured construct with as much validity as possible. b. Sets of items that are embedded within a large inventory and are intended to “trick” respondents into responding without any response biases. c. Sets of items that are embedded within a large inventory and are intended to quantify the degree to which a respondent is manifesting specific response biases d. Sets of items that are embedded within a small inventory and are intended to qualify the degree to which a respondent is responding without any response biases.

Ans: C Learning Objective: 10-3: Describe how psychologists manage test content and use specialized tests to detect bias and intervene.

Cognitive Domain: Knowledge Answer Location: Managing Test Content to Detect Bias and Intervene Difficulty Level: Easy 23. The F scale on the MMPI is composed of items that: a. most honest people endorse as true b. consisted of items that are endorsed by very few respondents c. showed a significant response shift under instructions to fake good d. differentiated normal from psychiatric samples Ans: B Learning Objective: 10-3: Describe how psychologists manage test content and use specialized tests to detect bias and intervene.

Cognitive Domain: Comprehension Answer Location: Managing Test Content to Detect Bias and Intervene Difficulty Level: Medium 24. Which validity scale includes seven scales intended to provide information about the likelihood that a respondent manifested a variety of biases? a. MMPI

Instructor Resource Furr, Psychometrics, 4e SAGE Publishing, 2022 b. MMPI-2 c. NEO-PI d. CPI Ans: B Learning Objective: 10-3: Describe how psychologists manage test content and use specialized tests to detect bias and intervene.

Cognitive Domain: Comprehension Answer Location: Managing Test Content to Detect Bias and Intervene Difficulty Level: Medium 25. What is the status of empirical evidence regarding the “validity of validity scales”? a. Despite the theoretical logic of such scales, most evidence indicates that they do not work well. b. The evidence overwhelmingly indicates that they work very well. c. Evidence regarding their effectiveness is mixed, though much research does indicate that they work fairly well. d. Despite the theoretical logic of such scales, most evidence indicates that they cannot be declared to work well or not.

Ans: C Learning Objective: 10-3: Describe how psychologists manage test content and use specialized tests to detect bias and intervene.

Cognitive Domain: Comprehension Answer Location: Managing Test Content to Detect Bias and Intervene Difficulty Level: Medium True/False 1. All bias occurs when an individual consistently agrees with statements without regard for the meaning of those statements.

Ans: F Learning Objective: 10-1: Summarize the six different response biases and how they can potentially compromise the quality of psychological measurement.

Cognitive Domain: Comprehension Answer Location: Acquiescence Bias (“Yea-Saying and Nay-Saying”) Difficulty Level: Medium 2. Each scale has its own random measurement error, which is not connected to the other.

Ans: T Learning Objective: 10-1: Summarize the six different response biases and how they can potentially compromise the quality of psychological measurement.

Cognitive Domain: Comprehension Answer Location: Acquiescence Bias (“Yea-Saying and Nay-Saying”) Difficulty Level: Medium

Instructor Resource Furr, Psychometrics, 4e SAGE Publishing, 2022 3. Acquiescence bias is only “yea-saying” and never “nay-saying.”

Ans: F Learning Objective: 10-1: Summarize the six different response biases and how they can potentially compromise the quality of psychological measurement.

Cognitive Domain: Comprehension Answer Location: Acquiescence Bias (“Yea-Saying and Nay-Saying”) Difficulty Level: Medium 4. There is a debate about the existence of acquiescence bias.

Ans: T Learning Objective: 10-1: Summarize the six different response biases and how they can potentially compromise the quality of psychological measurement.

Cognitive Domain: Comprehension Answer Location: Acquiescence Bias (“Yea-Saying and Nay-Saying”) Difficulty Level: Medium 5. The use of extreme response options is itself a bias and problem.

Ans: F Learning Objective: 10-1: Summarize the six different response biases and how they can potentially compromise the quality of psychological measurement.

Cognitive Domain: Comprehension Answer Location: Extreme and Moderate Responding Difficulty Level: Medium 6. The effects of social desirability bias parallel those for acquiescence bias and extremity bias.

Ans: T Learning Objective: 10-1: Summarize the six different response biases and how they can potentially compromise the quality of psychological measurement.

Cognitive Domain: Analysis Answer Location: Social Desirability (“Faking Good”) Difficulty Level: Medium 7. Research results can be compromised by social desirability bias.

Ans: T Learning Objective: 10-1: Summarize the six different response biases and how they can potentially compromise the quality of psychological measurement.

Cognitive Domain: Comprehension Answer Location: Social Desirability (“Faking Good”) Difficulty Level: Medium 8. Social desirability bias can artificially inflate the correlations among measures and cause researchers to draw inaccurate psychological conclusions.

Ans: T

Instructor Resource Furr, Psychometrics, 4e SAGE Publishing, 2022 Learning Objective: 10-1: Summarize the six different response biases and how they can potentially compromise the quality of psychological measurement.

Cognitive Domain: Comprehension Answer Location: Social Desirability (“Faking Good”) Difficulty Level: Medium 9. Anonymity decreases honest responding and random responding.

Ans: F Learning Objective: 10-2: Discuss the ways psychologists attempt to minimize the existence and effects of bias.

Cognitive Domain: Analysis Answer Location: Minimizing the Existence of Bias by Managing the Testing Context Difficulty Level: Medium 10. Forced-choice formats prevent respondents from simply picking the more desirable choice.

Ans: T Learning Objective: 10-2: Discuss the ways psychologists attempt to minimize the existence and effects of bias.

Cognitive Domain: Analysis Answer Location: Minimizing the Existence of Bias by Managing Test Content Difficulty Level: Medium Short Answer 1. What is a potential threat to the psychometric quality of the psychological measurement that acquiescent responders may present?

Ans: Answer may vary.

Learning Objective: 10-1: Summarize the six different response biases and how they can potentially compromise the quality of psychological measurement.

Cognitive Domain: Analysis Answer Location: Acquiescence Bias (“Yea-Saying and Nay-Saying”) Difficulty Level: Medium 2. Social desirability bias can be affected by at least three sources. What is one of them?

Ans: Answer may vary.

Learning Objective: 10-1: Summarize the six different response biases and how they can potentially compromise the quality of psychological measurement.

Cognitive Domain: Comprehension Answer Location: Social Desirability (“Faking Good”) Difficulty Level: Medium 3. Which response bias involves responding in a way that exaggerates one’s problems, shortcomings, or disabilities?

Instructor Resource Furr, Psychometrics, 4e SAGE Publishing, 2022 Ans: Answer may vary.

Learning Objective: 10-1: Summarize the six different response biases and how they can potentially compromise the quality of psychological measurement.

Cognitive Domain: Knowledge Answer Location: Malingering (“Faking Bad”) Difficulty Level: Easy 4. What bias is effectively opposite of social desirability?

Ans: Answer may vary.

Learning Objective: 10-1: Summarize the six different response biases and how they can potentially compromise the quality of psychological measurement.

Cognitive Domain: Knowledge Answer Location: Malingering (“Faking Bad”) Difficulty Level: Easy 5. What is a condition, according to the text, that might reduce respondents’ motivation to be careful and thoughtful when creating responses? Said another way, what could cause careless or random responding?

Ans: Answer may vary.

Learning Objective: 10-1: Summarize the six different response biases and how they can potentially compromise the quality of psychological measurement.

Cognitive Domain: Comprehension Answer Location: Careless or Random Responding Difficulty Level: Medium 6. What is the best way to cope with response bias, according to the text?

Ans: Answer may vary.

Learning Objective: 10-2: Discuss the ways psychologists attempt to minimize the existence and effects of bias.

Cognitive Domain: Comprehension Answer Location: Minimizing the Existence of Bias by Managing the Testing Context Difficulty Level: Medium 7. Consider two items (below) that you might find on a measure of loneliness, where higher scores represent greater loneliness. Which is a negatively keyed item below, A or B? a. I am by myself more often than I like b. I often spend time with other people Ans: Answer may vary.

Learning Objective: 10-2: Discuss the ways psychologists attempt to minimize the existence and effects of bias.

Cognitive Domain: Analysis Answer Location: Minimizing the Effects of Bias by Managing Test Content or Scoring Difficulty Level: Medium

Instructor Resource Furr, Psychometrics, 4e SAGE Publishing, 2022 8. Let’s say you are a clinical psychologist and you administer the MMPI to a client.

When you get the client’s scores, you see that he has a high score on the F scale.

Which of the following would be a reasonable option to consider, A or B? a. Ignore or pay little attention to the client’s other scores on the MMPI. b. Give extra weight to the meaning of the client’s other scores on the MMPI. Ans: Answer may vary.

Learning Objective: 10-3: Describe how psychologists manage test content and use specialized tests to detect bias and intervene.

Cognitive Domain: Analysis Answer Location: Managing Test Content to Detect Bias and Intervene Difficulty Level: Medium 9. A researcher is interested in studying the link between charisma and intelligence— hypothesizing that relatively charismatic people are also relatively intelligent. He recruits a sample of people, and he administers two tests. One is a self-report questionnaire measuring charisma (e.g., “People tend to find me charming”). The other is a standard test of intelligence, in which participants are presented with challenging right/wrong questions. Let’s say that he gathers the data and finds a positive correlation between the measures. Which of the following, A or B, might be a legitimate statement about this finding, as related to social desirability? a. The correlation between tests might not truly reflect a link between charisma and intelligence. Both measures are likely to be affected by impression management or self-deception. This could create a correlation between the measures even if the constructs (charisma and intelligence) are NOT related to each other. b. The correlation between tests is probably not simply a product of social desirability.

Although self-reports of Charisma might be affected by motivations toward impression management or self-deception, one’s answers on a standard intelligence test aren’t likely to be affected by such motivations (i.e., one can’t easily exaggerate one’s score on the IQ test to impress others). Since social desirability doesn’t affect both tests, it can’t produce a correlation between the tests.

Ans: Answer may vary.

Learning Objective: 10-3: Describe how psychologists manage test content and use specialized tests to detect bias and intervene.

Cognitive Domain: Application Answer Location: Using Specialized Tests to Detect Bias and Intervene Difficulty Level: Hard 10. What is one way to identify participants who seem to be manifesting some form of response bias?

Ans: Answer may vary.

Learning Objective: 10-3: Describe how psychologists manage test content and use specialized tests to detect bias and intervene.

Cognitive Domain: Comprehension Answer Location: Managing Test Content to Detect Bias and Intervene Difficulty Level: Medium

Instructor Resource Furr, Psychometrics, 4e SAGE Publishing, 2022

Instructor Resource Furr, Psychometrics, 4e SAGE Publishing, 2022 Chapter 11: Test Bias Test Bank Multiple Choice 1. A researcher wants to test whether there are sex differences in math ability. He gives his students a multiplication test and finds that females score lower, on average, than males. In terms of construct bias, why might he need to be very cautious before interpreting this finding as evidence that females truly have lower “multiplication ability” (on average) than males? a. The males and females in his class might not accurately represent the “population” of males and females. b. The test scores might not reflect multiplication ability equally well in the two groups (e.g., the test scores might not be a good measure of multiplication ability for most/many females). c. The males in his class might have had more lucky guesses than females, thus earning higher scores. d. Math ability is not an important construct, so bias is inherent in its examination.

Ans: B Learning Objective: 11-1: Describe the procedures used to estimate the existence and degree of construct bias.

Cognitive Domain: Analysis Answer Location: Detecting Construct Bias: Internal Evaluation of a Test Difficulty Level: Medium 2. Suppose we are cross-cultural psychologists, and we study cultural differences in negative emotionality (NE, the tendency to experience negative emotions). We administer a three-item test of NE to a group of Canadians and a group of Russians.

Because we’re aware of the potential for construct bias, we conduct a factor analysis of the NE items separately for each group. Within each group, we seem to have a onefactor structure, with the factor loadings in the table below. Based on these results, which of the following can we not conclude about the NE test?

Item Anger Sadness Guilt

Factor Loadings Canadians Russians .80 .00 .00 .80 .20 .20

a. There is no concern about construct bias, according to these results. b. Among Canadians, NE test scores generally reflect differences in Anger. c. Among Russians, NE test scores generally reflect differences in Sadness. d. If we compare the NE test scores across cultures, we’re mainly comparing Anger (for Canadians) to Sadness (for Russians).

Instructor Resource Furr, Psychometrics, 4e SAGE Publishing, 2022 Ans: A Learning Objective: 11-1: Describe the procedures used to estimate the existence and degree of construct bias.

Cognitive Domain: Application Answer Location: Detecting Construct Bias: Internal Evaluation of a Test Difficulty Level: Hard 3. Which of the following would indicate the presence of construct bias? a. group differences in a test’s means b. group differences in a test’s factorial structure c. group differences in the correlation between test scores and a key criterion variable d. group differences in admission/hiring rates, where admission/hiring is at least partially based upon test scores Ans: B Learning Objective: 11-1: Describe the procedures used to estimate the existence and degree of construct bias.

Cognitive Domain: Analysis Answer Location: Detecting Construct Bias: Internal Evaluation of a Test Difficulty Level: Medium 4. Which approach to evaluating construct bias would suggest if a test is not quite "working" in the same way in two different groups? a. item discrimination index b. rank order c. reliability d. factor analysis Ans: C Learning Objective: 11-1: Describe the procedures used to estimate the existence and degree of construct bias.

Cognitive Domain: Analysis Answer Location: Reliability Difficulty Level: Medium 5. Which approach to evaluating construct bias might involve a 10-item math test that can be evaluated in order of difficulty? a. item discrimination index b. rank order c. reliability d. differential item functioning analysis Ans: B Learning Objective: 11-1: Describe the procedures used to estimate the existence and degree of construct bias.

Cognitive Domain: Analysis Answer Location: Rank Order Difficulty Level: Medium

Instructor Resource Furr, Psychometrics, 4e SAGE Publishing, 2022 6. A test developer creates a new test to assess academic aptitude. She examines item discrimination values and finds that the test items have low discrimination values among males and high discrimination values among females. What does this mean? a. The items are less biased among males than among females. b. The test’s items do not differentiate well between males who have higher and lower aptitude, but they do differentiate well between females who have higher and lower aptitude. c. The test likely suffers from predictive bias. d. The test’s items reflect the construct equally well for both genders.

Ans: B Learning Objective: 11-1: Describe the procedures used to estimate the existence and degree of construct bias.

Cognitive Domain: Analysis Answer Location: Item Discrimination Index Difficulty Level: Medium 7. Which approach to evaluating construct bias reflects the degree to which an item is related to the total test score? a. item discrimination index b. rank order c. reliability d. factor analysis Ans: A Learning Objective: 11-1: Describe the procedures used to estimate the existence and degree of construct bias.

Cognitive Domain: Comprehension Answer Location: Item Discrimination Index Difficulty Level: Medium 8. Which approach to evaluating construct bias was developed in association with classical test theory and is particularly useful for binary items? a. item discrimination index b. differential item functioning analysis c. reliability d. factor analysis Ans: A Learning Objective: 11-1: Describe the procedures used to estimate the existence and degree of construct bias.

Cognitive Domain: Comprehension Answer Location: Item Discrimination Index Difficulty Level: Medium 9. Which approach to evaluating construct bias is an important tool for evaluating the internal structure or dimensionality of a test? a. item discrimination index b. rank order

Instructor Resource Furr, Psychometrics, 4e SAGE Publishing, 2022 c. reliability d. factor analysis Ans: D Learning Objective: 11-1: Describe the procedures used to estimate the existence and degree of construct bias.

Cognitive Domain: Comprehension Answer Location: Factor Analysis Difficulty Level: Medium 10. Which approach to evaluating construct bias is a statistical procedure for partitioning the variance or covariance among test items into clusters that in some sense "hang together"? a. item discrimination index b. rank order c. differential item functioning analysis d. factor analysis Ans: D Learning Objective: 11-1: Describe the procedures used to estimate the existence and degree of construct bias.

Cognitive Domain: Comprehension Answer Location: Factor Analysis Difficulty Level: Medium 11. If an item’s ____________________ is not invariant across group 1 and group 2, then the psychological difference between two people in group 1 who score 1 point different on the item is NOT the same as the psychological difference between two people in group 2 who score 1 point different on the item. For example, imagine that Allison and Barbara are members of group 1, with Allison scoring 1 point higher than Barbara on a particular test item. Now imagine that Carl and Dave are members of group 2, with Carl scoring 1 point higher than Dave on the same particular test item. If the item’s ____________________ is not invariant across the groups, then the psychological difference between Allison and Barbara is NOT the same as the psychological difference between Carl and Dave, even though the observed/test difference is identical. (Note: both blanks are the same answer.) a. intercept b. mean c. factor loading d. fairness Ans: C Learning Objective: 11-1: Describe the procedures used to estimate the existence and degree of construct bias.

Cognitive Domain: Application Answer Location: Factor Analysis Difficulty Level: Hard

Instructor Resource Furr, Psychometrics, 4e SAGE Publishing, 2022 12. Say we are examining a new personality scale to assess “feelings of loneliness.” We conduct a factor analysis of the scale separately for males and females, and we find the same factor structure in the two groups. This would be evidence of: a. convergent validity b. discriminant validity c. a lack of construct bias d. the presence of construct bias Ans: C Learning Objective: 11-1: Describe the procedures used to estimate the existence and degree of construct bias.

Cognitive Domain: Comprehension Answer Location: Factor Analysis Difficulty Level: Medium 13. A test developer develops a new ability test and estimates the reliability of the new test’s scores separately for males and females. She finds that the test is much more reliable among males than among females. This indicates that: a. females are more likely to get an artificially high score on the test than males b. the test is pretty good at telling us which males have a better or worse ability, but it is not very good at telling us which females have better or worse ability c. the test is pretty good at telling us which females have a better or worse ability, but it is not very good at telling us which males have a better or worse ability d. the test is not good at telling us if either females or males have a better or worse ability Ans: B Learning Objective: 11-1: Describe the procedures used to estimate the existence and degree of construct bias.

Cognitive Domain: Analysis Answer Location: Factor Analysis Difficulty Level: Medium 14. Which approach to evaluating construct bias is a feature of a psychometric approach called item response theory? a. item discrimination index b. rank order c. differential item functioning analysis d. factor analysis Ans: C Learning Objective: 11-1: Describe the procedures used to estimate the existence and degree of construct bias.

Cognitive Domain: Knowledge Answer Location: Differential Item Functioning Analyses Difficulty Level: Easy

Instructor Resource Furr, Psychometrics, 4e SAGE Publishing, 2022 15. One way that test developers try to detect and understand construct bias is through a procedure called Item Response Theory (IRT). In Item Response Theory, an “Item Characteristic Curve” reflects: a. The connection between a) trait levels, and b) the likelihood of making a correct response to a test item b. The connection between a) trait levels, and b) the likelihood of passing a test c. The connection between a) the likelihood of making a correct response to a test item, and b) the likelihood of passing the whole test d. The connection between a) trait levels, b) the likelihood of making a correct response to a test item, and c) the likelihood of passing the whole test Ans: A Learning Objective: 11-1: Describe the procedures used to estimate the existence and degree of construct bias.

Cognitive Domain: Comprehension Answer Location: Differential Item Functioning Analyses Difficulty Level: Medium 16. One way that test developers try to detect and understand construct bias is through a procedure called Item Response Theory (IRT). Say that a test developer is evaluating a new measure of mechanical aptitude, gathers test data from a sample of males and a sample of females, conducts an IRT-based analysis of the data, and obtains the two Item Characteristics Curves below for one of the items on the test. Which of the following statements is a correct interpretation of this result?

a. This suggests the presence of bias in the item—even if a female and male have *the same exact level* of mechanical aptitude (say 1 SD below the mean), the female has a lower chance of answering the item correctly. b. This suggests the presence of bias in the item—males on average have a lower level of mechanical aptitude.

Instructor Resource Furr, Psychometrics, 4e SAGE Publishing, 2022 c. This suggests the absence of bias in the item—the pattern of connection between item responses and trait levels is the same for males and females. d. This suggests the absence of bias in the item—females on average have a higher level of mechanical aptitude.

Ans: A Learning Objective: 11-1: Describe the procedures used to estimate the existence and degree of construct bias.

Cognitive Domain: Application Answer Location: Differential Item Functioning Analyses Difficulty Level: Hard 17. Detection of ___________ begins with the assumption that "one size fits all." a. criterion score bias b. predictive bias c. slope bias d. intercept bias Ans: B Learning Objective: 11-2: Explain the concerns of predictive bias and how researchers detect prediction bias.

Cognitive Domain: Knowledge Answer Location: Detecting Predictive Bias: External Evaluation of a Test Difficulty Level: Easy 18. Suppose a teacher creates a multiplication test, and its items involve questions related to American football (e.g., “how many yards does a team need to gain in order to get three first-downs?”). So, to do well on the test, respondents must be familiar with football and have an understanding of multiplication. This could raise concerns about construct bias because: a. Among people who are familiar with football, a poor test score indicates a “lack of understanding of multiplication,” however, among people who are NOT familiar with football, the same poor test score does not necessarily indicate “lack of understanding of multiplication.” b. Among people who are familiar with football, those who do well on the test actually “lack an understanding of multiplication” whereas those who do poorly actually have a good “understanding of multiplication.” c. Among people who are not familiar with football, those who do well on the test truly do have a good “understanding of multiplication,” whereas those who do poorly truly have a poor “understanding of multiplication.” d. Among people who are not familiar with football, a poor test score indicates a good "understanding of multiplication," whereas those who do well on a test have a poor "understanding of multiplication."

Ans: A Learning Objective: 11-2: Explain the concerns of predictive bias and how researchers detect prediction bias.

Cognitive Domain: Analysis

Instructor Resource Furr, Psychometrics, 4e SAGE Publishing, 2022 Answer Location: Detecting Predictive Bias: External Evaluation of a Test Difficulty Level: Medium 19. Which of the following is the best description of predictive bias? a. When creating and revising tests, test developers employ multiple strategies to predict which items will show group-based biases and which will not. If the test developers identify items that they believe will probably suffer from bias, then predictive bias occurs. b. Test scores are used to represent or predict actual levels of the characteristic being measured by the test (e.g., scores on the WAIS intelligence test are used to represent or predict actual levels of intelligence). If a test’s scores are not related to “true levels” of the underlying characteristic in the same way for different groups, then the test suffers from predictive bias. c. Decision-makers use test-based information to help choose which people to admit, select, hire, promote, etc. This practice is based on the idea that test scores can help the decision-makers predict which people will perform well. Predictive bias concerns the degree to which test scores are related similarly to performance for different groups of test-takers. d. Test developers avoid writing items with double negatives because such items would prevent us from detecting predictive bias and could prevent us from intervening to deal with any biases that may have occurred.

Ans: C Learning Objective: 11-2: Explain the concerns of predictive bias and how researchers detect prediction bias.

Cognitive Domain: Comprehension Answer Location: Detecting Predictive Bias: External Evaluation of a Test Difficulty Level: Medium 20. Imagine that we are hiring for a sales department of a large corporation. We believe that the personality trait “Extraversion” is relevant to performance as a salesperson, so we want to use applicants’ extraversion test scores to help us decide who to hire (preferring people who score higher on extraversion). Say we gather data in which we measure the extraversion levels of salespeople in the corporation and we examine extraversion’s link to “number of dollars in sales.” We run a regression analysis in which we use Extraversion to predict the “number of dollars in sales.” We get the following results: Sales = 10,000 + 5,000 (Extraversion) Based on these results (and assuming that the intercept and slope are statistically significant), which of the following statements is valid? a. A salesperson with an average level of extraversion is expected to have $10,000 in sales. b. A salesperson with a score of 0 on the extraversion test is expected to have $10,000 in sales.

Instructor Resource Furr, Psychometrics, 4e SAGE Publishing, 2022 c. A salesperson with an average level of extraversion is expected to have $5,000 in sales. d. A salesperson with a score of 0 on the extraversion test is expected to have $5,000 in sales.

Ans: B Learning Objective: 11-2: Explain the concerns of predictive bias and how researchers detect prediction bias.

Cognitive Domain: Application Answer Location: Intercept Bias Difficulty Level: Hard 21. Let’s imagine now that we run the regression analysis separately for female and male salespeople in the sample. Say we get the following results: Females: Sales = 15,000 + 5,000 (Extraversion) Males: Sales = 5,000 + 5,000 (Extraversion) Which of the following is an implication of this result? a. A female with an extraversion score of 20 is predicted to have $115,000 in sales, while a male with the same score is predicted to have only $105,000 in sales b. A one-unit increase in extraversion is associated with a larger increase in sales for females than for males. c. For every 5,000-point increase in extraversion test scores, sales increases by one dollar, and this is true for both females and males. d. “Slope bias” exists.

Ans: A Learning Objective: 11-2: Explain the concerns of predictive bias and how researchers detect prediction bias.

Cognitive Domain: Application Answer Location: Intercept Bias Difficulty Level: Hard 22. Say we ran the regression analysis for female and male salespeople using both extraversion and sex as predictors of sales in the entire sample. In the sample, we found the following results: Females: Sales = 10,000 + 2,000 (Extraversion) Males: Sales = 10,000 + 8,000 (Extraversion) Which of the following is an implication of this result? a. There is no predictive bias, as the sales for the average male and the average female are identical at $10,000 b. The difference (in sales) between females and males is not the same at all levels of extraversion (e.g., the sex difference in sales is relatively small at low levels of extraversion, but relatively large and higher levels of extraversion) c. Extraversion is more strongly associated with sales among females than among males d. The average male has $6,000 more in sales than does the average female.

Ans: B

Instructor Resource Furr, Psychometrics, 4e SAGE Publishing, 2022 Learning Objective: 11-2: Explain the concerns of predictive bias and how researchers detect prediction bias.

Cognitive Domain: Application Answer Location: Intercept Bias Difficulty Level: Hard 23. If we do a regression analysis and find ________________, then we would conclude that two people (from different groups) who have the same actual trait/ability obtain different scores on the measure of that trait/ability a. intercept bias b. construct bias c. regression bias d. criterion bias Ans: A Learning Objective: 11-2: Explain the concerns of predictive bias and how researchers detect prediction bias.

Cognitive Domain: Comprehension Answer Location: Intercept Bias Difficulty Level: Medium 24. A researcher studies potential test bias in a new test designed to measure mechanical aptitude (in terms of gender bias). She administers the test to a large sample of males and females who were hired at an automobile assembly plant. Later, she collects information about their performance in a variety of mechanical tasks to obtain a criterion variable of “mechanical performance”. She conducts a regression analysis to evaluate the degree to which test scores are more predictive of performance for males than for females (or vice versa). Here are the results of her analysis: Females: Males:

Performance = 43.56 + 4.56 (Test score) Performance = 39.32 + 4.56 (Test score)

Which of the following is not a legitimate conclusion based on these results? a. the degree of bias changes across scores on the test b. a male with a “0” on the test is predicted to achieve a score of 39.32 on the performance variable c. for a male and female who have the same test score, the male is predicted to have a higher score on the performance variable. d. among females, a one-unit difference in test scores is associated with a 4.56-point difference in the performance variable.

Ans: A Learning Objective: 11-2: Explain the concerns of predictive bias and how researchers detect prediction bias.

Cognitive Domain: Analysis Answer Location: Intercept Bias Difficulty Level: Medium

Instructor Resource Furr, Psychometrics, 4e SAGE Publishing, 2022 25. _____________ can occur when there is differential validity and is more accurate for one group than for another. a. criterion score bias b. predictive bias c. slope bias d. intercept bias Ans: C Learning Objective: 11-2: Explain the concerns of predictive bias and how researchers detect prediction bias.

Cognitive Domain: Knowledge Answer Location: Slope Bias Difficulty Level: Easy True/False 1. For composite tests, the overall degree of test score bias is determined by the bias associated with each of the items in the test.

Ans: T Learning Objective: 11-1: Describe the procedures used to estimate the existence and degree of construct bias.

Cognitive Domain: Comprehension Answer Location: Detecting Construct Bias: Internal Evaluation of a Test Difficulty Level: Medium 2. If two discrimination index values are approximately equal, this would indicate the item reflects the construct in a different way for each item, and we can conclude that it is not biased.

Ans: F Learning Objective: 11-1: Describe the procedures used to estimate the existence and degree of construct bias.

Cognitive Domain: Analysis Answer Location: Item Discrimination Index Difficulty Level: Medium 3. Factor analysis can be used to evaluate the internal structure of a test separately for two groups of people.

Ans: T Learning Objective: 11-1: Describe the procedures used to estimate the existence and degree of construct bias.

Cognitive Domain: Comprehension Answer Location: Factor Analysis Difficulty Level: Medium 4. IRT analyses are simple and straightforward, therefore have little to no downside.

Ans: F

Instructor Resource Furr, Psychometrics, 4e SAGE Publishing, 2022 Learning Objective: 11-1: Describe the procedures used to estimate the existence and degree of construct bias.

Cognitive Domain: Knowledge Answer Location: Differential Item Functioning Analyses Difficulty Level: Easy 5. Predictive bias is of great concern in important areas of applied psychological and educational measurement.

Ans: T Learning Objective: 11-2: Explain the concerns of predictive bias and how researchers detect prediction bias.

Cognitive Domain: Comprehension Answer Location: Detecting Predictive Bias: External Evaluation of a Test Difficulty Level: Medium 6. Say we ran a multiple regression analyses, using both extraversion and sex as predictors of sales in the entire sample. Say the “sex” variable is coded as 0 = Male, 1 = Female: Sales = 5,000 + 5,000 (Extraversion) + 10,000 (Sex).

For a male and female who have the same extraversion test score, the female is likely to have $10,000 more in sales.

Ans: T Learning Objective: 11-2: Explain the concerns of predictive bias and how researchers detect prediction bias.

Cognitive Domain: Analysis Answer Location: Intercept Bias Difficulty Level: Medium 7. Test fairness is the degree to which a test’s scores are equally valid (in representing a particular construct) for two groups of respondents.

Ans: F Learning Objective: 11-3: Distinguish test fairness from test score bias and why test fairness is an important issue in psychological testing.

Cognitive Domain: Knowledge Answer Location: Test Fairness Difficulty Level: Easy 8. Test bias is how a test is used or misused in terms of a broad set of social/economic/etc. considerations.

Ans: F Learning Objective: 11-3: Distinguish test fairness from test score bias and why test fairness is an important issue in psychological testing.

Cognitive Domain: Knowledge Answer Location: Test Fairness Difficulty Level: Easy

Instructor Resource Furr, Psychometrics, 4e SAGE Publishing, 2022 9. Test fairness refers to a psychometric property of a test.

Ans: F Learning Objective: 11-3: Distinguish test fairness from test score bias and why test fairness is an important issue in psychological testing.

Cognitive Domain: Knowledge Answer Location: Test Fairness Difficulty Level: Easy 10. UC’s analysis of the SAT provided a real-life examination of test bias.

Ans: T Learning Objective: 11-3: Distinguish test fairness from test score bias and why test fairness is an important issue in psychological testing.

Cognitive Domain: Comprehension Answer Location: Example: Is the SAT Biased in Terms of Race or Socioeconomic Status?

Difficulty Level: Medium Short Answer 1. If a test suffers from “construct bias,” then its (choose A or B below): a. test scores are not related to a criterion variable in the same way for people in two different groups b. test scores do not mean the same thing, psychologically speaking, for people in two different groups Ans: Answer may vary.

Learning Objective: 11-1: Describe the procedures used to estimate the existence and degree of construct bias.

Cognitive Domain: Analysis Answer Location: Detecting Construct Bias: Internal Evaluation of a Test Difficulty Level: Medium 2. A test developer creates a new test to assess academic aptitude. Imagine that males tend to score lower on this test, on average, than do females. Which of the following statements is true, psychometrically speaking: A or B? a. This difference in group averages is evidence of test bias against males. b. This difference in group averages might (or might not) be due to test bias against males.

Ans: Answer may vary.

Learning Objective: 11-1: Describe the procedures used to estimate the existence and degree of construct bias.

Cognitive Domain: Analysis Answer Location: Detecting Construct Bias: Internal Evaluation of a Test Difficulty Level: Medium

Instructor Resource Furr, Psychometrics, 4e SAGE Publishing, 2022 3. Construct bias is often evaluated by examining responses to individual items on a test. Therefore, when would a test item be considered biased? Name one of the two qualifications.

Ans: Answer may vary.

Learning Objective: 11-1: Describe the procedures used to estimate the existence and degree of construct bias.

Cognitive Domain: Comprehension Answer Location: Detecting Construct Bias: Internal Evaluation of a Test Difficulty Level: Medium 4. What is one of the methods for detecting construct bias?

Ans: Answer may vary.

Learning Objective: 11-1: Describe the procedures used to estimate the existence and degree of construct bias.

Cognitive Domain: Comprehension Answer Location: Detecting Construct Bias: Internal Evaluation of a Test Difficulty Level: Medium 5. Which approach to evaluating construct bias is independent of the number of people in the groups that are being compared who answer an item correctly?

Ans: Answer may vary.

Learning Objective: 11-1: Describe the procedures used to estimate the existence and degree of construct bias.

Cognitive Domain: Comprehension Answer Location: Item Discrimination Index Difficulty Level: Medium 6. “Measurement invariance” is most associated with what approach to evaluating construct bias?

Ans: Answer may vary.

Learning Objective: 11-1: Describe the procedures used to estimate the existence and degree of construct bias.

Cognitive Domain: Comprehension Answer Location: Factor Analysis Difficulty Level: Medium 7. A procedure called Confirmatory Factor Analysis (CFA) is often used to evaluate construct bias. From the perspective of CFA, this issue is framed in terms of “measurement invariance.” Measurement invariance is (A or B): a. good, because it means that the factorial structure (dimensionality) of a test is the same across groups, suggesting no bias b. bad, because it means that the factorial structure differs across groups, suggesting bias Ans: Answer may vary.

Learning Objective: 11-1: Describe the procedures used to estimate the existence and degree of construct bias.

Instructor Resource Furr, Psychometrics, 4e SAGE Publishing, 2022 Cognitive Domain: Analysis Answer Location: Factor Analysis Difficulty Level: Medium 8. Which approach to evaluating construct bias is a feature of a psychometric approach called item response theory?

Ans: Answer may vary.

Learning Objective: 11-1: Describe the procedures used to estimate the existence and degree of construct bias.

Cognitive Domain: Comprehension Answer Location: Differential Item Functioning Analyses Difficulty Level: Medium 9. Is it uniform or non-uniform bias in a situation where ICCs differ in shape as well as location?

Ans: Answer may vary.

Learning Objective: 11-1: Describe the procedures used to estimate the existence and degree of construct bias.

Cognitive Domain: Comprehension Answer Location: Differential Item Functioning Analyses Difficulty Level: Medium 10. If a test suffers from “predictive bias,” then its (A or B): a. test scores are not related to a criterion variable in the same way for people in two different groups b. test scores do not mean the same thing, psychologically speaking, for people in two different group Ans: Answer may vary.

Learning Objective: 11-2: Explain the concerns of predictive bias and how researchers detect prediction bias.

Cognitive Domain: Analysis Answer Location: Detecting Predictive Bias: External Evaluation of a Test Difficulty Level: Medium

Instructor Resource Furr, Psychometrics, 4e SAGE Publishing, 2022 Chapter 12: Confirmatory Factor Analysis Test Bank Multiple Choice 1. When is EFA most appropriate in test use? a. early phases b. mid-phase c. later phases d. anytime; EFA is not more appropriate at one time or another Ans: A Learning Objective: 12-1: Compare and contrast exploratory factor analysis and confirmatory factor analysis.

Cognitive Domain: Analysis Answer Location: The Frequency and Roles of EFA and CFA Difficulty Level: Medium 2. When is CFA most appropriate in test use? a. early phases b. mid-phase c. later phases d. anytime; EFA is not more appropriate at one time or another Ans: C Learning Objective: 12-1: Compare and contrast exploratory factor analysis and confirmatory factor analysis.

Cognitive Domain: Analysis Answer Location: The Frequency and Roles of EFA and CFA Difficulty Level: Medium 3. What kind of factor analysis is most appropriate for use when test developers have gone through initial evaluations of item properties and dimensionality and after significant revisions of test content? a. CFA b. EFA c. a hybrid of CFA and EFA d. either factor analysis would be equally appropriate Ans: A Learning Objective: 12-1: Compare and contrast exploratory factor analysis and confirmatory factor analysis.

Cognitive Domain: Analysis Answer Location: The Frequency and Roles of EFA and CFA Difficulty Level: Medium

Instructor Resource Furr, Psychometrics, 4e SAGE Publishing, 2022 4. What kind of factor analysis is most appropriate for use when test developers are clarifying their understanding of the constructs and of the test itself? a. CFA b. EFA c. a hybrid of CFA and EFA d. either factor analysis would be equally appropriate Ans: B Learning Objective: 12-1: Compare and contrast exploratory factor analysis and confirmatory factor analysis.

Cognitive Domain: Analysis Answer Location: The Frequency and Roles of EFA and CFA Difficulty Level: Medium 5. Which factor analysis allows test developers and test evaluators to understand the degree to which their hypothesized measurement models are consistent with actual data produced by respondents? a. CFA b. EFA c. a hybrid of CFA and EFA d. either factor analysis would be equally appropriate Ans: A Learning Objective: 12-1: Compare and contrast exploratory factor analysis and confirmatory factor analysis.

Cognitive Domain: Comprehension Answer Location: Using CFA to Evaluate Measurement Models Difficulty Level: Medium 6. Which of the following is not part of the authenticity scale? a. self-alienation b. authentic living c. accepting external influence d. dynamic self-exploration Ans: D Learning Objective: 12-2: Describe the process of conducting confirmatory factor analysis and how to interpret its results.

Cognitive Domain: Comprehension Answer Location: Overview of CFA and an Example Difficulty Level: Medium 7. What is the definition of self-alienation? a. the degree to which a person really understands himself or herself b. the degree to which a person behaves and expresses emotion in a way that is an honest reflection of his or her self-perception c. the degree to which a person understands that other people can influence one’s life and conforms to these influences

Instructor Resource Furr, Psychometrics, 4e SAGE Publishing, 2022 d. the degree to which a person continually seeks new experiences as a way toward inner exploration Ans: A Learning Objective: 12-2: Describe the process of conducting confirmatory factor analysis and how to interpret its results.

Cognitive Domain: Knowledge Answer Location: Overview of CFA and an Example Difficulty Level: Easy 8. What is the definition of authentic living? a. the degree to which a person really understands himself or herself b. the degree to which a person behaves and expresses emotion in a way that is an honest reflection of his or her self-perception c. the degree to which a person understands that other people can influence one’s life and conforms to these influences d. the degree to which a person continually seeks new experiences as a way toward inner exploration Ans: B Learning Objective: 12-2: Describe the process of conducting confirmatory factor analysis and how to interpret its results.

Cognitive Domain: Knowledge Answer Location: Overview of CFA and an Example Difficulty Level: Easy 9. How is ‘accepting external influence’ best defined? a. the degree to which a person really understands himself or herself b. the degree to which a person behaves and expresses emotion in a way that is an honest reflection of his or her self-perception c. the degree to which a person understands that other people can influence one’s life and conforms to these influences d. the degree to which a person continually seeks new experiences as a way toward inner exploration Ans: C Learning Objective: 12-2: Describe the process of conducting confirmatory factor analysis and how to interpret its results.

Cognitive Domain: Knowledge Answer Location: Overview of CFA and an Example Difficulty Level: Easy 10. Rahul is conducting a CFA and working through the preliminary steps. What is the first step Rahul is likely to take? a. clarifying the psychological construct to be assessed and initial development of the test items b. collecting a large number of responses to his test c. reversing any negatively keyed items d. specifying the measurement model

Instructor Resource Furr, Psychometrics, 4e SAGE Publishing, 2022 Ans: A Learning Objective: 12-2: Describe the process of conducting confirmatory factor analysis and how to interpret its results.

Cognitive Domain: Analysis Answer Location: Preliminary Steps Difficulty Level: Medium 11. Which step of conducting a CFA involves translating our hypothesized measurement model into a statistical software package designed to conduct CFA? a. specification of the measurement model b. computations c. interpreting and reporting output d. model modification and reanalysis Ans: A Learning Objective: 12-2: Describe the process of conducting confirmatory factor analysis and how to interpret its results.

Cognitive Domain: Comprehension Answer Location: Step 1: Specification of the Measurement Model Difficulty Level: Medium 12. Which step of conducting a CFA involves asking our statistical software to conduct the CFA based on specifications and the data we have collected? a. specification of the measurement model b. computations c. interpreting and reporting output d. model modification and reanalysis Ans: B Learning Objective: 12-2: Describe the process of conducting confirmatory factor analysis and how to interpret its results.

Cognitive Domain: Comprehension Answer Location: Step 2: Computations Difficulty Level: Medium 13. When Asa is in the _______________ step of conducting a CFA, he examines the fit indices that address the overall adequacy of his hypothesized measurement model. a. specification of the measurement model b. computations c. interpreting and reporting output d. model modification and reanalysis Ans: C Learning Objective: 12-2: Describe the process of conducting confirmatory factor analysis and how to interpret its results.

Cognitive Domain: Analysis Answer Location: Step 3: Interpreting and Reporting Output Difficulty Level: Medium

Instructor Resource Furr, Psychometrics, 4e SAGE Publishing, 2022 14. What is the last step of conducting a CFA? a. specification of the measurement model b. computations c. interpreting and reporting output d. model modification and reanalysis Ans: D Learning Objective: 12-2: Describe the process of conducting confirmatory factor analysis and how to interpret its results.

Cognitive Domain: Comprehension Answer Location: Step 4: Model Modification and Reanalysis (If Necessary) Difficulty Level: Medium 15. Which of the following is true about a test’s internal structure that is invariant? a. it is not the same across groups b. it is not good c. it has evidence against construct bias d. its test scores likely have different interpretations Ans: C Learning Objective: 12-5: Discuss measurement invariance, its levels, and how it relates to confirmatory factor analysis.

Cognitive Domain: Comprehension Answer Location: The Meaning of Measurement Invariance Difficulty Level: Medium 16. Which of the following is true about a test’s internal structure that lacks invariance? a. it is not the same across groups b. it is good c. it has evidence against construct bias d. its test scores likely have similar interpretations Ans: A Learning Objective: 12-5: Discuss measurement invariance, its levels, and how it relates to confirmatory factor analysis.

Cognitive Domain: Comprehension Answer Location: The Meaning of Measurement Invariance Difficulty Level: Medium 17. Which of the following has some invariance (the least amount of invariance)? a. configural MI b. weak/metric MI c. strong/scalar MI d. strict MI Ans: A Learning Objective: 12-5: Discuss measurement invariance, its levels, and how it relates to confirmatory factor analysis.

Cognitive Domain: Knowledge

Instructor Resource Furr, Psychometrics, 4e SAGE Publishing, 2022 Answer Location: Levels of Invariance: Meaning and Detection Difficulty Level: Medium 18. Which of the following has invariance for all item parameters? a. configural MI b. weak/metric MI c. strong/scalar MI d. strict MI Ans: D Learning Objective: 12-5: Discuss measurement invariance, its levels, and how it relates to confirmatory factor analysis.

Cognitive Domain: Knowledge Answer Location: Levels of Invariance: Meaning and Detection Difficulty Level: Medium 19. Which of the following has much more invariance, but not all? a. configural MI b. weak/metric MI c. strong/scalar MI d. strict MI Ans: C Learning Objective: 12-5: Discuss measurement invariance, its levels, and how it relates to confirmatory factor analysis.

Cognitive Domain: Knowledge Answer Location: Levels of Invariance: Meaning and Detection Difficulty Level: Medium 20. Which of the following has somewhat more invariance (more than ‘some’ but less than ‘even more’)? a. configural MI b. weak/metric MI c. strong/scalar MI d. strict MI Ans: B Learning Objective: 12-5: Discuss measurement invariance, its levels, and how it relates Cognitive Domain: Knowledge Answer Location: Levels of Invariance: Meaning and Detection Difficulty Level: Medium 21. Which level of variance has residual variances (i.e., each item’s error variance is the same in each group)? a. configural MI b. weak/metric MI c. strong/scalar MI d. strict MI Ans: D

Instructor Resource Furr, Psychometrics, 4e SAGE Publishing, 2022 Learning Objective: 12-5: Discuss measurement invariance, its levels, and how it relates to confirmatory factor analysis.

Cognitive Domain: Comprehension Answer Location: Levels of Invariance: Meaning and Detection Difficulty Level: Medium 22. Which level of variance does not have exact factor loadings (i.e., each item’s factor loading is the same across groups)? a. configural MI b. weak/metric MI c. strong/scalar MI d. strict MI Ans: A Learning Objective: 12-5: Discuss measurement invariance, its levels, and how it relates to confirmatory factor analysis.

Cognitive Domain: Comprehension Answer Location: Levels of Invariance: Meaning and Detection Difficulty Level: Medium 23. Which level of variance does not have associations between latent variables? a. configural MI b. weak/metric MI c. strong/scalar MI d. strict MI Ans: A Learning Objective: 12-5: Discuss measurement invariance, its levels, and how it relates to confirmatory factor analysis.

Cognitive Domain: Comprehension Answer Location: Levels of Invariance: Meaning and Detection Difficulty Level: Medium 24. Which level of variance has group differences in items (i.e., item covariances, variances, and means) due to group differences on the latent variables? a. configural MI b. weak/metric MI c. strong/scalar MI d. strict MI Ans: D Learning Objective: 12-5: Discuss measurement invariance, its levels, and how it relates to confirmatory factor analysis.

Cognitive Domain: Comprehension Answer Location: Levels of Invariance: Meaning and Detection Difficulty Level: Medium 25. Which of the following implications affects each level of measurement invariance?

Instructor Resource Furr, Psychometrics, 4e SAGE Publishing, 2022 a. Items are affected by the same LVs in each group (i.e., the items reflect the same LVs) b. The groups have the same unit of measurement c. No systematic response biases across groups d. Group differences in the items (i.e., item covariances, variances, and means) are due to group differences in the latent variables Ans: A Learning Objective: 12-5: Discuss measurement invariance, its levels, and how it relates to confirmatory factor analysis.

Cognitive Domain: Analysis Answer Location: Levels of Invariance: Meaning and Detection Difficulty Level: Medium

True/False 1. EFA is most appropriate during later phases of test use.

Ans: F Learning Objective: 12-1: Compare and contrast exploratory factor analysis and confirmatory factor analysis.

Cognitive Domain: Comprehension Answer Location: The Frequency and Roles of EFA and CFA Difficulty Level: Medium 2. CFA is most appropriate during later phases of test use.

Ans: T Learning Objective: 12-1: Compare and contrast exploratory factor analysis and confirmatory factor analysis.

Cognitive Domain: Comprehension Answer Location: The Frequency and Roles of EFA and CFA Difficulty Level: Medium 3. CFA can be an iterative, back-and-forth process.

Ans: T Learning Objective: 12-2: Describe the process of conducting confirmatory factor analysis and how to interpret its results.

Cognitive Domain: Comprehension Answer Location: Overview of CFA and an Example Difficulty Level: Medium 4. The last step of CFA involves specifying the links between items and factors.

Ans: F Learning Objective: 12-2: Describe the process of conducting confirmatory factor analysis and how to interpret its results.

Instructor Resource Furr, Psychometrics, 4e SAGE Publishing, 2022 Cognitive Domain: Knowledge Answer Location: Step 1: Specifications of the Measurement Model Difficulty Level: Easy 5. In the first phase of CFA computations, the collected data are used to compute the items’ variances and the covariances among the items.

Ans: T Learning Objective: 12-2: Describe the process of conducting confirmatory factor analysis and how to interpret its results.

Cognitive Domain: Knowledge Answer Location: Step 2: Computations Difficulty Level: Easy 6. When conducting a CFA of a test, test developers and evaluators often evaluate competing measurement models.

Ans: T Learning Objective: 12-2: Describe the process of conducting confirmatory factor analysis and how to interpret its results.

Cognitive Domain: Comprehension Answer Location: Comparing Models Difficulty Level: Medium 7. The alpha method allows us to estimate reliability in a way that is more widely valid and applicable than a CFA-based approach.

Ans: F Learning Objective: 12-3: Summarize how confirmatory factor analysis can evaluate reliability.

Cognitive Domain: Comprehension Answer Location: Estimating Reliability (Omega Index) Difficulty Level: Medium 8. Omega may be valid when test items are essentially tau-equivalent, but not parallel or congeneric.

Ans: F Learning Objective: 12-3: Summarize how confirmatory factor analysis can evaluate reliability.

Cognitive Domain: Analysis Answer Location: Estimating Reliability (Omega Index) Difficulty Level: Medium 9. Multitrait-multimethod can be used with CFA to evaluate convergent and discriminant validity.

Ans: T Learning Objective: 12-4: Summarize how confirmatory factor analysis can evaluate validity.

Cognitive Domain: Comprehension

Instructor Resource Furr, Psychometrics, 4e SAGE Publishing, 2022 Answer Location: CFA and Validity Difficulty Level: Medium 10. There has been a rapidly decreasing use of CFA to examine group differences in tests’ psychometric properties.

Ans: F Learning Objective: 12-5: Discuss measurement invariance, its levels, and how it relates to confirmatory factor analysis.

Cognitive Domain: Comprehension Answer Location: CFA and Measurement Invariance Difficulty Level: Medium Short Answer 1. Is EFA considered more hypothesis-driven or exploratory?

Ans: Answer may vary.

Learning Objective: 12-1: Compare and contrast exploratory factor analysis and confirmatory factor analysis.

Cognitive Domain: Analysis Answer Location: The Frequency and Roles of EFA and CFA Difficulty Level: Medium 2. Is CFA considered more hypothesis-driven or exploratory?

Ans: Answer may vary.

Learning Objective: 12-1: Compare and contrast exploratory factor analysis and confirmatory factor analysis.

Cognitive Domain: Analysis Answer Location: The Frequency and Roles of EFA and CFA Difficulty Level: Medium 3. What technique is a blend of EFA and CFA? Ans: Answer may vary.

Learning Objective: 12-1: Compare and contrast exploratory factor analysis and confirmatory factor analysis.

Cognitive Domain: Knowledge Answer Location: The Frequency and Roles of EFA and CFA Difficulty Level: Easy 4. What is the name given to the scale which was intended to measure the degree to which a person “knows himself or herself” and “acts accordingly”?

Ans: Answer may vary.

Learning Objective: 12-2: Describe the process of conducting confirmatory factor analysis and how to interpret its results.

Cognitive Domain: Comprehension

Instructor Resource Furr, Psychometrics, 4e SAGE Publishing, 2022 Answer Location: Overview of CFA and an Example Difficulty Level: Medium 5. What is the first step needed before conducting a CFA? Ans: Answer may vary.

Learning Objective: 12-2: Describe the process of conducting confirmatory factor analysis and how to interpret its results.

Cognitive Domain: Knowledge Answer Location: Preliminary Steps Difficulty Level: Easy 6. Which step of conducting a CFA involves linking items to only one latent variable?

Ans: Answer may vary.

Learning Objective: 12-2: Describe the process of conducting confirmatory factor analysis and how to interpret its results.

Cognitive Domain: Knowledge Answer Location: Step 1: Specification of the Measurement Model Difficulty Level: Easy 7. What is one of the phases of computation?

Ans: Answer may vary.

Learning Objective: 12-2: Describe the process of conducting confirmatory factor analysis and how to interpret its results.

Cognitive Domain: Comprehension Answer Location: Step 2: Computations Difficulty Level: Medium 8. What is one of the (at least) two important uses of CFA when evaluating reliability?

Ans: Answer may vary.

Learning Objective: 12-3: Summarize how confirmatory factor analysis can evaluate reliability.

Cognitive Domain: Comprehension Answer Location: CFA and Reliability Difficulty Level: Medium 9. What is one of the steps that can help test users understand which (if any) specific CTT-based measurement model fits a test’s responses?

Ans: Answer may vary.

Learning Objective: 12-3: Summarize how confirmatory factor analysis can evaluate reliability.

Cognitive Domain: Knowledge Answer Location: Evaluating Types of CTT Measurement Models Difficulty Level: Easy 10. What is one way CFA can be used to examine convergent validity?

Ans: Answer may vary.

Instructor Resource Furr, Psychometrics, 4e SAGE Publishing, 2022 Learning Objective: 12-4: Summarize how confirmatory factor analysis can evaluate validity.

Cognitive Domain: Comprehension Answer Location: CFA and Validity Difficulty Level: Medium

Instructor Resource Furr, Psychometrics, 4e SAGE Publishing, 2022 Chapter 13: Generalizability Theory Test Bank Multiple Choice 1. Which of the following is true about CTT? a. error can be differentiated b. can tease apart the effects that multiple facets of a measurement strategy might have on scores c. applicable to simple measurement designs d. has one or more facets Ans: C Learning Objective: 13-1: Describe how different measurement strategies affect scores in generalizability theory.

Cognitive Domain: Analysis Answer Location: Multiple Facets of Measurement Difficulty Level: Medium 2. Which of the following is true about G theory? a. error can be differentiated b. cannot tease apart the effects that multiple facets of a measurement strategy might have on scores c. applicable to simple measurement designs only d. has only one facet Ans: A Learning Objective: 13-1: Describe how different measurement strategies affect scores in generalizability theory.

Cognitive Domain: Analysis Answer Location: Multiple Facets of Measurement Difficulty Level: Medium 3. In terms of generalizability, _________ is the whole list of items reflecting a certain factor, while a sample is a few items that are representative of that list. a. variance b. universe c. population d. design Ans: B Learning Objective: 13-2: Discuss how generalizability theory relates to generalizability, universes, and variance components.

Cognitive Domain: Knowledge Answer Location: Generalizability, Universes, and Variance Components Difficulty Level: Easy

Instructor Resource Furr, Psychometrics, 4e SAGE Publishing, 2022 4. G theory is an extension of __________. a. CTT b. domain-sampling theory c. item response theory d. psychometric Ans: B Learning Objective: 13-2: Discuss how generalizability theory relates to generalizability, universes, and variance components.

Cognitive Domain: Comprehension Answer Location: Generalizability, Universes, and Variance Components Difficulty Level: Medium 5. Which of the following is true about G study? a. use data from measurements and estimate variance components b. use data from measurements and estimate generalizability of scores’ various measurement strategies c. use estimate variance components and estimate generalizability of scores’ various measurement strategies d. use estimated variance components and estimate "universe" data from measurement strategies Ans: A Learning Objective: 13-3: Summarize the two-phase process of generalizability theory: G study and D study.

Cognitive Domain: Comprehension Answer Location: G Studies and D Studies Difficulty Level: Medium 6. Which of the following is true about D study? a. use data from measurements and estimate variance components b. use data from measurements and estimate generalizability of scores’ various measurement strategies c. use estimate variance components and estimate generalizability of scores’ various measurement strategies d. use estimated variance components and estimate "universe" data from measurement strategies Ans: C Learning Objective: 13-3: Summarize the two-phase process of generalizability theory: G study and D study.

Cognitive Domain: Comprehension Answer Location: G Studies and D Studies Difficulty Level: Medium 7. Archie is a researcher using variance components to estimate data collected through a measurement strategy of interest. The facets most strongly affecting observed score variance are identified, and Archie can estimate the effects. What is Archie most likely working through?

Instructor Resource Furr, Psychometrics, 4e SAGE Publishing, 2022 a. G study phase b. D study phase c. a nested design d. a crossed design Ans: A Learning Objective: 13-3: Summarize the two-phase process of generalizability theory: G study and D study.

Cognitive Domain: Analysis Answer Location: G Studies and D Studies Difficulty Level: Medium 8. Archie moves on in his study and begins to estimate the generalizability of various combinations of facets. What is Archie most likely working through at this point? a. G study phase b. D study phase c. a nested design d. a crossed design Ans: B Learning Objective: 13-3: Summarize the two-phase process of generalizability theory: G study and D study.

Cognitive Domain: Comprehension Answer Location: G Studies and D Studies Difficulty Level: Medium 9. If a test taker has a score of "1" for three areas (Hostile, Angry, and Belligerent), they will have a target mean of: a. 1 b. 1.3 c. 3.1 d. 3 Ans: A Learning Objective: 13-4: Explain the generalizability theory one-facet design.

Cognitive Domain: Application Answer Location: Conducting and Interpreting Generalizability Theory Analysis: A OneFacet Design Difficulty Level: Hard 10. The _________ reflects the degree to which targets elicit different mean ratings averaged across all items. a. ANOVA b. sum of squares c. residual effect d. target effects Ans: D Learning Objective: 13-4: Explain the generalizability theory one-facet design.

Cognitive Domain: Knowledge

Instructor Resource Furr, Psychometrics, 4e SAGE Publishing, 2022 Answer Location: Phase 1: G Study (Conducting and Interpreting Generalizability Theory Analysis: A One-Facet Design) Difficulty Level: Easy 11. This design is most likely to be used during Phase 1 of the G study. a. ANOVA b. sum of squares c. residual effect d. target effects Ans: A Learning Objective: 13-4: Explain the generalizability theory one-facet design.

Cognitive Domain: Comprehension Answer Location: Phase 1: G Study (Conducting and Interpreting Generalizability Theory Analysis: A One-Facet Design) Difficulty Level: Medium 12. The ___________ is the noise that potentially masks the signal of the target effect. a. ANOVA b. sum of squares c. residual effect d. target effects Ans: C Learning Objective: 13-4: Explain the generalizability theory one-facet design.

Cognitive Domain: Knowledge Answer Location: Phase 1: G Study (Conducting and Interpreting Generalizability Theory Analysis: A One-Facet Design) Difficulty Level: Easy 13. The variance components for ___________ indicates the degree to which some _______ elicit higher mean ratings than other _______. (Answer is the same for all blanks.) a. participants b. targets c. items d. studies Ans: C Learning Objective: 13-4: Explain the generalizability theory one-facet design.

Cognitive Domain: Comprehension Answer Location: Phase 1: G Study (Conducting and Interpreting Generalizability Theory Analysis: A One-Facet Design) Difficulty Level: Medium 14. To conduct a D study, test users estimate "coefficients of __________" for various measurement strategies. a. facets b. design

Instructor Resource Furr, Psychometrics, 4e SAGE Publishing, 2022 c. reliability d. generalizability Ans: D Learning Objective: 13-4: Explain the generalizability theory one-facet design.

Cognitive Domain: Comprehension Answer Location: Phase 2: D Study (Conducting and Interpreting Generalizability Theory Analysis: A One-Facet Design) Difficulty Level: Medium 15. A coefficient of _____________ is analogous to _________ as defined by CTT, in that it represents the degree to which the observed differences among target participants are consistent with the differences that would be obtained in a nearly unlimited number of observations were obtained. a. facets; reliability b. facets; generalizability c. generalizability; reliability d. generalizability; sample Ans: C Learning Objective: 13-4: Explain the generalizability theory one-facet design.

Cognitive Domain: Comprehension Answer Location: Phase 2: D Study (Conducting and Interpreting Generalizability Theory Analysis: A One-Facet Design) Difficulty Level: Medium 16. Generalizability coefficient = _________ / ______ + Noise (Both blanks are the same.) a. variance b. signal c. random d. fixed Ans: B Learning Objective: 13-4: Explain the generalizability theory one-facet design.

Cognitive Domain: Comprehension Answer Location: Phase 2: D Study (Conducting and Interpreting Generalizability Theory Analysis: A One-Facet Design) Difficulty Level: Medium 17. In multiple-facet designs, generalizability coefficients are estimated through an analysis of _________ to __________ as represented by ratios of variance components. a. variance; noise b. signal; noise c. facets; noise d. facets; signal Ans: B Learning Objective: 13-5: Explain the generalizability theory two-facet design.

Instructor Resource Furr, Psychometrics, 4e SAGE Publishing, 2022 Cognitive Domain: Comprehension Answer Location: Conducting and Interpreting Generalizability Theory Analysis: A TwoFacet Design Difficulty Level: Medium 18. Three components are most likely to be found in a _____-facet design. a. one b. two c. three d. four Ans: A Learning Objective: 13-5: Explain the generalizability theory two-facet design.

Cognitive Domain: Comprehension Answer Location: Conducting and Interpreting Generalizability Theory Analysis: A TwoFacet Design Difficulty Level: Medium 19. Additional complexity adds complexity to the _______ or error variance of generalizability coefficients. a. noise b. signal c. components d. targets Ans: A Learning Objective: 13-5: Explain the generalizability theory two-facet design.

Cognitive Domain: Comprehension Answer Location: Conducting and Interpreting Generalizability Theory Analysis: A TwoFacet Design Difficulty Level: Medium 20. A large _________ ´ Item interaction would indicate the items operate somewhat inconsistently across the __________(s). (Answer is the same for both blanks.) a. signal b. target c. observer d. component Ans: B Learning Objective: 13-5: Explain the generalizability theory two-facet design.

Cognitive Domain: Comprehension Answer Location: Phase 1: G Study (Conducting and Interpreting Generalizability Theory Analysis: A Two-Facet Design) Difficulty Level: Medium 21. A target ´ __________ interaction reflects the degree to which the _______ provides different rank orderings of the target. (Answer is the same for both blanks.) a. signal

Instructor Resource Furr, Psychometrics, 4e SAGE Publishing, 2022 b. item c. observer d. component Ans: C Learning Objective: 13-5: Explain the generalizability theory two-facet design.

Cognitive Domain: Comprehension Answer Location: Phase 1: G Study (Conducting and Interpreting Generalizability Theory Analysis: A Two-Facet Design) Difficulty Level: Medium 22. The _________ represents elements that might produce additional noise in a measurement. a. observer b. target c. item d. residual Ans: D Learning Objective: 13-5: Explain the generalizability theory two-facet design.

Cognitive Domain: Comprehension Answer Location: Phase 1: G Study (Conducting and Interpreting Generalizability Theory Analysis: A Two-Facet Design) Difficulty Level: Medium 23. With ________ factors, units of the facet in the study are of primary interest, not a sample from a universe, and not exchangeable with other units. a. random b. fixed c. crossed d. nested Ans: B Learning Objective: 13-6: Describe the variety of generalizability theory strategies based on measurement situations.

Cognitive Domain: Comprehension Answer Location: Random Versus Fixed Facets Difficulty Level: Medium 24. This kind of design occurs when all possible combinations of two facets are not included in the study. a. random b. fixed c. crossed d. nested Ans: D Learning Objective: 13-6: Describe the variety of generalizability theory strategies based on measurement situations.

Cognitive Domain: Comprehension

Instructor Resource Furr, Psychometrics, 4e SAGE Publishing, 2022 Answer Location: Crossed Versus Nested Designs Difficulty Level: Medium 25. These decisions are often called "criterion-referenced tests." a. relative b. absolute c. crossed d. nested Ans: B Learning Objective: 13-6: Describe the variety of generalizability theory strategies based on measurement situations.

Cognitive Domain: Comprehension Answer Location: Relative Versus Absolute Decisions Difficulty Level: Medium

True/False 1. Validity in a set of scores can be created by different facets of the measurement strategy.

Ans: T Learning Objective: 13-1: Describe how different measurement strategies affect scores in generalizability theory.

Cognitive Domain: Comprehension Answer Location: Multiple Facets of Measurement Difficulty Level: Medium 2. CTT can efficiently differentiate the effects of items and observers as separate sources of error.

Ans: F Learning Objective: 13-1: Describe how different measurement strategies affect scores in generalizability theory.

Cognitive Domain: Analysis Answer Location: Multiple Facets of Measurement Difficulty Level: Medium 3. CTT is applicable to complex measurement designs (having one or more facets).

Ans: F Learning Objective: 13-1: Describe how different measurement strategies affect scores in generalizability theory.

Cognitive Domain: Comprehension Answer Location: Multiple Facets of Measurement Difficulty Level: Medium

Instructor Resource Furr, Psychometrics, 4e SAGE Publishing, 2022 4. When a psychological or behavioral variable is measured, an infinite number of observations can be made.

Ans: F Learning Objective: 13-2: Discuss how generalizability theory relates to generalizability, universes, and variance components.

Cognitive Domain: Analysis Answer Location: Generalizability, Universes, and Variance Components Difficulty Level: Medium 5. In G theory analysis, estimates of generalizability are based on variance components representing the degree to which differences exist in the “universe” for each element of the design.

Ans: T Learning Objective: 13-2: Discuss how generalizability theory relates to generalizability, universes, and variance components.

Cognitive Domain: Comprehension Answer Location: Generalizability, Universes, and Variance Components Difficulty Level: Medium 6. A high level of generalizability would mean that there are minimal differences among items and does not produce much measurement error.

Ans: T Learning Objective: 13-4: Explain the generalizability theory one-facet design.

Cognitive Domain: Comprehension Answer Location: Conducting and Interpreting Generalizability Theory Analysis: A OneFacet Design Difficulty Level: Medium 7. G theory is not an alternative to CTT. Ans: F Learning Objective: 13-5: Explain the generalizability theory two-facet design.

Cognitive Domain: Analysis Answer Location: Conducting and Interpreting Generalizability Theory Analysis: A TwoFacet Design Difficulty Level: Medium 8. The analysis of a multiple-facet design works much like a one-facet design.

Ans: T Learning Objective: 13-5: Explain the generalizability theory two-facet design.

Cognitive Domain: Comprehension Answer Location: Conducting and Interpreting Generalizability Theory Analysis: A TwoFacet Design Difficulty Level: Medium 9. In multiple-facet designs, generalizability coefficients are estimated through an analysis of signal to noise as represented by ratios of variance components.

Instructor Resource Furr, Psychometrics, 4e SAGE Publishing, 2022 Ans: T Learning Objective: 13-5: Explain the generalizability theory two-facet design.

Cognitive Domain: Analysis Answer Location: Conducting and Interpreting Generalizability Theory Analysis: A TwoFacet Design Difficulty Level: Medium 10. The interpretation for the T ´ Ia effect means that targets were rank-ordered differently across observers.

Ans: F Learning Objective: 13-5: Explain the generalizability theory two-facet design.

Cognitive Domain: Comprehension Answer Location: Phase 1: G Study (Conducting and Interpreting Generalizability Theory Analysis: A Two-Facet Design) Difficulty Level: Medium Short Answer 1. Is G theory or CTT more suited for complex measurement strategies in which multiple facets might affect the scores?

Ans: Answer may vary.

Learning Objective: 13-1: Describe how different measurement strategies affect scores in generalizability theory.

Cognitive Domain: Analysis Answer Location: Multiple Facets of Measurement Difficulty Level: Medium 2. What two components in CTT make up the total variance in a measure’s observed scores?

Ans: Answer may vary.

Learning Objective: 13-1: Describe how different measurement strategies affect scores in generalizability theory.

Cognitive Domain: Comprehension Answer Location: Multiple Facets of Measurement Difficulty Level: Medium 3. What term is viewed as undifferentiated, amorphous, and monolithic?

Ans: Answer may vary.

Learning Objective: 13-1: Describe how different measurement strategies affect scores in generalizability theory.

Cognitive Domain: Comprehension Answer Location: Multiple Facets of Measurement Difficulty Level: Medium 4. What is the term used for an element of the measurement strategy, as in G theory?

Instructor Resource Furr, Psychometrics, 4e SAGE Publishing, 2022 Ans: Answer may vary.

Learning Objective: 13-1: Describe how different measurement strategies affect scores in generalizability theory.

Cognitive Domain: Knowledge Answer Location: Multiple Facets of Measurement Difficulty Level: Easy 5. In terms of G theory, measurement quality is evaluated in terms of the ability to make inferences in two ways. Name one of them.

Ans: Answer may vary.

Learning Objective: 13-2: Discuss how generalizability theory relates to generalizability, universes, and variance components.

Cognitive Domain: Application Answer Location: Generalizability, Universes, and Variance Components Difficulty Level: Hard 6. Which phase comes first, G study or D study?

Ans: Answer may vary.

Learning Objective: 13-3: Summarize the two-phase process of generalizability theory: G study and D study.

Cognitive Domain: Analysis Answer Location: G Studies and D Studies Difficulty Level: Medium 7. What is the range for generalizability coefficients?

Ans: Answer may vary.

Learning Objective: 13-4: Explain the generalizability theory one-facet design.

Cognitive Domain: Application Answer Location: Phase 2: D Study (Conducting and Interpreting Generalizability Theory Analysis: A One-Facet Design) Difficulty Level: Hard 8. What does signal / signal + noise equal?

Ans: Answer may vary.

Learning Objective: 13-4: Explain the generalizability theory one-facet design.

Cognitive Domain: Comprehension Answer Location: Phase 2: D Study (Conducting and Interpreting Generalizability Theory Analysis: A One-Facet Design) Difficulty Level: Medium 9. The residual or error term reflects which: consistency or inconsistency variance?

Ans: Answer may vary.

Learning Objective: 13-7: Describe how variance components tell us about data.

Cognitive Domain: Comprehension

Instructor Resource Furr, Psychometrics, 4e SAGE Publishing, 2022 Answer Location: A Practical, Consistency-Oriented Interpretation of Variance Components Difficulty Level: Medium 10. In the one-facet design, the target effects and the item effects are what kind of effects ?

Ans: Answer may vary.

Learning Objective: 13-7: Describe how variance components tell us about data.

Cognitive Domain: Comprehension Answer Location: A Practical, Consistency-Oriented Interpretation of Variance Components Difficulty Level: Medium

Instructor Resource Furr, Psychometrics, 4e SAGE Publishing, 2022 Chapter 14: Item Response Theory and Rasch Models Test Bank Multiple Choice 1. Item response theory is a psychometric approach emphasizing the fact that an individual’s response to a particular __________ is influenced by qualities of the individual and by qualities of the item. a. sample b. population c. test item d. theory Ans: C Learning Objective: 14-1: Describe determinants of responses to test items.

Cognitive Domain: Comprehension Answer Location: Factors Affecting Responses to Test Items Difficulty Level: Medium 2. Which of the following is not considered an item characteristic that could affect an individual’s response to a test item? a. item difficulty b. item discrimination c. guessing d. individual’s “trait level” Ans: D Learning Objective: 14-1: Describe determinants of responses to test items.

Cognitive Domain: Analysis Answer Location: Factors Affecting Responses to Test Items Difficulty Level: Medium 3. Which of the following is not considered an item response theory model? a. Rasch b. 1PL c. GRM d. qs Ans: D Learning Objective: 14-1: Describe determinants of responses to test items.

Cognitive Domain: Analysis Answer Location: Factors Affecting Responses to Test Items Difficulty Level: Medium 4. An employee who has a high level of job satisfaction endorses an item that measures job satisfaction. What is this an example of in terms of determinant of item response? a. respondent trait level

Instructor Resource Furr, Psychometrics, 4e SAGE Publishing, 2022 b. item difficulty c. item discrimination d. guessing Ans: A Learning Objective: 14-1: Describe determinants of responses to test items.

Cognitive Domain: Analysis Answer Location: Respondent Trait Level as a Determinant of Item Responses Difficulty Level: Medium 5. A very challenging math item is often answered incorrectly. What is this an example of in terms of determinant of item response? a. respondent trait level b. item difficulty c. item discrimination d. guessing Ans: B Learning Objective: 14-1: Describe determinants of responses to test items.

Cognitive Domain: Analysis Answer Location: Item Difficulty as a Determinant of Item Responses Difficulty Level: Medium 6. Which determinant of item response is defined as the trait level required for participants to have a .50 probability of answering the item correctly? a. respondent trait level b. item difficulty c. item discrimination d. guessing Ans: B Learning Objective: 14-1: Describe determinants of responses to test items.

Cognitive Domain: Comprehension Answer Location: Item Difficulty as a Determinant of Item Responses Difficulty Level: Medium 7. Which determinant of item response tries to differentiate individuals who have high trait levels from individuals who have low trait levels? a. respondent trait level b. item difficulty c. item discrimination d. guessing Ans: C Learning Objective: 14-1: Describe determinants of responses to test items.

Cognitive Domain: Comprehension Answer Location: Item Discrimination as a Determinant of Item Responses Difficulty Level: Medium

Instructor Resource Furr, Psychometrics, 4e SAGE Publishing, 2022 8. Which determinant of item response is analogous to an item-total correlation from CTT? a. respondent trait level b. item difficulty c. item discrimination d. guessing Ans: C Learning Objective: 14-1: Describe determinants of responses to test items.

Cognitive Domain: Comprehension Answer Location: Item Discrimination as a Determinant of Item Responses Difficulty Level: Medium 9. What is true about the following question: How many pecks are in three bushels? (a) 12 (b) 24 a. it has a high discrimination value b. it has a low discrimination value c. it does a good job of discriminating students who have a relatively high level of mathematical ability d. we can feel confident discerning the mathematical ability of up to two responders Ans: B Learning Objective: 14-1: Describe determinants of responses to test items.

Cognitive Domain: Analysis Answer Location: Item Discrimination as a Determinant of Item Responses Difficulty Level: Medium 10. Which component is related to the probability that participants will answer an item correctly, purely based on chance? a. respondent trait level b. item difficulty c. item discrimination d. guessing Ans: D Learning Objective: 14-1: Describe determinants of responses to test items.

Cognitive Domain: Comprehension Answer Location: Guessing Difficulty Level: Medium 11. Which components will produce a correct answer 50% of the time? a. item difficulty and item discrimination b. item discrimination and respondent trait level c. respondent trait level and guessing d. item difficulty and guessing Ans: D Learning Objective: 14-1: Describe determinants of responses to test items.

Cognitive Domain: Analysis

Instructor Resource Furr, Psychometrics, 4e SAGE Publishing, 2022 Answer Location: Guessing Difficulty Level: Medium 12. Which of the following is not a polytomous format for difficulty and discrimination parameters? a. partial credit model b. graded response model c. generalized partial credit model d. nominal response model Ans: A Learning Objective: 14-2: Summarize the measurement models developed from the IRT perspective.

Cognitive Domain: Comprehension Answer Location: IRT Measurement Models Difficulty Level: Medium 13. Which of the following is the simplest IRT model? a. 1PL/Rasch Model b. 2PL c. 3PL d. graded response (GRM) Ans: A Learning Objective: 14-2: Summarize the measurement models developed from the IRT perspective.

Cognitive Domain: Knowledge Answer Location: One-Parameter Logistic Model (or Rasch Model) Difficulty Level: Easy 14. Which IRT model only relates to item difficulty? a. 1PL/Rasch Model b. 2PL c. 3PL d. graded response (GRM) Ans: A Learning Objective: 14-2: Summarize the measurement models developed from the IRT perspective.

Cognitive Domain: Comprehension Answer Location: One-Parameter Logistic Model (or Rasch Model) Difficulty Level: Medium 15. According to this model, a person’s response to a binary item is determined by the individual’s trait level, item difficulty, and item discrimination. a. 1PL/Rasch Model b. 2PL c. 3PL d. graded response (GRM)

Instructor Resource Furr, Psychometrics, 4e SAGE Publishing, 2022 Ans: B Learning Objective: 14-2: Summarize the measurement models developed from the IRT perspective.

Cognitive Domain: Comprehension Answer Location: Two-Parameter Logistic Model Difficulty Level: Medium 16. Which model states that the probability of a respondent answering an item correctly is conditional on the respondent’s trait level, the item’s difficulty, the item’s discrimination, and a "guessing parameter"? a. 1PL/Rasch Model b. 2PL c. 3PL d. graded response (GRM) Ans: C Learning Objective: 14-2: Summarize the measurement models developed from the IRT perspective.

Cognitive Domain: Comprehension Answer Location: Three-Parameter Logistic Model Difficulty Level: Medium 17. Which model produces several difficulty parameters for each item? a. 1PL/Rasch Model b. 2PL c. 3PL d. graded response (GRM) Ans: D Learning Objective: 14-2: Summarize the measurement models developed from the IRT perspective.

Cognitive Domain: Comprehension Answer Location: Graded Response Model Difficulty Level: Medium 18. When estimating trait levels and item parameters, what happens after getting the actual responses to the test? a. compute probabilities b. initial estimates c. compare probabilities d. retain most recent estimates Ans: B Learning Objective: 14-3: Discuss how to estimate IRT-based parameters.

Cognitive Domain: Analysis Answer Location: Obtaining Parameter Estimates: A 1PL Example Difficulty Level: Medium

Instructor Resource Furr, Psychometrics, 4e SAGE Publishing, 2022 19. Item _______________ curves present and evaluate characteristics of the items on a test. a. model b. parameter c. characteristic d. application Ans: C Learning Objective: 14-5: Describe how IRT provides information about items and tests.

Cognitive Domain: Knowledge Answer Location: Item Characteristic Curves Difficulty Level: Easy 20. Higher information values indicate _________ psychometric quality. a. lower b. greater c. mixed d. immeasurable Ans: B Learning Objective: 14-5: Describe how IRT provides information about items and tests.

Cognitive Domain: Analysis Answer Location: Item Information and Test Information Difficulty Level: Medium 21. If we compute values at multiple trait levels, we could display the results in a graph called an: a. item characteristic curve b. respondent trait level c. model fit d. item information curve Ans: D Learning Objective: 14-5: Describe how IRT provides information about items and tests.

Cognitive Domain: Comprehension Answer Location: Item Information and Test Information Difficulty Level: Medium 22. What might we use to illustrate the degree to which a test provides a different quality of information at different trait levels? a. item characteristic curve b. respondent trait level c. model fit d. item information curve Ans: D Learning Objective: 14-5: Describe how IRT provides information about items and tests.

Cognitive Domain: Comprehension Answer Location: Item Information and Test Information Difficulty Level: Medium

Instructor Resource Furr, Psychometrics, 4e SAGE Publishing, 2022 23. What occurs when an item’s properties in one group are different from the item’s properties in another group? a. item information curve b. model fit c. differential item functioning d. person fit Ans: C Learning Objective: 14-6: Discuss how IRT can be applied to a variety of psychological domains.

Cognitive Domain: Comprehension Answer Location: Difficulty Level: Medium 24. The analysis of __________ is an attempt to identify individuals whose response pattern does not seem to fit any of the expected patterns of responses to a set of items. a. model fit b. computerized adaptive testing c. differential item functioning d. person fit Ans: D Learning Objective: 14-6: Discuss how IRT can be applied to a variety of psychological domains.

Cognitive Domain: Knowledge Answer Location: Person Fit Difficulty Level: Easy 25. _______________ works by using a very large item pool for which IRT has been used to obtain information about the psychometric properties of items. a. model fit b. computerized adaptive testing c. differential item functioning d. person fit Ans: B Learning Objective: 14-6: Discuss how IRT can be applied to a variety of psychological domains.

Cognitive Domain: Knowledge Answer Location: Computerized Adaptive Testing Difficulty Level: Easy

True/False 1. CTT is more conceptually complex than IRT.

Instructor Resource Furr, Psychometrics, 4e SAGE Publishing, 2022 Ans: F Learning Objective: 14-1: Describe determinants of responses to test items.

Cognitive Domain: Analysis Answer Location: Item Response Theory and Rasch Models Difficulty Level: Medium 2. Discrimination can affect an individual’s response to a test item.

Ans: T Learning Objective: 14-1: Describe determinants of responses to test items.

Cognitive Domain: Comprehension Answer Location: Factors Affecting Responses to Test Items Difficulty Level: Medium 3. The Rasch model is related to discrimination and guessing.

Ans: F Learning Objective: 14-2: Summarize the measurement models developed from the IRT perspective.

Cognitive Domain: Comprehension Answer Location: IRT Measurement Models Difficulty Level: Medium 4. The difference between 1PL and 2PL is the inclusion of the item discrimination parameter.

Ans: T Learning Objective: 14-2: Summarize the measurement models developed from the IRT perspective.

Cognitive Domain: Analysis Answer Location: Two-Parameter Logistic Model Difficulty Level: Medium 5. There is statistical software that can do the computational work needed for a variety of response models.

Ans: T Learning Objective: 14-3: Discuss how to estimate IRT-based parameters.

Cognitive Domain: Comprehension Answer Location: Obtaining Parameter Estimates: A 1PL Example Difficulty Level: Medium 6. In an IRT analysis, item characteristics are combined to reflect the characteristics of the test as a whole.

Ans: T Learning Objective: 14-5: Describe how IRT provides information about items and tests.

Cognitive Domain: Comprehension Answer Location: Item and Test Information Difficulty Level: Medium

Instructor Resource Furr, Psychometrics, 4e SAGE Publishing, 2022 7. Items do not have curves.

Ans: F Learning Objective: 14-5: Describe how IRT provides information about items and tests.

Cognitive Domain: Comprehension Answer Location: Item Characteristic Curves Difficulty Level: Medium 8. Item information curves are unable to vary in terms of location.

Ans: F Learning Objective: 14-5: Describe how IRT provides information about items and tests.

Cognitive Domain: Comprehension Answer Location: Item Information and Test Information Difficulty Level: Medium 9. IRT analyses can help create tests with strong psychometric properties across a range of trait levels.

Ans: T Learning Objective: 14-6: Discuss how IRT can be applied to a variety of psychological domains.

Cognitive Domain: Comprehension Answer Location: Test Development and Improvement Difficulty Level: Medium 10. Person fit is a method of computerized test administration that is intended to provide an accurate and very efficient assessment of individuals’ trait levels.

Ans: F Learning Objective: 14-6: Discuss how IRT can be applied to a variety of psychological domains.

Cognitive Domain: Knowledge Answer Location: Computerized Adaptive Testing Difficulty Level: Easy Short Answer 1. IRT provides procedures for obtaining information about: (name one of three options) Ans: Answer may vary.

Learning Objective: 14-1: Describe determinants of responses to test items.

Cognitive Domain: Comprehension Answer Location: Factors Affecting Responses to Test Items Difficulty Level: Medium 2. Which factor affecting a person’s probability of responding in a particular way to an item is related to, for example, a person who is good at math being able to respond correctly to a math item?

Ans: Answer may vary.

Instructor Resource Furr, Psychometrics, 4e SAGE Publishing, 2022 Learning Objective: 14-1: Describe determinants of responses to test items.

Cognitive Domain: Comprehension Answer Location: Respondent Trait Level as a Determinant of Item Responses Difficulty Level: Medium 3. Which two determinants of item responses are intrinsically connected?

Ans: Answer may vary.

Learning Objective: 14-1: Describe determinants of responses to test items.

Cognitive Domain: Comprehension Answer Location: Item Difficulty as a Determinant of Item Responses Difficulty Level: Medium 4. Which two determinants of item responses are scored on a standardized metric so that their means are 0 and the standard deviations are 1?

Ans: Answer may vary.

Learning Objective: 14-1: Describe determinants of responses to test items.

Cognitive Domain: Comprehension Answer Location: Item Difficulty as a Determinant of Item Responses Difficulty Level: Medium 5. What term refers to the ability to express the links between an outcome and the components that affect the outcome?

Ans: Answer may vary.

Learning Objective: 14-2: Summarize the measurement models developed from the IRT perspective.

Cognitive Domain: Knowledge Answer Location: IRT Measurement Models Difficulty Level: Easy 6. When considering which term is the key question whether the actual responses to a set of items are well represented by a given measurement model?

Ans: Answer may vary.

Learning Objective: 14-4: Identify how to choose the most appropriate model for IRT. Cognitive Domain: Comprehension Answer Location: Model Fit Difficulty Level: Medium 7. How should a test user proceed with interpreting information if the fit indices suggest a good fit between the model and the actual responses?

Ans: Answer may vary.

Learning Objective: 14-4: Identify how to choose the most appropriate model for IRT. Cognitive Domain: Analysis Answer Location: Model Fit Difficulty Level: Medium

Instructor Resource Furr, Psychometrics, 4e SAGE Publishing, 2022 8. Which term reflect the probabilities with which individuals across a range of trait levels are likely to answer each item correctly?

Ans: Answer may vary.

Learning Objective: 14-5: Describe how IRT provides information about items and tests.

Cognitive Domain: Knowledge Answer Location: Item Characteristic Curves Difficulty Level: Easy 9. From which perspective does a test’s psychometric quality vary across trait levels?

Ans: Answer may vary.

Learning Objective: 14-5: Describe how IRT provides information about items and tests.

Cognitive Domain: Comprehension Answer Location: Item Information and Test Information Difficulty Level: Medium 10. Which term’s presense means that a male and a female who have the same trait level have different probabilities of answering the item correctly?

Ans: Answer may vary.

Learning Objective: 14-6: Discuss how IRT can be applied to a variety of psychological domains.

Cognitive Domain: Comprehension Answer Location: Differential Item Functioning Difficulty Level: Medium