Psychology 442 — Test Theory
Course Syllabus for Spring, 2020
Section 01 — Tue & Thu 2:00 pm to 3:15 pm — DeG 13
InstructorDr. Matthew Hesson-McInnis
415 DeGarmo Hall
Office HoursTuesdays and Thursdays from 11:00 am to 12:00 pm
or by appointment
Crocker, L., & Algina, J. (2006). Introduction to classical and modern test theory. Fort Worth, TX: Harcourt Brace Jovanovich.
Any student needing to arrange a reasonable accommodation for a documented disability should contact Access and Accommodation Services at 350 Fell Hall, 438-5853 (voice), 438-8620 (TDD).
As described in the ISU catalogue, this course covers "theory and principles of test reliability and validity, topics in test theory, and statistical procedures applicable to psychometric methods." This course is an extension of the basic measurement principles covered in undergraduate courses on testing and measurement. Demonstration of proficiency in these basic measurement principles (established by completion of an undergraduate course in measurement, independent reading and study in measurement, etc.) is required for continued enrollment in Test Theory.
This course is geared toward understanding the statstical theoreis that underlie test scores and the methods used to determine the technical adequacy of test scores. In this class, we will balance the conceptual, in the form of theory and statistical bases, with the applied, in the form of data analysis using SPSS and writing about psychometric data. PSY 440 (Data Analysis and Methodology) is a prerequisite for this course, and students must have an understanding of basic statistical issues and familiarity with SPSS to succeed in this course.
- An understanding of the principles of test theory at both a theoretical and mathematical level, including a thorough understanding of classicla and modern views of reliability and validity.
- Familiarity with statistical procedures used to gain information about the psychometric adequacy of test scores and how to interpret data from these procedures.
- Competence with SPSS as a tool to manage and analyze data from tests.
- The ability to communicate information about test scores, reliability, and validity in a scientifically appropriate manner.
Approximate Course Schedule
Chapters refer to Crocker & Algina (1986).
Week Of Topic Jan 14 Introduction to Test Theory (Chapter 1) Jan 21 Statistical Concepts (Chapter 2) Jan 28 Process of Test Construction (Chapter 4) Feb 4 Linear Combinations & Composite Scores (Chapter 5) Feb 11 Reliability, True Scores, & Classical Test Theory (Chapter 6) Feb 18 Estimating Reliability (Chapter 7) Feb 25 Generalizability Theory (Chapter 8) Mar 5 Review (time permitting) and Midterm (Chapters 1 — 8) Mar 10 Spring Break Mar 17 Introduction to Validity (Chapter 10) Mar 24 Prediction & Classification (Chapter 11) Mar 31 Bias in Selection (Chapter 12) Apr 7 Exploratory Factor Analysis: Extraction (Chapter 13) Apr 14 Exploratory Factor Analysis: Rotation (Chapter 13) Apr 21 Item Analysis (Chapter 14) Apr 28 Item Response Theory (Chapter 15) TBD Final Exam (Chapters 1 — 8 & 10 — 15 )
Policies and Procedures
Homework (200 Points)
Homework assignments will be assigned throughout the semester using ReggieNet.
Students are encouraged to work on these assignments at the same time, allowing students to consult with each other on how to address each question or to compare answers. This type of consultation will facilitate students learning from each other. Despite the encouragement to work collaboratively to discuss solutions to the problems, each student is required to produce his or her own work product. It is not acceptable for the collaboration to be so extensive that a single assignment is produced, that verbatim copies are turned in, or that assignments differ only in superficial terms. In other words, group efforts should be process oriented and not product oriented. Thus, group work should never result in a single document or other work-product being created that represents more than a single person's direct efforts. Each student must express an independent understanding of any group discussion, and each student must produce an independent work product to demonstrate the mastery of the material covered in the assignment. Given these principles, group members should never share documents, data, or other work products created for the course. Please refer to the General Policies section of this syllabus for a discussion of the very serious penalties for academic dishonesty.
Midterm Examination (200 Points)
The midterm exam will consist of short-answer questions designed to assess your understanding of conceptual and mathematical issues covered in the first half of the semester.
Final Examination (400 Points)
The final examination will be formatted much like the midterm but will cover all material in the course cumulatively.
Make-up exams will only be administered under the most extreme of circumstances. It is highly unlikely that a make-up exam will be given unless the student has made arrangements in advance for the make-up exam, and students who are allowed to make up an exam will be expected to take the exam before the specified time in the syllabus if at all possible. The decisions of the instructor regarding make-up exams will be final. Do not fail to notify the instructor based on the assumption that your excuse is valid: Students who have what would have otherwise been a valid excuse for missing an exam but who do not contact the instructor in advance with an opportunity to do so will not be allowed to take a make-up exam.
Research Paper (200 Points)
The research project will require you to review the literature regarding the reliability or validity of scores from an existing test in psychology. This paper must be formatted using APA style. This paper will be due in class on the last day of class (Thursday, April 30). Papers may be turned in early, but late papers will not be accepted and will be scored as zero points.
The research paper requires you to review empirical evidence of the reliability or validity of a psychological test. The test you select is up to you, but it must be cleared with the instructor ahead of time. You have three options for your paper:
- Option A: Conduct a reliability generalization study in which you informally meta-analyze published reliability coefficients of scores from a given test in psychology. For an example, see Yin, P. & Fan, X. (2000). Assessing the reliability of Beck Depression Inventory scores: Reliaiblity generalization across studies. Educational and Psychological Measurement, 60, 201-223.
- Option B: Review the construct validity of scores from a given test in psychology and draw a conclusion about the construct the test scores measure. A formal meta-analysis is not necessary. For an example, see Kinicki, A.J., McKee-Ryan, F. M., Schriesheim, C.A., & Carson, K.P. (2002). Assessing the construct validity of the Job Descriptive Index: A review and meta-analysis. Journal of Applied Psychology, 87, 14-42.
- Option C: review the evidence of criterion-related validity of scores from a given test in psychology as they relate to predicting a given criterion. A formal meta-analysis is not necessary. For an example, see Morrison, T., & Morrison, M. (1995). A meta-analytic assessment of the predictive validity of the quantitative and verbal components of the Graduate Record Examination with graduate grade point average representing the criterion of graudate successs. Educational and Psychological Measurement, 55, 309-316.
Grading will be based on the following criteria:
- The purposes and goals of the paper are clearly specified.
- Data from a large and diverse group of research studies are incorporated into the paper.
- The results of the research studies (i.e., reliability and validity findings) are clearly communicated in the paper, either singly or in a summary form.
- The conclusions drawn concerning reliability or validity are appropriate given the results of the reviewed research studies.
- The paper is well organized, the arguments are strong, and the writing is clear (including proper grammar, spelling, mechanics, and adherence to APA style). You are encouraged to use sub-headings.
Final grades will be determined (solely) on the total number of points earned during the semester:
900 to 1000 A 800 to 899 B 700 to 799 C 600 to 699 D 0 to 599 F
I welcome the opportunity to discuss with each and every student her or his progress in the course, problem areas, and strategies for improving performance. I am available during my office hours and by appointment. I welcome students during these times for clarification on lecture topics, material in the book, assistance with homework problems, and general problem solving for issues related to the course. I reserve the right to change my office hours to accommodate student need. Any questions about grading should be directed to the instructor; appeals for additional point values on graded work, however, must be made in writing. All appeals must be made within one week after the work has been returned.
Instances of academic dishonesty (e.g., cheating, plagiarism, making false statements to obtain any exception to course policies or deadlines) will result in a failing grade (i.e., "F") being assigned for the course and may result in referral to the appropriate University office for disciplinary action. Additionally, academic dishonesty and other behavior inconsistent with the APA Code of Ethics will be handled on a case-by-case manner, in consultation with members of the student's program or sequence faculty, in a manner consistent with the spirit of the APA Code of Ethics. Please refer to the Grading Policies entry for Homework Assignments for further discussion of academic honesty as it applies to working in small groups. Finally, the consequences of academic dishonesty will be applied equally to all students involved; the instructor will make no effort to determine who copied work from whom, as this distinction is meaningless in that all individuals involved are culpable.
Finally, all students are expected to conduct themselves in the classroom in a polite and respectful manner and in accordance with the student code of conduct. I expect all students to respect their colleagues by not behaving in any way that disrupts the classroom or makes it difficult for others to participate in class. Students who need to leave early are expected to sit close to the door and leave quietly. Likewise, students who arrive late are expected to sit in the most easily accessible seat.
School Psychology Students
Please note the following aims
PSY 442, Test Theory
Aim I: To prepare competent entry-level school psychologists who possess foundational and integrated knowledge across the basic areas of the discipline of psychology, which they will demonstrate by using basic psychological principles to understand and explain human behavior.
Demonstrate advanced knowledge of psychometrics
Illinois State University, as an equal opportunity/affirmative action employer, complies with all applicable federal and state laws regarding affirmative action, nondiscrimination, and anti-harassment. Illinois State University is committed to a policy of equal opportunity for all persons and does not discriminate on the basis of race, color, religion, sex, national origin, sexual orientation, order of protection, gender identity and expression, ancestry, age, marital status, disability, genetic information, unfavorable military discharge, or status as a veteran in employment, educational programs and activities, or admissions. Inquiries or complaints may be addressed by contacting the director of the Office of Equal Opportunity and Access by email at EqualOpportunity@IllinoisState.edu, by calling (309) 438-3383, or by mailing to the office at Illinois State University, Campus Box 1280, Normal, IL 61790.