my picture

Matthew Hesson-McInnis, Ph.D.

Professor and Coordinator

Psychology 442 — Test Theory
Course Syllabus for Spring, 2018

Section 01 — Tue & Thu 12:35 pm to 1:50 pm — DeG 13

Contact Information

Instructor

Dr. Matthew Hesson-McInnis
415 DeGarmo Hall
309.438.7266 (voice)
309.438.5789 (fax)
mshesso@ilstu.edu
http://www.ilstu.edu/~mshesso

Office Hours

Wednesdays from 1:00 pm to 3:00 pm
or by appointment

Required Text

Crocker, L., & Algina, J. (2006). Introduction to classical and modern test theory. Fort Worth, TX: Harcourt Brace Jovanovich.

Additional Readings

As assigned

Accommodations

Any student needing to arrange a reasonable accommodation for a documented disability should contact Access and Accommodation Services at 350 Fell Hall, 438-5853 (voice), 438-8620 (TDD).

Course Description

As described in the ISU catalogue, this course covers "theory and principles of test reliability and validity, topics in test theory, and statistical procedures applicable to psychometric methods." This course is an extension of the basic measurement principles covered in undergraduate courses on testing and measurement. Demonstration of proficiency in these basic measurement principles (established by completion of an undergraduate course in measurement, independent reading and study in measurement, etc.) is required for continued enrollment in Test Theory.

This course is geared toward understanding the statstical theoreis that underlie test scores and the methods used to determine the technical adequacy of test scores. In this class, we will balance the conceptual, in the form of theory and statistical bases, with the applied, in the form of data analysis using SPSS and writing about psychometric data. PSY 440 (Data Analysis and Methodology) is a prerequisite for this course, and students must have an understanding of basic statistical issues and familiarity with SPSS to succeed in this course.

Course Goals

  1. An understanding of the principles of test theory at both a theoretical and mathematical level, including a thorough understanding of classicla and modern views of reliability and validity.
  2. Familiarity with statistical procedures used to gain information about the psychometric adequacy of test scores and how to interpret data from these procedures.
  3. Competence with SPSS as a tool to manage and analyze data from tests.
  4. The ability to communicate information about test scores, reliability, and validity in a scientifically appropriate manner.

Approximate Course Schedule

Chapters refer to Crocker & Algina (1986).

Week OfTopic
Jan 16Introduction to Test Theory (Chapter 1)
Jan 23Statistical Concepts (Chapter 2)
Jan 30Process of Test Construction (Chapter 4)
Feb 6Linear Combinations & Composite Scores (Chapter 5)
Feb 13Reliability, True Scores, & Classical Test Theory (Chapter 6)
Feb 20Estimating Reliability (Chapter 7)
Feb 27Generalizability Theory (Chapter 8)
Mar 6Review (time permitting) and Midterm (Chapters 1 — 8)
Mar 13Spring Break
Mar 20Introduction to Validity (Chapter 10)
Mar 27Prediction & Classification (Chapter 11)
Apr 3Bias in Selection (Chapter 12)
Apr 10Exploratory Factor Analysis: Extraction (Chapter 13)
Apr 17Exploratory Factor Analysis: Rotation (Chapter 13)
Apr 24Item Analysis (Chapter 14)
May 1Item Response Theory (Chapter 15)
TBDFinal Exam (Chapters 1 — 8 & 10 — 15 )

Policies and Procedures

  • Homework (200 Points)

    Homework assignments will be assigned throughout the semester using ReggieNet.

    Students are encouraged to work on these assignments at the same time, allowing students to consult with each other on how to address each question or to compare answers. This type of consultation will facilitate students learning from each other. Despite the encouragement to work collaboratively to discuss solutions to the problems, each student is required to produce his or her own work product. It is not acceptable for the collaboration to be so extensive that a single assignment is produced, that verbatim copies are turned in, or that assignments differ only in superficial terms. In other words, group efforts should be process oriented and not product oriented. Thus, group work should never result in a single document or other work-product being created that represents more than a single person's direct efforts. Each student must express his or her own understanding of any group discussion, and each student must produce his or her own work product to demonstrate the mastery of the material covered in the assignment. Given these principles, group members should never share documents, data, or other work products created for the course. Please refer to the General Policies section of this syllabus for a discussion of the very serious penalties for academic dishonesty.

  • Midterm Examination (200 Points)

    The midterm exam will consist of short-answer questions designed to assess your understanding of conceptual and mathematical issues covered in the first half of the semester.

  • Final Examination (400 Points)

    The final examination will be formatted much like the midterm but will cover all material in the course cumulatively.

    Make-up exams will only be administered under the most extreme of circumstances. It is highly unlikely that a make-up exam will be given unless the student has made arrangements in advance for the make-up exam, and if a student is allowed to make up an exam, she or he will be expected to take the exam before the specified time in the syllabus if at all possible. The decisions of the instructor regarding make-up exams will be final. Do not fail to notify the instructor based on the assumption that your excuse is valid: Students who have what would have otherwise been a valid excuse for missing an exam but who do not contact the instructor in advance when he or she had the opportunity to do so will not be allowed to take a make-up exam.

  • Research Paper (200 Points)

    The research project will require you to review the literature regarding the reliability or validity of scores from an existing test in psychology. This paper must be formatted using APA style. This paper will be due in class on the last day of class (Thursday, April 30). Papers may be turned in early, but late papers will not be accepted and will be scored as zero points.

    The research paper requires you to review empirical evidence of the reliability or validity of a psychological test. The test you select is up to you, but it must be cleared with the instructor ahead of time. You have three options for your paper:

    • Option A: Conduct a reliability generalization study in which you informally meta-analyze published reliability coefficients of scores from a given test in psychology. For an example, see Yin, P. & Fan, X. (2000). Assessing the reliability of Beck Depression Inventory scores: Reliaiblity generalization across studies. Educational and Psychological Measurement, 60, 201-223.
    • Option B: Review the construct validity of scores from a given test in psychology and draw a conclusion about the construct the test scores measure. A formal meta-analysis is not necessary. For an example, see Kinicki, A.J., McKee-Ryan, F. M., Schriesheim, C.A., & Carson, K.P. (2002). Assessing the construct validity of the Job Descriptive Index: A review and meta-analysis. Journal of Applied Psychology, 87, 14-42.
    • Option C: review the evidence of criterion-related validity of scores from a given test in psychology as they relate to predicting a given criterion. A formal meta-analysis is not necessary. For an example, see Morrison, T., & Morrison, M. (1995). A meta-analytic assessment of the predictive validity of the quantitative and verbal components of the Graduate Record Examination with graduate grade point average representing the criterion of graudate successs. Educational and Psychological Measurement, 55, 309-316.

    Grading will be based on the following criteria:

    • The purposes and goals of the paper are clearly specified.
    • Data from a large and diverse group of research studies are incorporated into the paper.
    • The results of the research studies (i.e., reliability and validity findings) are clearly communicated in the paper, either singly or in a summary form.
    • The conclusions drawn concerning reliability or validity are appropriate given the results of the reviewed research studies.
    • The paper is well organized, the arguments are strong, adn the writing is clear (including proper grammar, spelling, mechanics, and adherence to APA style). You are encouraged to use sub-headings.

Final Grades

Final grades will be determined (solely) on the total number of points earned during the semester:

900 to 1000A
800 to 899B
700 to 799C
600 to 699D
0 to 599F

General Policies

I welcome the opportunity to discuss with each and every student her or his progress in the course, problem areas, and strategies for improving performance. I am available during my office hours and by appointment. I welcome students during these times for clarification on lecture topics, material in the book, assistance with homework problems, and general problem solving for issues related to the course. I reserve the right to change my office hours to accommodate student need. Any questions about grading should be directed to the instructor; appeals for additional point values on graded work, however, must be made in writing. All appeals must be made within one week after the work has been returned.

Instances of academic dishonesty (e.g., cheating, plagiarism, making false statements to obtain any exception to course policies or deadlines) will result in a failing grade (i.e., "F") being assigned for the course and may result in referral to the appropriate University office for disciplinary action. Additionally, academic dishonesty and other behavior inconsistent with the APA Code of Ethics will be handled on a case-by-case manner, in consultation with members of the student's program or sequence faculty, in a manner consistent with the spirit of the APA Code of Ethics. Please refer to the Grading Policies entry for Homework Assignments for further discussion of academic honesty as it applies to working in small groups. Finally, the consequences of academic dishonesty will be applied equally to all students involved; the instructor will make no effort to determine who copied work from whom, as this distinction is meaningless in that all individuals involved are culpable.

Finally, all students are expected to conduct themselves in the classroom in a polite and respectful manner and in accordance with the student code of conduct. I expect all students to respect their colleagues by not behaving in any way that disrupts the classroom or makes it difficult for others to participate in class. Students who need to leave early are expected to sit close to the door and leave quietly. Likewise, students who arrive late are expected to sit in the most easily accessible seat and not climb over backpacks and books to sit in the back row.