Chapter 1: Introduction to Educational Assessment

I. Introduction

II. The Language of Assessment

A. Tests, Measurement, & Assessment

B. Types of Tests

C. Types of Scores Interpretations

III. Assumptions of Educational Assessment

A. Psychological and educational constructs exist.

B. Psychological and educational constructs can be measured.

C. While we can measure constructs, our measurement is not perfect.

D. There are different ways to measure any given construct.

E. All assessment procedures have strengths and limitations.

F. Multiple sources of information should be part of the assessment process.

G. Performance on tests can be generalized to non-test behaviors.

H. Assessment can provide information that helps educators make better educational decisions.

I. Assessments can be conducted in a fair manner.

J. Testing and assessment can benefit our educational institutions and society as a whole.

IV. Participants in the Assessment Process

A. People who develop tests.

B. People who use tests.

C. People who take tests.

D. Other people involved in the assessment process.

V. Common Application of Educational Assessments

A. Student Evaluation

B. Instructional Decisions

C. Selection, Placement, and Classification Decisions

D. Policy Decisions

E. Counseling and Guidance Decisions

VI. What Teachers Need to Know About Assessment

A. Teachers should be proficient in selecting professionally developed assessment

procedures that are appropriate for making instructional decisions.

B. Teachers should be proficient in developing assessment procedures that are

appropriate for making instructional decisions.

C. Teachers should be proficient in administering, scoring, and interpreting

professionally developed and teacher-made assessment procedures.

D. Teachers should be proficient in using assessment results when making

educational decisions.

E. Teachers should be proficient in developing valid grading procedures that

incorporate assessment information.

F. Teachers should be proficient in communicating assessment results.

G. Teachers should be proficient in recognizing unethical, illegal, and other

inappropriate uses of assessment procedures or information.

VII. Educational Assessment in the 21st Century

A. Computerized Adaptive Testing (CAT) and Other Technological Advances.

B. Authentic Assessments

C. Educational Accountability and High-Stakes Assessment

D. Trends in the Assessment of Students with Disabilities

VIII. Summary

Tables

A. Table 1.1: Major Categories of Tests

B. Table 1.2: Norm- and Criterion-Referenced Scores

C. Table 1.3: Assumptions of Educational Assessment

D. Table 1.4: Common Applications of Educational Assessments

E. Table 1.5: Teacher Competencies in Educational Assessment

Special Interest Topics

A. Special Interest Topic 1.1: Cognitive Diagnostic Assessment Another Step Toward Unifying Assessment and Instruction

B. Special Interest Topic 1.2: Technology and Assessment in the Schools

C. Special Interest Topic 1.3: Princeton Review's Rankings of High-Stakes Testing Programs

D. Special Interest Topic 1.4: The Nation's Report Card

E. Special Interest Topic 1.5: What Does the 21st Century Hold for the Assessment Profession?

Chapter 2: The Basic Mathematics of Measurement

I. The Role of Mathematics in Assessment

II. Scales of Measurement

A. What is Measurement?

B. Nominal Scales

C. Ordinal Scales

D. Interval Scales

E. Ratio Scales

III. The Description of Test Scores

A. Distributions

B. Measures of Central Tendency

C. Measures of Variability

IV. Correlation Coefficients

A. Scatterplots

B. Correlation and Prediction

C. Types of Correlation Coefficients

D. Correlation and Causality

V. Summary

Tables

Table 2.1: Common Nominal, Ordinal, Interval, & Ratio Scales

Table 2.2: Distribution of Scores for 20 Students

Table 2.3: Ungrouped Frequency Distribution

Table 2.4: Group Frequency Distribution

Table 2.5: Calculating the Standard Deviation and Variance

Table 2.6: Calculating a Pearson Correlation Coefficient

Figures

Figure 2.1: Graph of the Homework Scores

Figure 2.2: Hypothetical Distribution of Large Standardization Sample
Figure 2.3: Negatively Skewed Distribution

Figure 2.4: Positively Skewed Distribution

Figure 2.5: Bimodal Distribution

Figure 2.6: Relationship between Mean, Median, and Mode in Normal and Skewed Distributions

Figure 2.7: Three Distributions with Different Degrees of Variability

Figure 2.8: Scatterplots of Different Correlation Coefficients

Special Interest Topics

Special Interest Topic 2.1: Population Parameters and Sample Statistics

Special Interest Topic 2.2: A Public Outrage: Physicians Overcharge Their Patients

Special Interest Topic 2.3: Is the Variance Always Larger Than the Standard Deviation?

Special Interest Topic 2.4: Caution: Drawing Conclusions of Causality

Chapter 3: The Meaning of Test Scores

I. Introduction

II. Norm-Referenced & Criterion-Referenced Score Interpretations

A. Norm-Referenced Interpretations

B. Criterion-Referenced Interpretations

III. Norm-Referenced, Criterion-Referenced, or Both?

IV. Qualitative Description of Scores

V. Summary

Tables

Table 3.1: Transforming Raw Scores to Standard Scores

Table 3.2: Relationship of Different Standard Score Formats

Table 3.3: Converting Standard Scores From One Format to Another

Table 3.4: Characteristics of Norm-Referenced and Criterion-Referenced Scores

Figures

Figure 3.1: Illustration of the Normal Distribution

Figure 3.2: Normal Distribution with Mean, Standard Deviation, & Percentages.

Figure 3.3: Normal Distribution Illustrating the Relationship among Standard Scores.

Special Interest Topics Special Interest Topic 3.1: The Flynn Effect. Special Interest Topic 3.2: Whence the Normal Curve?

Special Interest Topic 3.3: Why do IQ Tests use a Mean of 100 and Standard Deviation of 15?

Special Interest Topic 3.4: The History of Stanine Scores

Special Interest Topic 3.5: Every Child on Grade Level?

Chapter 4: Reliability for Teachers

I. Introduction

II. Errors of Measurement

A. Sources of Measurement Error

III. Methods of Estimating Reliability

A. Test-Retest Reliability

B. Alternate Form Reliability

C. Internal Consistency Reliability

D. Inter-Rater Reliability

E. Reliability of Composite Scores

F. Selecting a Reliability Coefficient

G. Evaluating Reliability Coefficients

H. How to Improve Reliability

I. Special Problems in Estimating Reliability

IV. The Standard Error of Measurement

A. Evaluating the Standard Error of Measurement

V. Reliability: Practical Strategies for Teachers

VI. Summary

Tables

Table 4.1: Major Types of Reliability

Table 4.2: Half-Test Coefficients and Corresponding Full-Test Coefficients Corrected

with the Spearman-Brown Formula

Table 4.3: Calculation of KR 20

Table 4.4: Calculation of Coefficient Alpha

Table 4.5: Calculating Inter-Rater Agreement

Table 4.6: Source of Error Variance Associated with Major Types of Reliability

Table 4.7: Reliability Expected When Increasing the Numbers of Items

Table 4.8: Standard Errors of Measurement for Values of Reliability and Standard Deviations

Table 4.9: Reliability Estimates for Tests with a Mean of 80%

Figures

Figure 4.1: Partitioning the Variance

Special Interest Topics

Special Interest Topic 4.1: Generalizability Theory

Special Interest Topic 4.2: Consistency of Classification with Mastery Tests

Special Interest Topic 4.3: A Quick Way To Estimate Reliability for Classroom Exams

Chapter 5: Validity for Teachers

I. Introduction

A. Threats to Validity

B. Reliability & Validity

II. "Types of Validity" versus "Types of Validity Evidence"

III. Types of Validity Evidence

A. Evidence Based on Test Content

B. Evidence Based on Relations to Other Variables

C. Evidence Based on Internal Structure

D. Evidence Based on Response Processes

E. Evidence Based on Consequences of Testing

F. Integrating Evidence of Validity

IV. Validity: Practical Strategies for Teachers

V. Chapter Summary

Tables

Table 5.1: Tracing Historical Trends in the Concept of Validity

Table 5.2: Sources of Validity Evidence

Figures

Figure 5.1: Illustration of Item Relevance

Figure 5.2: Illustration of Content Coverage

Figure 5.3: Predictive and Concurrent Studies

Figure 5.4: Graph of a Regression Line

Special Interest Topic

Special Interest Topic 5.1: Regression, Prediction, and Your First Algebra Class

Chapter 6: Item Analysis for Teachers

I. Introduction

II. Item Difficulty Index (or Item Difficulty Level)

A. Special Assessment Situations and Item Difficulty

III. Item Discrimination

A. Item Discrimination on Mastery Tests

B. Difficulty and Discrimination on Speed Tests

IV. Distracter Analysis

A. How Distracters Influence Item Difficulty and Discrimination

V. Item Analysis: Practical Strategies for Teachers

VI. Using Item Analysis...