Psychological statistics

Psychometrics

Psychometrics deals with measurement of psychological attributes. It involves developing and applying statistical models for mental measurements. The measurement theories are divided into two major areas: (1) Classical test theory; (2) Item Response Theory.

Classical test theory

The classical test theory or true score theory or reliability theory in statistics is a set of statistical procedures useful for development of psychological tests and scales. It is based on a fundamental equation, X = T + E where, X is total score, T is a true score and E is error of measurement. For each participant, it assumes that there exist a true score and it need to be obtained score (X) has to be as close to it as possible. The closeness of X has with T is expressed in terms of ratability of the obtained score. The reliability in terms of classical test procedure is correlation between true score and obtained score. The typical test construction procedures has following steps:

(1) Determine the construct (2) Outline the behavioral domain of the construct (3) Write 3 to 5 times more items than desired test length (4) Get item content analyzed by experts and cull items (5) Obtain data on initial version of the test (6) Item analysis (Statistical Procedure) (7) Factor analysis (Statistical Procedure) (8) After the second cull, make final version (9) Use it for research

Reliability

The reliability is computed in specific ways. (A) Inter-Rater reliability: Inter-Rater reliability is estimate of agreement between independent raters. This is most useful for subjective responses. Cohen's Kappa, Krippendorff's Alpha, Intra-Class correlation coefficients, Correlation coefficients, Kendal's concordance coefficient, etc. are useful statistical tools. (B) Test-Retest Reliability: Test-Retest Procedure is estimation of temporal consistency of the test. A test is administered twice to the same sample with a time interval. Correlation between two sets of scores is used as an estimate of reliability. Testing conditions are assumed to be identical. (C) Internal Consistency Reliability: Internal consistency reliability estimates consistency of items with each other. Split-half reliability (Spearman- Brown Prophecy) and Cronbach Alpha are popular estimates of this reliability. (D) Parallel Form Reliability: It is an estimate of consistency between two different instruments of measurement. The inter-correlation between two parallel forms of a test or scale is used as an estimate of parallel form reliability.

Validity

Validity of a scale or test is ability of the instrument to measure what it purports to measure. Construct validity, Content Validity, and Criterion Validity are types of validity. Construct validity is estimated by convergent and discriminant validity and factor analysis. Convergent and discriminant validity are ascertained by correlation between similar of different constructs. Content Validity: Subject matter experts evaluate content validity. Criterion Validity is correlation between the test and a criterion variable (or variables) of the construct. Regression analysis, Multiple regression analysis, and Logistic regression are used as an estimate of criterion validity. Software applications: The R software has ‘psych’ package that is useful for classical test theory analysis.

Modern test theory

The modern test theory is based on latent trait model. Every item estimates the ability of the test taker. The ability parameter is called as theta (θ). The difficulty parameter is called b. the two important assumptions are local independence and unidimensionality. The Item Response Theory has three models. They are one parameter logistic model, two parameter logistic model and three parameter logistic model. In addition, Polychromous IRT Model are also useful.

The R Software has ‘ltm’, packages useful for IRT analysis.