IRT-Style Scoring in AI Practice Apps Falls Short of True IQ Measurement

IntelligenceMax, an AI-powered learning platform, updates learner ability estimates after each answer and displays results on a familiar IQ scale using a logistic formula resembling item response theory. However, the item difficulty and discrimination parameters are assigned by the AI rather than derived from a calibrated norming sample, meaning the scores lack the empirical grounding of validated psychometric instruments. While the system can adaptively select questions near a learner's current estimate, it cannot establish clinical IQ, measure lasting cognitive change, or predict real-world performance. The platform also tracks a separate confidence-calibration score based on a Brier-style metric, but this does not validate item difficulty or the underlying ability estimate. Genuine validation would require stable reusable items, sufficient response data per item, population-level calibration checks, and controls for factors like retesting effects and regression to the mean.