Beyond Performance: A Diagnostic Method for Evaluating Human-Advancement Claims in AI-Mediated Systems

Abstract

Claims that artificial intelligence has improved human performance increasingly appear in education, professional work, and public administration. Yet improvement in an AI-mediated system does not by itself establish that the human participant has advanced. Stronger outputs can coexist with losses in capacities necessary for meaningful agency, while formal oversight persists after practical authority has moved elsewhere.

This paper develops a pre-validation diagnostic method for evaluating those claims. Each claim must identify the human functioning at issue, the relevant population and role, the required contrast, the durability window, and the failure signs that would defeat it. The method then examines governance-critical capacity, authority in practice, governability and continuity, distribution, and time.

Applied to published educational evidence, the method supports claims of improved assisted performance but does not support a general claim of improved human learning. The paper offers a decision procedure for determining whether a progress claim succeeds, remains unproven, or fails given the evidence available; it does not present a validated instrument or universal index.