Phase Four Empirical Evidence Review

California Education and AI Empirical Evidence Review

Phase Four Learning, mental fitness, wellness, and course design

Main finding

The research does not support a simple conclusion that AI either improves or damages education. It supports a more specific finding: general-purpose AI can raise immediate task performance while weakening unaided learning when it supplies answers or performs the central mental act. Purpose-built tutoring, staged assistance, and AI support for human educators can improve learning when they preserve reasoning and feedback. The outcome depends on what the tool does, when it enters the task, and how the student later demonstrates understanding.

The strongest present evidence concerns short-term mathematics, physics, programming, writing, and instructional support. Evidence about long-term memory, independent judgment, student relationships, mental health, and development remains limited. Claims about a well-formed mind therefore require direct measurement over time; course policies and student products alone cannot establish the result.

Evidence standard

This review gives greater weight to peer-reviewed randomized or quasi-experimental studies, then to credible preprints and research syntheses. A result is treated as direct only when the study measures the same population, task, exposure, and outcome. General learning-science findings can justify a design principle, but they do not prove that a particular AI policy works in a particular school.

Empirical evidence at a glance

What the 2026 evidence synthesis adds

Stanford SCALE reviewed more than 800 papers relevant to AI in K–12 education and identified only 20 with causal evidence strong enough for its findings. None of the student-facing causal studies took place in a U.S. K–12 setting. Most evidence remains recent, narrow, and short term. The synthesis nevertheless found a recurring pattern: assisted performance usually improves, unaided transfer is mixed, pedagogical guardrails matter, and human mediation can improve results. It also found little causal evidence on equity, emotional development, relationships, or wellness. [1]

Performance is not the same as learning

Bastani and colleagues randomly provided nearly 1,000 high-school mathematics students with either a standard GPT-4 interface, a tutoring version designed to protect learning, or no AI. During practice, grades rose 48 percent with the standard interface and 127 percent with the guarded tutor. When AI was removed, the standard interface group scored 17 percent lower than students who never had access. The guarded tutor largely avoided that loss. This is direct evidence that a better assisted product can conceal weaker independent learning. [2]

The study supports three requirements: separate assisted practice from unaided assessment; configure AI to prompt steps rather than return complete answers; and test whether students can transfer the skill after support is removed. It does not establish the same effect in every subject, age group, or AI system.

Purpose-built tutoring can improve learning

A peer-reviewed crossover experiment in a Harvard college physics course found that students using a custom AI tutor learned more in less time than students receiving an in-class active-learning lesson on the same material. Students also reported greater engagement and motivation. The tutor was deliberately built around established pedagogical practices, so the result should not be generalized to an ordinary chatbot or an entire course. [3]

Tutor CoPilot provides a different model. In a randomized field trial, AI supplied real-time instructional suggestions to human tutors serving students from underserved communities. Students were 4 percentage points more likely to master lesson topics; gains reached 9 points for students assigned to lower-rated tutors. Message analysis showed more probing questions and less generic praise. The study is a working paper, but its field scale and randomized design make it important evidence for using AI to strengthen human instruction. [4]

Cognitive offloading and memory

Cognitive offloading means using an external action or tool to reduce the mental work required by a task. Offloading is not inherently harmful. Notes, calculators, search engines, and AI can free limited attention for higher-order work. The risk arises when the tool performs the retrieval, representation, or reasoning that the student is supposed to learn. A major review describes offloading as a strategy shaped by task demands and by people’s beliefs about their own memory. [5]

Earlier experiments found that people remember less of information they expect a computer to store, while remembering where to find it. This “Google effect” concerns transactive memory and does not prove that internet use or AI causes broad cognitive decline. It does show that expected access changes what people encode. [6]

Retrieval practice supplies a stronger educational anchor. In a controlled vocabulary-learning experiment, repeated testing after initial learning produced substantially better delayed recall than repeated study, while students’ predictions did not track their actual performance. [7] AI policies should therefore preserve regular opportunities to produce answers, explanations, and arguments from memory without assistance.

Writing and authorship

The most discussed direct study of LLM-assisted essay writing remains preliminary. The MIT Media Lab preprint assigned 54 participants to LLM, search-engine, or no-tool conditions for three sessions; 18 completed a fourth crossover session. The LLM group showed the weakest measured EEG connectivity, the lowest self-reported ownership, and poorer ability to quote its own essays. The authors describe the work as a preprint with a small, geographically limited sample and warn against generalizing beyond the essay task or the tested system. [8]

The study justifies testing recall, ownership, and process when AI participates in writing. It does not justify claims that AI makes students less intelligent or permanently changes the brain. A defensible course design protects direct reading, an unaided conception or first draft, source checking, revision decisions, and a brief oral account of the argument.

Academic integrity and reliable verification

Authorship cannot be inferred safely from a detector score. Liang and colleagues tested several GPT detectors and found that they frequently misclassified writing by non-native English writers while classifying native English writing more accurately; simple prompting could also bypass detection. [9] The appropriate response is assessment design and evidence of process, followed by fair human review when concerns arise.

• Require a short disclosure of the tools used, the permitted purpose, and the student’s decisions.
• Retain drafts, notes, code history, calculations, or source annotations when these reveal the learning process.
• Use an oral explanation, supervised problem, or independent revision to verify understanding.
• Treat automated detection as a lead requiring corroboration, never as the sole basis for a penalty.

Attention, relationships, and wellness

Mental fitness is not a single clinical measure. For this project it should be treated as a defined set of educational capacities: sustained attention, recall, tolerance of productive difficulty, explanation, judgment, self-regulation, responsibility, and healthy participation with other people. Each capacity needs its own measure.

Current causal AI research says little about long-term attention, anxiety, depression, sleep, belonging, loneliness, family relationships, or prosocial development. Stanford’s 2026 synthesis identifies these as major evidence gaps. [1] Schools should not label convenience, satisfaction, or engagement as mental fitness. They should measure whether AI use changes these outcomes and whether effects differ by age, disability, language, prior achievement, family support, or existing distress.

Mental fitness across time

A historical comparison is useful if it compares equivalent measures and separates overlapping causes. It should not compare a nostalgic description of the past with a broad impression of the present. The most useful design uses repeated indicators such as reading and mathematics performance, attendance, sleep, distress, school connectedness, sustained attention, and independent task performance.

CDC data show that adolescent mental-health indicators were worsening before the COVID-19 pandemic. In 2023, 39.7 percent of U.S. high-school students reported persistent sadness or hopelessness and 28.5 percent reported poor mental health. Some measures improved slightly from 2021 to 2023, while ten-year trends remained concerning. These surveillance data document prevalence and associations; they do not identify AI or COVID as the single cause. [10]

The same rule applies to academic performance. National data show both a large pandemic-era shock and longer declines that began before 2020. In 2024, average fourth-grade mathematics performance improved from 2022 but remained below 2019. The Education Scorecard reports substantial variation in recovery and identifies chronic absenteeism as an ongoing constraint. [11] [12] A valid comparison must therefore model pre-pandemic trend, pandemic disruption, recovery conditions, and later AI exposure separately.

COVID, school closure, and family conditions

COVID-19 must be an explicit part of this investigation. School closures moved responsibility for instruction, technology, scheduling, and supervision into homes with sharply unequal resources. Students experienced different combinations of live teaching, emergency remote instruction, independent work, interrupted services, and little or no effective instruction. A single “online learning” category would erase these differences.

CDC’s 2021 pandemic survey found that 37 percent of high-school students reported poor mental health during the pandemic, 55 percent reported emotional abuse by a parent or other adult in the home, and 29 percent reported an adult in the household losing a job. The survey also states that mental health had been worsening before the pandemic. [13] These findings support measuring family stress and support; they do not establish that closure caused every reported condition.

What can and cannot be concluded

Evidence grading of Phase Three course practices

These ratings evaluate the research support for each design feature, not the quality of an instructor or the measured success of a course. Phase Three found no public causal outcome evaluation for any sampled course.

Interpretation of the course ratings

• Stat 20 and Stat 133 receive the strongest ratings because their policies combine boundaries with independent assessment. The rating concerns design logic, not published course outcomes.
• Writing and humanities restrictions have credible theoretical support, but direct long-term experiments comparing specific AI rules remain limited.
• Permissive AI use can be educationally defensible when students already have foundations and must explain decisions. The evidence is weak when the final product is the only proof of competence.
• Courses centered on citizenship or flourishing address important human purposes, but participation, enthusiasm, and a successful event are not validated measures of judgment or wellness.

Application across California institutions

The same evidence standard can be used across California public schools, community colleges, CSU and UC campuses, private universities, independent schools, and home-based programs. It should not assume that the same rule fits every institution. Age, subject, student preparation, disability services, language, family role, residential life, and available human support can change both the benefit and the risk.

For Thomas Aquinas College and Mother of Divine Grace, the next institutional comparison should examine whether close reading, memorization, written composition, seminar or oral discussion, tutorial feedback, family participation, and technology limits are documented in current practice. Their stated educational philosophies are relevant, but they should be classified as institutional claims until implementation and outcomes are verified. Public and private institutions should be held to the same distinction.

Research program that should be conducted

A useful investigation should compare specific educational designs rather than ask whether AI is good or bad. It should follow students long enough to distinguish assisted performance from independent capacity.

Core comparison

Minimum outcome set

• Immediate task performance with AI access and an equivalent assessment without AI.
• Delayed retention after at least several weeks and transfer to a new problem or text.
• Quality of reasoning, source verification, error detection, originality, and oral explanation.
• Time on task, help-seeking, confidence calibration, workload, and academic-integrity incidents.
• Sustained attention, sleep, distress, belonging, loneliness, peer and instructor interaction, and family support.
• Effects by prior achievement, age, disability, language, income, school type, and COVID disruption history.

Evidence controls

• Record the exact tool, model version, system instructions, permitted task, and exposure time.
• Use a baseline before AI access and common assessments across comparison groups.
• Separate self-report from observed behavior and validated outcomes.
• Predefine the main outcomes and analysis; report null and harmful findings as well as gains.
• Protect student privacy and provide human review before any integrity or wellness consequence.
• Repeat the study across disciplines, grade levels, and institutions before making systemwide claims.

Decision rule for schools and universities

An institution should expand an AI practice only when students improve on an independent measure of the intended capacity, the benefit persists after assistance is removed, serious subgroup harms are not concealed by the average, and the design preserves accountable human relationships. If only speed, satisfaction, or assisted product quality improves, the institution should describe an efficiency gain rather than a learning gain.

Conclusion and next investigation

The evidence supports AI as a bounded educational aid, especially when it provides hints, feedback, or support to a human educator. It also supports protecting retrieval, first attempts, direct reading, original composition, and unaided demonstrations of understanding. The evidence does not yet show how sustained AI use affects a well-formed mind, student wellness, or human relationships over years.

The next phase should integrate the institutional profiles from Phases One through Three with this evidence standard. It should score each California institution on stated purpose, documented practice, independent verification, measured learning, mental fitness and wellness, family and social context, COVID comparison, and remaining evidence gaps. Thomas Aquinas College and Mother of Divine Grace should be included explicitly.

Sources

Research and official data accessed September 12, 2026. Links lead to the source or official record.

• 1 Stanford SCALE The Evidence Base on AI in K–12 A 2026 Review
• 2 Bastani and colleagues Generative AI without guardrails can harm learning PNAS 2025
• 3 Kestin and colleagues AI tutoring outperforms in-class active learning Scientific Reports 2025
• 4 Wang and colleagues Tutor CoPilot EdWorkingPaper 2025
• 5 Risko and Gilbert Cognitive Offloading Trends in Cognitive Sciences 2016
• 6 Sparrow, Liu, and Wegner Google Effects on Memory Science 2011
• 7 Karpicke and Roediger The Critical Importance of Retrieval for Learning Science 2008
• 8 Kosmyna and colleagues Your Brain on ChatGPT MIT Media Lab preprint 2025
• 9 Liang and colleagues GPT detectors are biased against non-native English writers Patterns 2023
• 10 CDC Youth Risk Behavior Survey Data Summary and Trends Report 2023
• 11 National Center for Education Statistics 2024 NAEP Mathematics
• 12 Education Scorecard 2025 National Findings
• 13 CDC Youth mental health during the COVID-19 pandemic 2021