Verification and pilot materials · Complete web edition

Phase Sixteen Local Review Execution Binder

This is the complete web edition of the canonical project document. Its narrative, recommendations, evidence tables and source links are presented below without abridgment.

2,292words

22tables

71text blocks

Download the Word document

Project, Research, Analysis, Editorial by Patrick Russell; assisted by ChatGPT (OpenAI).

Published as a web edition September 13, 2026

California Education and AI Local Review Execution Binder

Phase Sixteen Controlled educator session and evidence handoff

Binder date September 13 2026 Informational execution guide No outreach or student participation

Session field	Entry
Responsible person	________________________________________
Organization or office	________________________________________
Session date and location	________________________________________
Binder version	________________________________________
Final status	________________________________________

Purpose and current boundary

This binder organizes the Phase Fourteen tabletop package and the Phase Fifteen preimplementation determination into one controlled local review. It assigns materials by role, protects independent scoring, records key release and adjudication, verifies local safeguards, and transfers the completed record to the proper decision authority.

The binder does not record a completed review. No educator, student, course, or institution participated in its preparation. Student implementation remains paused. A local session may test whether the materials are understandable and governable, but it cannot prove educational benefit or authorize a student condition by itself.

Required outputs

Output	Minimum record	Owner
Independent scoring	Two original Packet A and Packet B score sets with evidence notes	Raters
Control record	Copy log, timing, key release, deviations, and stop events	Facilitator and recorder
Calibration result	Initial and post revision agreement figures	Recorder and adjudicator
Safeguard result	Evidence for every authority, privacy, accessibility, account, and workload gate	Policy reviewer
Determination	Authorize human-only benchmark review, revise, or pause	Named local authority

Session launch record

Required roles

Role	Responsibility	Assigned person
Facilitator	Protects sequence, timing, copy control, key release, and stop authority	________________________
Rater one	Scores independently and cites evidence for every scored dimension	________________________
Rater two	Scores independently without seeing Rater one or the keys	________________________
Adjudicator	Applies anchors after scoring and records unresolved questions	________________________
Policy reviewer	Verifies authority, privacy, accessibility, approved account, and equivalent route	________________________
Recorder	Preserves originals, calculates agreement, records revisions, and assembles the handoff	________________________
Decision authority	Selects the Phase Fifteen route after all records are complete	________________________

Launch checklist

Launch condition	Check	Evidence
All roles assigned and conflicts disclosed	____	____________________________
Phase Fourteen final copy is the 17 page reviewed version	____	____________________________
Phase Fifteen final copy is the 12 page reviewed version	____	____________________________
Rater copies exclude pages containing keys	____	____________________________
Examples contain no real student or client information	____	____________________________
Scoring dimensions and missing evidence rule are fixed	____	____________________________
Stop authority and record location are understood	____	____________________________
No student participation or tool account is required for the tabletop	____	____________________________

Materials distribution and copy control

The page assignments below refer to the visually reviewed 17 page Phase Fourteen document. The facilitator must confirm pagination if that source file is changed. Phase Fifteen remains with the facilitator, policy reviewer, recorder, and decision authority until the independent scoring record is locked.

Recipient	Phase Fourteen pages	Purpose	Excluded material
Rater one	4 through 6 8 through 10 12	Exercises, responses, score sheets, and challenge cards	Pages 7 11 13 keys
Rater two	4 through 6 8 through 10 12	Separate identical rater packet	Pages 7 11 13 keys
Facilitator	1 through 17	Controls complete package and release sequence	None
Adjudicator	7 11 13 after release	Uses keys only after initial figures are locked	Rater identities where unnecessary
Policy reviewer	3 14 17	Reviews evidence control, safeguards, and limits	Exercise keys unless also adjudicator
Recorder	3 14 through 16	Preserves control, workload, calibration, and decision records	Keys before recorded release
Decision authority	14 through 17 after session	Reviews completed records with Phase Fifteen	Unnecessary rater identity

Copy issue log

Copy code	Recipient code	Pages issued	Issue time	Return time	Complete
________	________	____________	________	________	Yes No
________	________	____________	________	________	Yes No
________	________	____________	________	________	Yes No
________	________	____________	________	________	Yes No
________	________	____________	________	________	Yes No

Copy discrepancy rule

Stop the session if a rater receives a key, the two rater packets differ, a page is missing, or any issued copy cannot be accounted for. Record the discrepancy before deciding whether a new unseen set is required.

Facilitator session script

Minutes	Facilitator action	Required record
0 to 10	Confirm scope, roles, copies, no real data, and stop authority	Signed launch record
10 to 25	Review shared anchors without displaying answers or expected bands	Questions and clarifications
25 to 50	Distribute Packet A materials and begin silent independent scoring	Two locked Packet A sheets
50 to 52	Collect or secure both Packet A sheets and record initial agreement	Initial Packet A figures
52 to 65	Release Packet A key to the adjudicator and record disagreements	Packet A adjudication record
65 to 90	Distribute Packet B and challenge cards for silent independent scoring	Two locked Packet B sets
90 to 92	Secure both Packet B sets and record initial agreement	Initial Packet B figures
92 to 105	Release Packet B and challenge keys and record disagreements	Packet B adjudication record
105 to 112	Complete calibration, safeguard, workload, and deviation records	Gate summary
112 to 120	Select revise, ready for determination, or pause	Signed tabletop decision

Required opening statement

This session tests the clarity and governability of the review materials. Score only the registered evidence. Do not reward fluency, vocabulary, confidence, agreement with a preferred conclusion, or assumptions about a writer. Do not discuss scores until both independent records are secured. Any participant may stop the session when separation, privacy, accessibility, or evidence controls fail.

Clarification log

Time	Question	Answer before scoring	Changes task or anchor
______	____________________________	____________________________	Yes No
______	____________________________	____________________________	Yes No
______	____________________________	____________________________	Yes No
______	____________________________	____________________________	Yes No

Independent scoring and key release

Independent scoring control

Control	Rater one	Rater two	Recorder evidence
Unique rater code assigned	Yes No	Yes No	____________________
Packet complete before start	Yes No	Yes No	____________________
No score discussion	Yes No	Yes No	____________________
Evidence note for every score	Yes No	Yes No	____________________
Unavailable evidence marked by dash	Yes No	Yes No	____________________
Finish time recorded	Yes No	Yes No	____________________
Original sheet secured	Yes No	Yes No	____________________

Key release log

Key	Release condition	Initial figures locked	Release time	Recipient
Packet A	Both Packet A sheets secured	Yes No	________	____________
Packet B	Both Packet B sheets secured	Yes No	________	____________
Challenge cards	Both challenge records secured	Yes No	________	____________

Premature access decision

Field	Entry
Premature access occurred	No Yes Time __________
Material accessed	____________________________________________
Affected records	____________________________________________
Decision	Continue unaffected record Replace with unseen set Stop
Basis and authority	____________________________________________

Adjudication and revision record

Adjudication begins only after the initial independent figures are locked. Its purpose is to identify why raters applied the registered instructions differently. It must not overwrite either original score sheet.

Item and dimension	Rater one	Rater two	Cause	Adjudicated interpretation
________________	____	____	Task Anchor Example Rater	________________________
________________	____	____	Task Anchor Example Rater	________________________
________________	____	____	Task Anchor Example Rater	________________________
________________	____	____	Task Anchor Example Rater	________________________
________________	____	____	Task Anchor Example Rater	________________________
________________	____	____	Task Anchor Example Rater	________________________

Revision log

Version	Exact change	Reason	Owner	New unseen set required
______	______________________	______________________	________	Yes No
______	______________________	______________________	________	Yes No
______	______________________	______________________	________	Yes No
______	______________________	______________________	________	Yes No

Post revision rule

Any change that affects the task, evidence anchor, score meaning, boundary, missing evidence rule, or safeguard requires a new unseen set. The examples that produced the revision become training material and cannot establish post revision agreement.

Local safeguard verification

Review area	Local evidence	Owner	Decision
Teaching authority	Operative rule and task purpose __________________	________	Pass Revise Stop
Research status	Instruction or research determination ______________	________	Pass Revise Stop
Approved product and account	Exact product account and configuration __________	________	Pass Revise Stop
Permitted information	Data classification and exclusions _________________	________	Pass Revise Stop
Retention and deletion	Outputs logs date owner and method _________________	________	Pass Revise Stop
Accessibility	Accommodations and assistive tools __________________	________	Pass Revise Stop
Equivalent route	Instruction time feedback credit and opportunity ______	________	Pass Revise Stop
Human first record	Freeze method and custodian _______________________	________	Pass Revise Stop
Protected checks	Oral transfer and delayed measures __________________	________	Pass Revise Stop
Workload	Time budget displaced work and reviewer capacity _________	________	Pass Revise Stop
Interpretation	Limits on ranking detection wellness and causality ______	________	Pass Revise Stop

Verification rule

The reviewer must cite the operative local document, decision, configuration, or accountable person. A public webpage, general statement, or product availability does not by itself establish course authorization, approved use, or lawful record handling. Any stop result blocks further implementation review until the proper authority resolves it in writing.

Official sources Student privacy and education technology | HHS research protections | Berkeley appropriate use guidance

Calibration workload and stop record

Calibration summary

Measure	Threshold	Initial result	Post revision result	Pass
Exact agreement	At least 80 percent	________	________	Yes No
Within one point	100 percent	________	________	Yes No
Two point differences	None unresolved	________	________	Yes No
Verification decisions	100 percent correct	________	________	Yes No
Boundary decisions	100 percent correct	________	________	Yes No
Missing evidence	100 percent correct	________	________	Yes No

Workload summary

Activity	Planned minutes	Actual minutes	Displaced work	Feasible
Preparation	______	______	________________	Yes No
Scoring	______	______	________________	Yes No
Adjudication	______	______	________________	Yes No
Safeguard review	______	______	________________	Yes No
Decision and follow up	______	______	________________	Yes No

Stop and incident log

Time	Condition	Action	Authority	Resolved
______	________________	________________	________	Yes No
______	________________	________________	________	Yes No
______	________________	________________	________	Yes No

Official sources What Works Clearinghouse standards and protocols

Phase Fifteen determination meeting

Gate summary

Gate	Result	Evidence location	Open condition
Original records complete	Pass Fail	________________	________________
Calibration thresholds met	Pass Fail	________________	________________
Required authorities verified	Pass Fail	________________	________________
Privacy and account verified	Pass Fail	________________	________________
Accessibility and alternative verified	Pass Fail	________________	________________
Workload feasible	Pass Fail	________________	________________
Protected measures registered	Pass Fail	________________	________________
No stop remains open	Pass Fail	________________	________________

Required decision

Decision	Select	Effect
Authorize human only benchmark review	____	Allows the named existing human-only activity to enter its applicable local review process.
Ready for bounded higher education determination	____	Allows the named higher-education condition to enter final local authorization. It does not itself authorize students.
Revise and repeat	____	Requires correction and a new unseen set before another determination.
Pause	____	Blocks further implementation work until every identified gap is resolved.

Decision record

Field	Entry
Selected decision	____________________________________________
Named activity and condition	____________________________________________
Evidence supporting decision	________________________________________________________________________________________
Required revision or open condition	________________________________________________________________________________________
Responsible person	____________________________________________
Approving authorities	____________________________________________
Decision and review dates	____________________________________________

Record assembly and custody

Final record order

Signed session launch record and role assignments.
Copy issue and return log.
Both original Packet A score sheets.
Initial Packet A agreement calculation and key release record.
Packet A adjudication and revision record.
Both original Packet B score sheets and challenge decisions.
Initial Packet B agreement calculation and key release record.
Packet B adjudication and revision record.
Any revised materials and the new unseen rescore set.
Safeguard, workload, stop, and incident records.
Completed Phase Fifteen gate summary and decision record.
Authority approvals or written unresolved conditions.

Custody record

Record group	Custodian	Location	Access	Retention or deletion
Original rater records	________	____________	____________	____________
Keys and training examples	________	____________	____________	____________
Calibration and revisions	________	____________	____________	____________
Safeguard evidence	________	____________	____________	____________
Decision record	________	____________	____________	____________
Incident record	________	____________	____________	____________

Privacy and identity rule

Use rater codes in analytic records unless identity is necessary for accountability. Store any identity crosswalk separately. Do not place real student or client information in the tabletop file. If a later authorized activity creates education records, the local authority must approve collection, access, reporting, retention, deletion, and incident handling before the activity begins.

Evidence status and handoff

What Phase Sixteen provides

Evidence layer	Phase Sixteen status
Claim	A complete local execution binder now connects the Phase Fourteen exercises to the Phase Fifteen determination.
Practice	The binder fixes role assignments, page distribution, copy control, key release, adjudication, safeguards, workload, decision, and custody records.
Immediate result	A responsible local team can conduct the tabletop without designing its own control process.
Durable result	None until human reviewers complete the session and a later authorized activity produces protected evidence.
Context	No outreach, educator participation, student participation, institutional approval, account use, or course implementation occurred.
Gap	Human calibration, local authority decisions, approved accounts, privacy and accessibility determinations, workload capacity, and student outcomes remain unverified.

Current decision

The investigation has reached the end of document preparation that can create new operational value without human review. Student implementation remains paused. The next evidentiary action is the controlled educator tabletop session, followed by the Phase Fifteen determination meeting.

Handoff after the session

If calibration or safeguards fail, preserve the pause and create a new unseen set after documented revision.
If only the human-only route passes, prepare a narrow local benchmark observation plan through the applicable approval process.
If every higher-education gate passes, prepare a condition-specific authorization and preregistered comparison plan for final local approval.
Do not proceed to a K through 12 student-facing artificial intelligence condition under this binder.
Do not publish institutional rankings, misconduct conclusions, wellness diagnoses, learning claims, or causal findings from the tabletop record.

Limits on further continuation

Further desk-generated phases should not be treated as progress toward implementation evidence. The missing evidence now belongs to human application of the controls and to the appropriate local authorities. Any later document should use the completed records, preserve unsuccessful and missing results, and state exactly which route was authorized or paused.

Official sources Berkeley Law artificial intelligence policy | Berkeley teaching guidance | Thomas Aquinas College California Code of Conduct