Phase Fourteen Educator Tabletop Review

California Education and AI Educator Tabletop Review

Phase Fourteen Unseen calibration exercises and readiness decision package

Package date September 13 2026 Informational review design No outreach or student participation

Phase Fourteen purpose

This package moves the Phase Twelve task packets and Phase Thirteen dry run into a controlled educator review. It supplies new synthetic examples, blank scoring records, separate adjudication guidance, boundary challenges, workload fields, and a final decision form. Educators can use it to determine whether the instructions are clear enough to enter the appropriate local review process.

The package does not record a completed educator review. No educator, student, course, or institution participated in its preparation. The adjudication bands are proposed reference points for calibration, not verified correct scores or evidence about Thomas Aquinas College, Berkeley Law, or any other institution.

Package sections

Standing boundary

K through 12 student facing artificial intelligence conditions remain paused until operative policy, age, consent, parent authority, privacy, accessibility, and developmental safeguards are verified. This tabletop package does not alter that boundary.

Tabletop review session plan

Suggested one hundred twenty minute session

Stop authority

Any reviewer may stop the exercise if the answer key was opened early, scoring was discussed before independent completion, a real person's information appears, or a participant cannot distinguish the target reasoning from writing style, confidence, disability, or background.

Separation and evidence control protocol

Before the session

1. Give each rater a copy containing the exercise pages and blank score sheets only.

2. Keep the Packet A and Packet B adjudication pages with the facilitator.

3. Assign anonymous rater codes and record the packet version.

4. Confirm that all examples are synthetic or previously cleared and contain no real student or client data.

5. Register the exact scoring dimensions, missing evidence rule, and agreement calculation before scoring.

During independent scoring

• Raters do not discuss the response, infer the intended band, or compare totals.
• Each score cites a phrase, reasoning step, omission, or oral record that supports it.
• A dash records unavailable evidence. It is excluded from agreement calculations.
• Fluency, confidence, vocabulary, speed, and conformity to a preferred conclusion do not substitute for the registered dimension.

After independent scoring

1. The recorder enters both scores without changing either original sheet.

2. The facilitator calculates exact agreement and agreement within one point.

3. The adjudication key is opened only after the initial figures are recorded.

4. Raters identify whether each disagreement arose from the task, anchor, example, or rater application.

5. A new unseen set must be scored after any substantive clarification. The training set cannot prove reliability.

Version record

Packet A unseen formation exercise

Synthetic source The Village Well Dialogue

A resident says the village well is safe because families have used it for fifty years. An inspector replies that long use matters, but safety depends on what entered the water, when it was tested, and whether the test could detect the relevant danger. The resident objects that waiting for perfect evidence would close every well. The inspector answers that action does not require certainty, but responsible judgment must connect the decision to the best available evidence, state the remaining risk, and identify what result would require a change.

Registered task

State the dialogue's central claim about responsible judgment. Identify the strongest evidence for continued use, explain the strongest objection to that evidence, answer the objection without demanding certainty, and name one result that should change the decision. Distinguish the history of use from direct evidence about present safety.

Scoring dimensions

Instruction to raters

Score each written response from 0 through 4 on the first five dimensions. Do not score transfer or oral ownership. Do not open the Packet A adjudication key until both score sheets are complete and the initial agreement record is locked.

Packet A unseen responses

Response A1

The well is safe because people have used it for a long time. The inspector wants more testing, but no one can ever be completely certain. It should stay open unless people become sick.

Response A2

The dialogue says a decision should use the best evidence available instead of waiting for certainty. Fifty years of use is some evidence because obvious harm may have appeared, but it does not show whether a new contaminant exists or whether anyone looked for it. Closing the well may also harm the village. A current test designed for the suspected danger could justify continued use, while a positive result should change the decision.

Response A3

Responsible judgment is conditional action tied to relevant evidence and a stated revision rule. The well's history lowers concern only for harms that ordinary experience would reveal; it says little about a recent or invisible contaminant. The strongest objection is that broad testing and closure can impose costs before a danger is proven. The answer is proportionate action: test for the plausible hazard, describe what remains unknown, and limit or continue use according to the result. A verified level above the registered safety limit would require a change.

Response A4

The epistemological tension concerns induction, risk, and prudential action. Tradition provides a socially embedded prior, whereas inspection offers instrumental rationality. Both perspectives contain value, so leaders should seek balance, dialogue, and community trust before acting.

Rater caution

Response A4 is intentionally polished. Score only the registered evidence. Do not reward specialized vocabulary when the response omits the decisive present evidence, serious objection, answer, or revision condition.

Packet A independent score sheet

Evidence citations

Independent completion record

Packet A adjudication key

Open this page only after both independent Packet A score sheets and the initial agreement figures are recorded. The bands are provisional calibration references, not verified student standards. A rater may depart from a band only by citing stronger evidence under the registered anchor.

Adjudication questions

1. Did either rater score the conclusion instead of the reasoning that supports it?

2. Did either rater reward fluency, vocabulary, confidence, or agreement with the expected conclusion?

3. Did the evidence selection score distinguish long use from direct testing?

4. Did the objection score require both a fair objection and a relevant answer?

5. Did correction and humility require a specific condition that would change the judgment?

Packet A revision record

Packet B unseen legal exercise

Fictional Campus Learning Records Rule

Fictional fact pattern

Professor Arden keeps a file containing student Leo Park's tutoring notes, disability accommodation, grades, and a message saying he is exhausted before final examinations. Leo signed a consent eighteen months ago allowing tutoring summaries to be sent to his academic advisor. The professor uploads the entire file to an approved writing vendor using the school account and instructs it to identify learning weaknesses. The vendor contract permits school directed processing but forbids model training. The professor also sends a summary labeled LP to a parent association, including Leo's rare major, hometown, accommodation, and exact grades. The professor says both actions are justified by consent, deidentification, and concern for Leo's safety.

Registered task

Using only Rules 11a through 11e, analyze the vendor processing and the parent association disclosure separately. Address consent, scope, expiration, emergency, deidentification, data minimization under the stated rules, and the fact most likely to change each conclusion.

The authority and facts are fictional. The exercise is not legal advice and contains no real person or record.

Packet B unseen responses

Response B1

The professor can use the vendor because it is approved. The parent association received only initials, so the summary is deidentified. The old consent and safety concern also support both disclosures.

Response B2

The consent expired and named only the advisor, so it does not authorize either recipient. The vendor may process records under Rule 11e, but the professor should have sent only tutoring material because the task was to identify learning weaknesses. The parent summary is not deidentified if the rare major, hometown, accommodation, and grades allow recognition. Exhaustion alone does not establish an imminent specific threat. A current consent or facts showing such a threat could change the analysis.

Response B3

Rule 11e supplies a processing basis for the approved vendor if the school instruction stays within the contract and the vendor does not train on the record. The upload nevertheless includes material unrelated to the stated learning task, which weakens the claimed scope. The parent association is not covered by Rule 11e or the expired advisor consent. Initials do not prevent reidentification when the combined details point to Leo. Rule 11c also fails on the stated facts because no imminent specific threat is identified and the association is not shown able to reduce one. Narrower vendor input and a current recipient specific consent would materially change the conclusions.

Response B4

Educational innovation requires flexibility. The professor acted compassionately and used an approved system, while the parent association serves the community. Privacy should be balanced against safety and educational improvement.

Instruction to raters

Score issue recognition, rule and authority, application, counterargument, verification, judgment, and authorship from 0 through 4. Oral ownership and transfer are unavailable and must remain unscored.

Packet B independent score sheet

Evidence citations

Independent completion record

Packet B adjudication key

Open this page only after both independent Packet B score sheets and the initial agreement figures are recorded. The expected bands test use of the registered fictional authority and do not establish a law school grading standard.

Adjudication questions

1. Did the rater analyze the vendor and parent association as separate recipients and legal bases?

2. Did rule and authority scoring require accurate use of the supplied provisions rather than citation form alone?

3. Did verification scoring require checking every attributed rule and fact against the closed packet?

4. Did judgment scoring connect the degree of certainty to the evidence instead of rewarding caution or confidence?

5. Did authorship remain unscored where no protected explanation established personal ownership?

Packet B revision record

Bounded critique challenge cards

Each rater marks Permit, Reject and continue, or Stop. The course rule and approved tool are assumed only for the synthetic tabletop exercise. In a real setting, missing authorization requires H0 human only.

Bounded critique facilitator key

Safeguard and feasibility review

Actual session workload

Official sources Student privacy and education technology | HHS research protections | Berkeley appropriate use guidance

Calibration calculation and interpretation

Score comparison record

Calculations

Exact agreement Exact agreements divided by comparable scores multiplied by 100 Result __________ percent

Within one point Exact agreements plus one point differences divided by comparable scores multiplied by 100 Result __________ percent

Critical rule agreement Correct verification errors, boundary violations, and missing evidence decisions divided by applicable decisions multiplied by 100 Result __________ percent

Proposed readiness thresholds

The proposed thresholds are operating rules for this investigation. They are not published institutional standards. Failure requires clarification and a new unseen scoring set; it does not justify deleting difficult examples or averaging unresolved scores.

Official sources What Works Clearinghouse standards and protocols

Tabletop readiness decision

Required decision

Decision record

Phase Fourteen evidence record and handoff

Evidence status

Phase Fifteen task

Phase Fifteen should convert the completed educator tabletop records into a final preimplementation determination. It should preserve both original score sheets, calculate agreement, document every revision, verify the course rule and approved account, and decide whether to authorize only a human benchmark, authorize a bounded higher education comparison, or pause.

Limits on expansion

• A tabletop pass permits formal local review; it does not authorize student participation.
• A human only benchmark must use an existing educational activity and preserve ordinary support and accommodations.
• A bounded artificial intelligence condition requires express course authorization, an approved account, a frozen human first record, a verified equivalent alternative, and protected C2 through C4 checks.
• No institutional score, ranking, wellness claim, learning claim, or causal conclusion may be drawn from the tabletop materials.
• K through 12 student facing artificial intelligence remains paused under the existing developmental safeguard rule.

Official sources Berkeley Law artificial intelligence policy | Berkeley teaching guidance | Berkeley licensed artificial intelligence tools | Thomas Aquinas College California Code of Conduct