The education world is stuck in a binary.
Either AI is a systemic evil that needs locking down, or it is the silver bullet we must swallow whole to keep up. Both extremes are wrong. They miss the actual risk: we are rushing to integrate AI into schooling without asking what happens when it gets it wrong.
Ercikan and Solano-FFlores (2025 put it bluntly:
“Rapid adoption of innovative sources… may compromise score meaning interpretation.”
Treating “educational AI” like one thing is a mistake. A spell checker and a graduation screener might share code, but they don’t share ethics, legality, or purpose. An adaptive dashboard and an automated gatekeeper are not the same beast.
The upcoming draft of the Standards for Educational and psychological testing is a chance to fix this. It is generational. We need to stop looking at the novelty of the code and start looking at the weight of its decisions. Kane (2016 was right to call it irresponsible to make decisions without considering consequences.
This is personal for me. I have two school-age kids. They learn differently.
When I see an educator who truly knows them, I feel relief. It is palpable. I trust that human connection. But I am fierce against assumptions. I don’t care if the assumption is born from data or bias, the damage is real.
So let’s get practical. Let an algorithm help my kid practice math. If the code notices they are stuck on fractions and loops them back to a precursor skill? Fine. It is reasonable. Reversible.
But if an opaque algorithm decides my child gets retained? If it tracks them out of Algebra? If it denies special services or blocks graduation? I object. Hard. A hint is not a Diploma.
Let the Algorithm Hold the Flashlight
How do we sort this out? Think about the Netflix versus Mortgage test.
We love Spotify. We trust Google Maps. If the playlist is off or the route is bad, it’s annoying. It is reversible. But we don’t let a black-box algorithm deny our mortgages. Or screen our resumes. Or triage our ER visits. Those decisions alter life chances. There is no “undo” button on redlining.
Convenience justifies automation. Consequence justifies safeguards.
Schools contain both ends of this spectrum under one roof. This means we need a consequence-calibrated framework, not a blanket ban or a wild west free-for-all.
Here is how it breaks down:
-
Tier 1: Low Stakes.
This is assistive and reversible. AI provides practice items. Adaptive hints. Formative feedback. The cost of error here is negligible because it is advisory. If the AI gives bad feedback, a teacher sees it and corrects it. Let innovation thrive here. A suggestion engine is not a judge. -
Tier 2: Moderate Stakes.
This is decision support. Placement recommendations. Dropout dashboards. Intervention referrals. These don’t have final legal say, but they shape reality. They influence teacher expectations. They direct resources. They create self-fulfilling paths. These need monitoring. Documented use cases. Bias checks for subgroups. And human review. Teachers should treat algorithmic alerts as hypotheses, not verdicts. -
Tier 3: High Stakes.
The gatekeepers. Promotion. Graduation eligibility. Special Ed identification. College admissions. Scholarships. Automated proctoring accusations.
Errors here are devastating. They alter trajectories without easy recourse.
Governance at Tier 3 demands maximum scrutiny. You need validity evidence tied directly to that specific decision. You need subgroup fairness audits. You need meaningful human appeal options. Bennett, LaMar, and Mazzeto (2025 say our validity arguments must ramp up as consequences rise.
This isn’t a radical new idea. Educational measurement has had this DNA. The Standards already say inferences are use-relative. Samuel Messick (1989) and Kane (2016 established this: evidence validating a low-stake hint doesn’t validate a high-stake graduation decision. Lane and Marion (2015 note that validity refers to decisions, test uses, and their consequences.
Fairness is core. It’s not a supplement.
Verify, Don’t Trust
We can build a blueprint. Merge modern AI law (NIST Framework, EU AI Act) with psychometrics.
“High-Risk Context” means assessment stakes.
“Explainable AI” means clear score reporting.
“Bias Audits” means subgroup-bias analyses.
The catch is oversight. We need to avoid ceremonial oversight. The European Data protection Supervisor calls it out: real oversight improves decisions. It isn’t a symbolic gesture. It isn’t an educator rubber-stamping a machine output because they are tired and rushed. That is automation bias in disguise.
Lane and Marion warn that mismatches between intended and enacted uses cause unintended social and personal consequences.
Real review takes time. Context. And authority to say “No.”
“We can’t take the recommendation of a machine on trust… they have to be verified.” — Dragoș Tudorache
This is the only way to protect learners and keep innovation alive. By putting heavy regulatory weight on Tier 3, we free up Tier 1 to experiment. We protect human potential.
AI can personalize learning. It can engage. It is powerful. But it should hold the flashlight. The gavel stays with us.
The line is clear, but where you stand on it defines the future of equity. Are you building a system that supports learners or one that processes them?
References
