Earning trust when AI is making decisions about you
This article is part of a three-part series examining how AI is reshaping trust between faculty, students, and the institutions governing them. Reading the full series is recommended.
Part 1: The Detection Default | Part 3: The Collision Point
Maya Chen is a first-year student in Elena Marsh’s composition course, and two weeks into the semester she has already sensed a contradiction she can’t quite put into words. The program she is enrolled in requires every first-year writing student to run their essay outline through the university’s approved AI tool before drafting; not because Elena asked for it, but because the college wants to show it’s “building AI literacy.” Maya does what the assignment asks. She types a prompt into the tool, screenshots the output for the completion credit, and writes her actual essay the way she always has. She has learned nothing about the tool, but then, that was never really the point of the exercise.
Then she submits her final draft, and it passes through the university’s writing-assessment pipeline, which is mix of an AI-assisted feedback tool and Elena’s own read. It comes back marked down half a grade for “irregular phrasing,” with no further explanation. Maya sits with that for a while. She wasn’t allowed to use AI to help write the essay; using it that way would have been an integrity violation. But something used AI to help judge it. She has no way of knowing which parts of her essay created the flag, whether a person looked closely at those sentences before the grade was finalized, or who she’d even ask to get more clarification. The tool that graded her didn’t have to explain itself, but she is required to do so in every essay. It just does not feel like a fair trade to Maya.
The Silent Loop
Call it the silent loop. The growing set of decisions that touch a student’s academic life; a grade, a feedback comment, a flag on their advising file, a nudge to switch majors, that are increasingly made or shaped by AI, all without the student being told when or how. No one set out to hide anything. Disclosure was never built into the system in the first place, and it’s nobody’s job is to notice the gap.
The loop is already running in places most students never see. Early-alert systems score retention risk. Advising platforms surface nudges. Assessment tools flag phrasing. Most of these are genuinely well-intentioned programs, and some of them demonstrably work. But in April 2026, the National Student Legal Defense Network published a Student AI Bill of Rights whose first article asserts that students have a right to know “when, where, and how AI systems are being used to evaluate them, track them or make decisions about their educational future” (National Student Legal Defense Network, 2026). Nobody writes that sentence unless the current answer is no. The tools work; whether the student knows they’re running is a separate question, and mostly an unasked one.
What Students Already Know… and What They Don’t
Here’s the part that should recalibrate how institutions think about this; students are not the passive party in the AI story. A 2026 Digital Education Council survey that found 77% of faculty now use AI in teaching, also found 88% of students already using it in their own learning (Digital Education Council, 2026). This generation does not need to be introduced to technology. They are more fluent in it than most of the adults setting policy for them. Students know this, and that adds fuel to the fire. Maya isn’t confused about what AI can do, she’s frustrated that the institution gets to use it without the same disclosure it demands from her.
Research on AI-assisted grading backs up the instinct behind that frustration. A small study looked at 27 undergraduate computer science students grading a programming project, not an essay, but it asked the same underlying question: do students trust AI feedback as much as they trust a human’s? Even when the AI’s scores and clarity ratings matched or exceeded a human teaching assistant’s, most students still preferred the human. Sixty percent of students rated the TA’s feedback as fairer, and 55% said they trusted it more overall (Riahi, Storozhevykh & Catete, 2026). They chose the grader who gave them worse marks and murkier explanations.
Students Want the Why
Students consistently pointed to the same specific frustration Maya has; the AI could tell them what was wrong, but not why it mattered, or what to do next. This is the kind of contextual judgment that comes from an instructor who knows where a particular student is in their development. That same complaint showed up, almost word for word, in an account heard directly while researching this piece. A parent said her daughter’s high school teacher admitted that an AI tool couldn’t grade this particular ninth-grader accurately; it couldn’t tell what the student already knew versus what she still didn’t. It could grade the words on the page, but not the student behind them.
Jisc’s 2025 survey of student perceptions of AI found the same pattern. Students want clear institutional guidance on AI use and consistently say they value personalized, human feedback over automated alternatives. This is not because the automation is inaccurate, but because it can’t yet account for who they specifically are (Jisc, 2025).
Why the Trust Gap Is Actually a Retention Problem
It would be easy to file this under ethics and move on, but that undersells what’s actually at risk. Students who feel monitored or misjudged by systems they don’t fully understand rarely file a complaint; they disengage instead. Students are not measuring an institution’s AI governance against another university’s policy manual. They’re measuring it against every other digital experience they have every day, from their banking app to their streaming service. Held to that standard, an AI decision that feels opaque, inconsistent, or impossible to question doesn’t just cost the tool their confidence; it costs the institution behind it.
The same can be said for faculty not enforcing bans they don’t believe in. A flagged essay with no explanation costs Maya half a letter grade, and it teaches her that the system’s judgments about her are unappealable. This lesson generalizes fast to other areas such as advising nudges, degree-progress flags, and every other place AI touches her file. Gen Z and Gen Alpha students are, by every available measure, more AI-literate and more skeptical of unclear automated decisions than most institutional AI rollouts assume. An institution that treats that skepticism as a communications problem rather than a design problem will keep manufacturing the mistrust it’s trying to avoid.
What Human Agency Actually Requires
“Human-centered AI” has become a phrase institutions attach to almost anything. For a student, it means three concrete things.
The first is disclosure. A student should always be able to find out, without having to ask directly, when AI shaped a decision about them, such as a grade, a flag, a recommendation. The second is a working path to a human. Not a hidden one, not one that requires escalating through multiple offices, but an accessible route to someone with the authority to actually look again. The third is explainability calibrated to the stakes. A scheduling suggestion doesn’t need the same depth of explanation as a probation flag or a grade that affects a scholarship. Treating every AI touchpoint with the same disclosure process either buries the important ones in noise or makes the whole system too heavy to use. The standard should scale with what the student stands to lose.
The first two of those are already written down. The Student AI Bill of Rights asks for disclosure, and it asks that “automated systems should not be the final arbiter of high-stakes decisions affecting a student’s admission, academic standing, financial stability or other aspects of fundamental well-being” (National Student Legal Defense Network, 2026). Which is to say the standard being proposed here is not a radical one, and institutions will not get to claim they were never told.
None of this asks institutions to slow down AI adoption, rather it asks them to build the disclosure and recourse in from the start. This is generally the way a well-governed system is designed with an audit trail from day one rather than bolted on after something goes wrong.
From far away, it can look like teachers wanting control over their own work and students wanting to be trusted are two completely separate issues; they aren’t. The exact moment Elena Marsh’s grading tool flags Maya’s essay is the same moment both stories collide; one instructor exercising legitimate professional judgment, one student on the receiving end of a decision she never saw coming.
That collision is discussed in Part 3 of this series. Read it now.
Punch List
| Action | Owner | Timeframe |
| Add a disclosure standard to every AI-touched academic decision: what tool was involved, and what it means for the student | Provost’s Office / Registrar | This academic year |
| Build a visible, low-friction path to human review for any AI-shaped grade, flag, or recommendation — not a buried appeals process | Academic Affairs | This term |
| Calibrate explanation depth to stakes: light-touch for scheduling nudges, full explanation and human sign-off for grades, flags, or probation decisions | Academic Affairs / Advising | This academic year |
| Audit existing predictive and advising tools (early-alert, degree-progress, recommendation systems) for whether students are ever told they’re in use | IT / Institutional Research | Next two quarters |
| Include student government or student affairs representation in any AI-in-decisions policy discussion, not just IT and faculty governance | Student Affairs | Ongoing |
Note: Maya Chen is a composite illustration informed by parent- and educator-reported experience with mandated AI tools and AI-assisted grading, not a named individual. The high school teacher’s account referenced above is a first-hand anecdote relayed to us during research for this piece, not a published or independently verified source — it’s included as illustrative color, not as data.
Talk to Robots & Pencils about designing agentic AI for education. Request an AI Briefing.
Lindsay Pineda is a Senior Delivery Manager at Robots & Pencils, where she leads delivery of an AI-powered student intervention platform for a major public research university. With over 20 years of experience spanning higher education, educational technology, and program and delivery management, she has held leadership roles at a range of organizations across the higher education and edtech sectors. Lindsay spent nearly a decade as an adjunct graduate faculty member at a large online university facilitating master’s level courses in project management leadership and PMP exam preparation while contributing to curriculum and instructional design. A PMP-certified leader with master’s degrees in psychology and management, she brings a rare blend of strategic delivery expertise and firsthand experience in online course facilitation and the student learning experience.
FAQs
Q: What is “the silent loop”?
A: The growing set of decisions touching a student’s academic life, a grade, a feedback flag, an advising nudge, a major-change suggestion, that AI increasingly shapes without the student being told when or how it happened.
Q: Do students actually trust AI-generated feedback?
A: Not as much as human feedback, even when it’s just as good. A study of 27 undergraduate computer science students found that even when AI feedback matched or exceeded a human TA’s accuracy, 60% still rated the TA’s feedback as fairer and 55% trusted it more (Riahi, Storozhevykh & Catete, 2026).
Q: Isn’t this generation comfortable with AI making decisions about them?
A: They’re comfortable using AI, not comfortable with asymmetry. 88% of students already use AI in their own learning (Digital Education Council, 2026), which makes them more attuned to, not less bothered by, an institution using AI on them without the same disclosure it demands from them.
Q: What does the Student AI Bill of Rights actually require?
A: Published by the National Student Legal Defense Network in April 2026, its first article states students have a right to know when AI is evaluating, tracking, or deciding their educational future, and that automated systems shouldn’t be the final arbiter of high-stakes decisions.
Q: Why treat this as a retention issue instead of an ethics issue?
A: Students don’t file complaints when they feel misjudged by an opaque system, they disengage. They measure institutional AI against their banking app or streaming service, not against a policy manual, and an unexplainable decision costs the institution their confidence.
Q: What are the three things human agency actually requires?
A: Disclosure (knowing when AI shaped a decision), a working path to a human who can look again, and explainability calibrated to stakes, a scheduling nudge needs less explanation than a probation flag or scholarship-affecting grade.
Key Takeaways
- The silent loop is AI shaping grades, advising nudges, and early-alert flags with no disclosure built in, not by malice, but because no one’s job is to notice the gap.
- Students already outpace institutions in AI fluency (88% vs. 77% of faculty), which sharpens rather than dulls their frustration at asymmetric disclosure.
- Even when AI feedback is equally or more accurate, students trust it less: 60% rated human TA feedback as fairer in a 2026 study.
- The Student AI Bill of Rights (NSLDN, April 2026) already sets the standard: disclosure, and no automated system as final arbiter of high-stakes decisions.
- Human agency requires three things scaled to stakes: disclosure, a real path to human review, and explanation depth proportional to what’s on the line.
