Lecture 7 — Trust in AI (Grimmelikhuijsen + Liefooghe)

Paper: Grimmelikhuijsen, S., & Meijer, A. (2022). Legitimacy of algorithmic decision-making: Six threats and the need for a calibrated institutional response. Perspectives on Public Management and Governance, 5(3), 232–242.

Type: Thematic (orange). Where L5 looked at NLP as an investigative tool used by institutions, L6 flips the lens: when an institution uses an algorithm to make a decision about a citizen (parole, welfare, fraud detection, immigration), what does it take for that decision to be legitimate?

⚠️ Note: the slide deck for this lecture is not in the course folder, so this summary is built from the obligatory paper, the lecture description in the student manual, and general knowledge of the public-administration / trust-in-government literature. Treat as scaffold — verify specifics against the paper's text and any slides shared on Teams.

Lecture in one paragraph

Public organisations increasingly use algorithmic decision-making (ADM) for high-stakes decisions (welfare allocation, predictive policing, child-protection risk scoring, immigration triage, tax-fraud detection). These systems can be more efficient and more consistent than human bureaucrats, but they can also fail in ways that erode the legitimacy of the state itself — the Dutch toeslagenaffaire (childcare-benefits fraud-detection scandal) is the canonical case in this course. Grimmelikhuijsen & Meijer's paper isolates six threats ADM poses to institutional legitimacy and argues for a calibrated institutional response matched to the type and depth of each threat — rather than blanket pro- or anti-AI policies.

The lecture also covers the cognitive-science side (Liefooghe): what trust actually is, psychologically, and how humans learn to trust (or distrust) AI expertise — algorithmic appreciation vs. algorithmic aversion.

Key concept 1 — Trust: a psychological definition

Trust in an actor (human or AI) is conventionally decomposed as:

Competence / ability — Can it do the task?
Benevolence — Does it act in my interest?
Integrity — Does it follow norms / values I endorse?

For AI specifically, Mayer, Davis & Schoorman's classic trust model is adapted: people decide whether to rely on an algorithmic recommendation by combining a competence judgement (accuracy, robustness) with an integrity judgement (transparency, explainability, governance).

Two empirically well-documented attitudes:

Algorithm aversion (Dietvorst et al. 2015): after seeing an algorithm make even one mistake, people prefer human judgement — even when the algorithm is on average better.
Algorithm appreciation (Logg, Minson & Moore 2019): in novel tasks with no prior exposure, people often prefer algorithmic advice over an equivalent human's.

Whether you get aversion or appreciation depends on visibility of errors, domain familiarity, framing, and perceived delegation of agency.

Paper 6 — Grimmelikhuijsen & Meijer (2022): Legitimacy of ADM

The framing question

Public administration uses ADM more and more. When does that strengthen and when does that threaten democratic legitimacy? The paper draws on input / throughput / output legitimacy (a standard taxonomy in public-admin theory):

Input legitimacy — Who decided to deploy this algorithm? Were affected citizens represented?
Throughput legitimacy — How does the decision process work? Is it transparent, accountable, contestable?
Output legitimacy — Does it produce good outcomes (effective, fair)?

The six threats (the core of the paper)

ADM poses six distinct threats to legitimacy. The paper's argument is that each threat needs a different institutional response. (Names from the paper; the brief gloss is mine.)

#	Threat	Plain meaning
1	Reduced expertise / deskilling	Frontline officials lose the discretionary judgement skills that gave their decisions democratic legitimacy in the first place.
2	Opacity / lack of transparency	Black-box systems mean citizens (and oversight bodies) cannot understand why a decision was made.
3	Bias and unequal treatment	Training data and proxy variables encode historical discrimination; protected groups bear the cost.
4	Privacy infringement	Wide-scale data integration to feed ADM corrodes privacy as a precondition of citizenship.
5	Reduced human oversight and accountability	Diffusion of responsibility: when an algorithm errs, no one is clearly accountable ("computer says no").
6	Erosion of public values	Subtle long-run shifts: speed and consistency get optimised at the expense of mercy, deliberation, and individualised judgement.

The argument: calibrated institutional response

Blanket positions ("ban all algorithms" or "let civil servants choose") are inadequate because the six threats demand different mitigations:

For opacity → mandate explainable / interpretable models for consequential decisions.
For bias → require pre-deployment audits, demographic disparity testing, ongoing bias monitoring.
For accountability → ensure a clear human-in-the-loop with the authority and the information to overrule.
For privacy → DPIAs (Data Protection Impact Assessments), minimisation, purpose limitation.
For deskilling → invest in keeping human officers' substantive judgement alive (training, periodic non-AI cases).
For public-value erosion → democratic deliberation about which values the algorithm is optimising for.

The paper's contribution is the catalogue and the map from threat-type to response-type — not a single magic bullet.

The Dutch toeslagenaffaire — the case that's almost certainly in the exam

(Childcare-benefits fraud-detection scandal, 2010s–2021)

The Belastingdienst (Dutch tax authority) used an algorithmic risk model to flag potentially fraudulent childcare-benefit claims.
Variables used included proxies for ethnicity (dual nationality, postal code) → systematic over-flagging of immigrant and dual-national families.
Flagged families were forced to repay tens of thousands of euros, often wrongly. Many were driven to bankruptcy. Children were taken into care.
The scandal led to the resignation of the entire Rutte III cabinet in January 2021.

Reading the six threats against this case:

Threat	How it manifested in toeslagenaffaire
Deskilling	Officials deferred to algorithm outputs rather than exercising judgement on individual circumstances.
Opacity	Affected families could not see why they had been flagged.
Bias	Algorithm used ethnic-proxy variables → systematic discrimination.
Privacy	Massive integration of administrative data without proportionality.
Accountability	Diffuse responsibility — no one official could be pinpointed; appeal processes were broken.
Public-value erosion	"Hard line on fraud" optimisation eroded values of fairness, family integrity, and proportionality.

This case is the paradigm IOS example of why algorithmic legitimacy is not academic — the legitimacy of an entire government depended on it.

Key concept 2 — Liefooghe's psychological side: how do humans learn to trust AI?

(Sketch — paper-only, no slides.)

The cognitive-psych side of the lecture likely covers:

Mental models of AI expertise: people anthropomorphise AI ("the algorithm knows X") and over- or under-attribute competence.
Calibration of trust to capability: trust should track actual model performance; in practice it tracks recent salient outcomes (one error can collapse trust; one success can inflate it).
Identity / source effects: same recommendation labelled "by AI" vs. "by an expert" lands differently — and the difference is domain-dependent (people accept AI for objective domains, resist it for moral/subjective ones).
Transparency vs. understandability: showing the user a SHAP plot is transparent but not understandable; only when explanation matches the user's mental model does trust calibrate properly.
Procedural trust: in public-admin contexts, perceived procedural fairness of the decision pathway matters as much as the decision itself.

Why this matters for an open society

L6 sits squarely on the Democracy & Good Governance pillar, with strong links to Equity & Diversity:

Algorithmic decisions by state agencies are how the constitutional democracy & rule of law element of the Open Society definition is increasingly mediated. If those decisions are illegitimate, the rule of law itself is degraded.
The contestability of state action — central to an open society — depends on a citizen being able to understand and challenge how a decision was made. Opacity directly threatens this.
Algorithmic discrimination concentrates the cost of public-sector AI failures on already-marginalised groups, undermining the equity pillar.

This is also the lecture where Elliott et al.'s TRUST (L1) re-enters: Transparency, Responsibility, Understanding, Stewardship, Truth. Grimmelikhuijsen & Meijer are essentially asking how a public institution discharges each of those five letters when it deploys an algorithm.

Likely essay-question angles

"List Grimmelikhuijsen & Meijer's six threats to algorithmic legitimacy. For each, name one institutional mechanism that mitigates it."
"Apply the six-threats framework to the Dutch toeslagenaffaire. Which threat materialised most acutely, and what institutional reform would specifically address it?"
"Distinguish algorithm aversion from algorithm appreciation. How does this distinction complicate the design of human-in-the-loop ADM systems?"
"Trust is conventionally decomposed into competence, benevolence and integrity. Discuss how each of these is harder to establish for an algorithmic decision-maker than for a human one — and which one matters most for public (as opposed to corporate) AI."

Quick self-test

Three psychological components of trust?
Define algorithm aversion vs. algorithm appreciation; give one moderator.
Three legitimacy categories (input/throughput/output) — what does each evaluate?
List the six threats of Grimmelikhuijsen & Meijer.
Map at least four of those six threats onto the toeslagenaffaire.
What does "calibrated institutional response" mean — why isn't a single mitigation enough?
Connect this lecture back to the Elliott TRUST framework from L1.