Lecture 6 — Medical AI & Digital Twins (Van Rooij + Bontje)

Paper: Wang, W., He, F., Li, Y., Tang, S., Li, X., Xia, J., & Lv, Z. (2023). Data information processing of traffic digital twins in smart cities using edge intelligent federation learning. Information Processing & Management, 60, 103171. (full paper now available)

Type: Thematic (orange). Two short half-lectures combined into one session: - Van Rooij: ML applied to medical / neuro data (classification and stratification). - Bontje: digital twins of urban traffic (sensor-driven simulations of cities).

Both halves share the Promises vs. Risks framing that has structured the thematic lectures.

Site numbering: this page is labelled "L6" here, but the student manual numbers Medical AI / Digital Twins as Lecture 7 (obligatory reading 7, Wang) and Trust in AI as Lecture 6 (reading 6, Grimmelikhuijsen & Meijer). Do not be thrown if the exam uses the manual's numbers; the content pairing (Medical AI / Digital Twins with Wang and Bontje) is what matters.

Must-know core → Minimum to Pass

Classification (supervised, known labels) vs stratification (unsupervised, hidden subgroups) · risks: biological determinism, malignant use · digital twin = bi-directional real-time city model · Bontje SWOT (threats: privacy, overreliance) · Wang 2023 = five-dimensional digital-twin model + edge computing + two optimised CNN traffic-sign detectors (SSD-ResNet50, AD-DarkNet-53) on GTSDB; the title says "federation learning" but the paper does not actually implement it.

Half A — AI in Medicine (Van Rooij)

Background

AI applications in healthcare have skyrocketed in the last decade: - AI tools to develop new medicines - AI tools to analyse wearables data - AI tools to diagnose patients - AI tools to investigate (neuro)biological underpinnings of disorders

Two methodological families

Method	Type	Purpose
Classification	Supervised ML	Predict a-priori group labels (patient vs. control, diagnosis) and identify which features are most predictive
Stratification	Unsupervised ML	Identify hidden / underlying subgroups within a population (data-driven phenotyping)

Three worked examples

Topic 1 — Classifying ADHD patients from fMRI

Data: fMRI activation maps during an Inhibition task.
Classifier: Gaussian Process Classifier (a Bayesian ML method).
Outcome: patient diagnosis + a feature-weight map showing which voxels drove the prediction.
Performance: N = 700; accuracy 77.1%; sensitivity 75%; specificity 80%; ROC AUC ≈ 0.82.

Promises	Risks
Better classification of patients	Determinism / wrongful interpretation (a 77%-accurate model is not "the brain causes ADHD")
More insight into neurobiology of disorders	Inaccurate individual predictions
More insight into between-subject heterogeneity	Malignant use (e.g. by insurance companies)

Topic 2 — Predicting COVID-19 cases & deaths from demographics

Data: COVID-19 clinical data + community-level demographics.
Method: logistic regression.
Outcome: odds-ratios / risk estimates for community-level COVID cases and deaths.

Promises	Risks
Better understanding of COVID-19 spread	Discrimination / inequality (some groups labelled "high-risk" → over-policing or exclusion)
Targeted interventions in at-risk populations	Wrongful attribution of causality (black-box confounders)
Improvement of healthcare system	Wrongful biological determinism
	"Garbage in, garbage out" — biased input data → biased policy

Topic 3 — Stratifying autism (ASD) subjects from brain structure

Data: structural morphometry of 53 brain segments.
Methods: normative modelling (deviation from a learned normative range) + spectral clustering.
Outcome: data-driven clusters of patients with distinct clinical profiles.

Promises	Risks
Insight into between-subject heterogeneity	Statistically spurious clusters that don't replicate
Targeted interventions based on subject profile

Van Rooij's wrap-up

Many different applications for AI in the medical field.
More data, better access, better methods.
Benefits to science and health are real, but so are the risks.
Understanding the AI methods themselves is necessary to mitigate the risks — black-box clinical decisions cannot be ethically justified.

Half B — Digital Twins of (Urban) Traffic (Bontje)

What is a digital twin?

A virtual, bi-directional model of a physical city that visualises urban processes in real time and supports planning, management, and decision-making. "Bi-directional" = the twin both reflects sensor data from the city and feeds decisions back to it.

Slide framing (a likely definition question): a digital twin is more than a simulation — its defining feature is being connected to the real world in real time (a simulation runs offline on assumptions; a twin is continuously fed by, and feeds back to, the live city).

Why for cities?

Cities are complex and constantly changing.
DT data makes that complexity visible.
DTs support faster and better-informed decisions — traffic, air pollution, accessibility, safety, livability.

Traffic specifically

Traffic is one of the most important urban systems.
Goal: predict and control traffic flow.
Use cases: safety and accessibility analysis; event planning / crowd management; scenario testing ("what if we close a road?").

The Dutch state of play (DMI programme)

No "network" of DTs yet, but lots of shared assets: 3D city models, dashboards, simulations, fieldlabs / pilots.
Aim: move from isolated pilots / closed systems toward open, modular, reusable systems — reusable building blocks, shared standards, "Digital Twin as a Service," a European Digital Twin Appstore.

Emerging techniques

2D/3D visualisation.
AI and language models as analysis layers.
Dynamic sensor data with local AI — sensors collect real-time traffic data; local computing (edge processing) close to the street processes it; AI recognises traffic situations; the system feeds back to vehicles or traffic infrastructure.

Challenges

Many twins are still pilots.
Expensive custom solutions.
Dependence on one software provider (vendor lock-in).
Need for standards.
Uncertainty in models.

SWOT analysis of DTs of urban traffic (Bontje's framing)

	Positive	Negative
Internal	Strengths: scenario testing; better data-based decisions	Weaknesses: limited real-time traffic data; model uncertainty
External	Opportunities: national network of local DTs; reusable standards/modules; AI + sensor data	Threats: privacy risks; overreliance on models

IOS connections explicitly drawn by Bontje

Open Cities — test redevelopment scenarios before implementation; compare effects on movement, accessibility, safety, livability.
Behaviour and Institutions — embed cognitive models of pedestrians/cyclists in DTs to simulate how people perceive, decide and move (this is Bontje's PhD topic; bridges to L2 cognitive modelling).
Fair Transitions — DTs make redevelopment impacts visible across different user groups, supporting inclusive and transparent decisions.

Paper 7 (Wang et al. 2023): Traffic digital twins, edge computing, and traffic-sign recognition

✅ Now verified from the full paper (the version previously summarised here was a reconstruction from the title; that reconstruction has been corrected below).

⚠️ Title-versus-content mismatch — exam-relevant. The title says "edge intelligent federation learning," but the body of the paper never actually implements federated learning (no clients, no local-update aggregation, no global-model rounds). The phrase appears only in the title and abstract framing. The real contribution is three-fold: (1) a conceptual five-dimensional digital-twin framework for the city; (2) an edge-computing traffic-perception architecture; and (3) two optimised convolutional-neural-network detectors for traffic-sign recognition, benchmarked on a standard dataset. If an exam answer claims "Wang uses federated learning to keep traffic data local and protect privacy," that is not supported by the paper — it was the plausible-sounding guess the earlier reconstruction made, and the paper does not deliver it.

What the paper actually contains

1. The five-dimensional digital-twin model (adopted from Tao et al., 2019). A digital twin is formalised as

\[ M_{DT} = (PE,\ VE,\ Ss,\ DD,\ CN) \]

PE = Physical Entity (three levels: Unit, System, System-of-systems).
VE = Virtual Entity, itself \( VE = (G_v,\ P_v,\ B_v,\ R_v) \): geometric, physical, behaviour, and rule models.
Ss = Services (new-function services + business services).
DD = twin Data (physical-entity data, virtual-entity data, service data, knowledge data, fusion-derived data).
CN = Connections among the parts: \( CN = (CN\text{-}PD,\ CN\text{-}PV,\ CN\text{-}PS,\ CN\text{-}VD,\ CN\text{-}VS,\ CN\text{-}SD) \), i.e. every pairwise link between physical entity, virtual entity, data, and services.

This is the same "bi-directional city model" Bontje describes, just written as a formal five-tuple. The paper uses it to argue a Digital-Twin city differs from a traditional smart city: it unifies the physical city and its social/economic characteristics into one live digital record, aims at digitalisation of all elements / visualisation of all states / intelligence of all activities, and tracks the whole urban ecosystem continuously rather than a single district or project phase.

2. Edge-computing traffic-perception architecture. Roadside and on-board sensors collect real-time environment and vehicle data → transmitted via roadside micro-cloud and base station to an edge server → edge server does preliminary processing → results pushed to a centralised platform in the 5G core network for caching / further processing → predictions fed back to the vehicle unit. This is the "local computing close to the street, feeding back to vehicles" idea from Bontje's slides, made concrete — but it is plain edge computing, not federated learning.

3. Two optimised deep-learning detectors for traffic-sign recognition (the bulk of the experimental work):

Model	Base	Key modifications
SSD-ResNet50	Single Shot MultiBox Detector (SSD)	Replace the VGG16 (Visual Geometry Group Network 16) backbone with ResNet50 (Residual Network, 50 layers; fewer parameters than VGG16, jump/skip connections fix accuracy degradation with depth); add a deconvolution fusion module to improve convergence and propagate semantic information from deep to shallow layers. Final network ≈ 65 layers (60 convolutional + 5 pooling), input 512×512.
AD-DarkNet-53 (improved DarkNet-53)	DarkNet-53	Redesigned classification module (Avgpool + classifier of linear → 1-D batch-norm → LeakyReLU → linear) and a Focal Loss replacing cross-entropy.

Deconvolution maps low-dimensional features to a higher-dimensional space by zero-filling; its output size is

\[ f_{(Deconvolution)} = [\,s(x-1) + n - 2\,pad\,]^{2} \]

where \( s \) = stride, \( x \) = input pixel size, \( n \) = deconvolution kernel size, \( pad \) = padding.

Focal Loss (Ross & Dollár, 2017) adds a modulating factor \( a\,(a>0) \) to the cross-entropy loss so that easy-to-classify samples contribute less and hard examples contribute more, plus a balance factor \( b \) to counter class imbalance — speeding convergence and lifting overall accuracy versus plain cross-entropy.

Experimental setup and results

Dataset: German Traffic Sign Detection Benchmark (GTSDB) — 600 training + 300 test images in three classes: prohibition, instruction, warning. Data enhancement (flip, rotate, recolour, crop, scale, grayscale, blur, contrast/brightness) expands this to 45,000 images to avoid over-/under-fitting.
Platform / training: TensorFlow; SSD-ResNet50 trained with batch size 8 over 100,000 steps; learning rate 0.001, decaying to 0.0001 at step 30,000 and 0.00001 at step 60,000; Intersection-over-Union (IoU) threshold 0.5 to label positive vs negative samples.
Training effect: SSD-ResNet50 settles to loss ≈ 0.12 and accuracy ≈ 94.6%, beating VGG16, ResNet and AlexNet. DarkNet-53 with Focal Loss converges faster and to a lower loss than with cross-entropy.
Speed (instantaneity): for 1 / 5 / 10 / 15 images, SSD-ResNet50 averages 146.2 / 155.5 / 167.3 / 184.8 ms — about 6.37 ms faster than the original SSD. Improved DarkNet-53 averages 37.2 / 43.3 / 49.2 / 55.5 ms — about 4.25 ms faster than the original DarkNet-53, and faster than SSD-ResNet50.
Accuracy (averaged over the three sign types): SSD-ResNet50 93.227%, original SSD 84.665%, original DarkNet-53 82.437%, AD-DarkNet-53 95.481%. The improved DarkNet-53 is marginally the best.
Data-enhancement benefit: with clean data the average recognition accuracy rises only +0.418%, but once "bad data" (typical interference) is injected the augmented models gain +7.937% on average — and the benefit carries over to other common models (AlexNet, ResNet), not just the two proposed here.

How it ties back to the course

Bontje's SWOT Threats (privacy, overreliance): the architecture the paper actually builds (edge computing + standard supervised detectors) does not address privacy any better than a normal pipeline — raw sensor data still flows to the edge server and 5G core. So the paper does not discharge the privacy threat the way its title implies. Use this as a concrete example of title-driven over-claiming when arguing about model legitimacy.
L1 CDR (Stewardship, Transparency/Truth) and L7 Grimmelikhuijsen & Meijer: a black-box traffic-sign detector deployed in a city twin is exactly the kind of opaque, high-stakes public-domain system L7 worries about. Accuracy ≈ 95% is not legitimacy: who audits the 5% failures at a real intersection? The paper reports benchmark metrics but no auditing, accountability, or governance layer — the gap L7 names.

Why this matters for an open society

This lecture is methodologically diverse but normatively unified by the Promises and Risks lens:

Medical AI → Transitions & Wellbeing pillar (healthcare delivery), with Equity & Diversity stakes (discriminatory misuse, biological determinism).
Digital twins → Open Cities and Behaviour & Institutions platforms, with privacy and overreliance as Equity & Democracy threats.
The unifying argument across both halves: AI in high-stakes thematic domains demands method-literate citizens, scientists, and policymakers — you cannot evaluate the legitimacy of a clinical classifier or a smart-city DT without understanding the basics of the underlying algorithm. This is essentially the course's core normative claim, re-stated in a thematic key.

Likely essay-question angles

"Distinguish classification from stratification in medical AI. Apply each to a concrete example from Van Rooij's lecture and discuss its risks."
"What is a digital twin of a city? Use Bontje's SWOT to argue whether the Dutch DMI programme is more likely to advance or threaten the IOS pillar 'Open Cities'."
"Wang et al. (2023) is titled around federated edge learning for traffic digital twins, yet the paper actually delivers an edge-computing pipeline plus two supervised traffic-sign detectors. Does the architecture it builds mitigate Bontje's privacy threat? Use the title-versus-content gap to discuss over-claiming and model legitimacy, tying back to Elliott's TRUST and Grimmelikhuijsen & Meijer."
"Compare medical AI and traffic DTs as deployments of opaque ML systems in high-stakes public domains. Which of Grimmelikhuijsen & Meijer's six threats is most acute for each?"

Quick self-test

Difference between classification and stratification in ML — give one medical example of each.
Three risks Van Rooij identifies for classification of psychiatric patients from neuroimaging.
Definition of a digital twin — and what does "bi-directional" mean here?
Bontje's SWOT: name one item in each of the four cells.
Wang et al. (2023): what does the paper actually build (three components), and why is its title misleading about federated learning?
Which IOS platforms does each half of this lecture (medical AI, digital twins) most directly speak to?