Final Presentation

Olivier Amacker

2026-08-27

Resources

This slide deck is available at this address:
https://olivier.amacker.dev/bachelor-thesis/site/presentations/cbi_presentation.html

Figure 1: Slide deck QR code

Figure 2: fMRI processing pipeline overview (generated with ChatGPT).

Numerical Variability and Reproducibility in fMRI pipelines

fMRI pipelines are computationally heavy with high-dimensional calculations.

Sources of numerical instability:

  • OS and version
  • CPU architecture
  • Parallelization strategies
Figure 3: Example of the same program giving different results depending on the OS

Numerical Perturbation Methods

A way to test numerical variability is actually introducing controlled numerical variability in pipelines of interest, at steps of interest, and see how that variability cascades downstream and in the final results.

One such approach is Monte Carlo Arithmetic (MCA) (Parker and Langley (1997)), which applies controlled random perturbations to floating-point operations.

Available libraries for pipeline instrumentation:

  • Verificarlo(Denis, Castro, and Petit (2018)): LLVM-level FP instrumentation
  • Fuzzy(Greg Kiar et al. (2026)): pre-instrumented Docker (NumPy, SciPy)

Previous Findings

Numerical instability confirmed across modalities:

  • Functional imaging: NPVR of graph metrics derived from FC matrices across correlation thresholds, ranging from 0.1 to 0.2 (Alizadeh et al. (2026))
  • Structural imaging: numerical variation in cortical thickness measures reaches 1/3 of population variability (Chatelain et al. (2026))
  • Diffusion imaging: different computational environments produce different structural connectivity results (Gregory Kiar et al. (2021))

Controlled numerical perturbations can also expand training datasets and improve deep learning generalization (Gonzalez-Pepe et al. (2026))

Heterogeneous methodologies prevent direct comparisons between studies.

Objectives

Collaboration with ATR to investigate numerical stability of fMRI connectivity biomarkers.

Three complementary analyses:

  1. Reproduce graph metrics stability assessment using NPVR (Alizadeh et al. (2026))
  2. Assess edge-wise FC matrix stability and global signal regression effect
  3. Evaluate PCA-based feature selection stability under perturbations (Yamashita et al. (2026))

Code Availability

Environment managed with Nix and uv

Datasets

  • SRPB (Tanaka et al. (2021)): multi-site dataset (1,410 participants, 11 sites) for psychiatric biomarker research
  • BMB (Koike et al. (2021)): Brain-MINDS Beyond dataset (~7,000 sessions, 23 sites) covering psychiatric and neurological disorders

Alizadeh et al. (2026) Paper Goals

Two goals:

  1. Show on artificial data that NPVR increases uncertainty in Cohen \(d\) estimates, especially at small sample sizes
  2. Assess how perturbations introduced by running fMRIPrep within a Fuzzy container impact graph metrics NPVR

\[NPVR = \frac{\sigma_{num}}{\sigma_{pop}}\]

\(\sigma_{num}\) is numerical variability. \(\sigma_{pop}\) is population variability.

Replication of Alizadeh et al. (2026)’s Artificial NPVR

Broadly replicated the artificial NPVR simulation results.

Found an error in the original code: the Cohen \(d\) plot used hardcoded NPVR values instead of computed ones. Hard to confirm as it was unclear whether the replication was wrong. Fix submitted and merged (PR).

Figure 4: Hard coded values for the 3rd graph

Alizadeh et al. (2026) Graph Metrics

FC matrices: Schaefer 2018 parcellation (100 regions, 7 networks) with and without confound regression

Local metrics:

  • Degree centrality
  • Clustering coefficient
  • Betweenness centrality
  • Eigenvector centrality

Global metrics:

  • Small-worldness
  • Average shortest path length

Graph Metrics Results: NPVR per Threshold

Figure 5: NPVR per threshold (paper)
Figure 6: NPVR per threshold (reproduced)

Graph Metrics Results: Confound Regression Effect

Figure 7: Confound regression effect (paper)
Figure 8: Confound regression effect (reproduced)

Methodological Differences from the Original Study

  • Dataset: BMB (Koike et al. (2021)) vs original PPMI (Marek et al. (2018))
  • Confounds: 15 vs 6 (added global signal, cerebrospinal fluid, white matter, 6 anatomical component correlation)
  • Container: custom fMRIPrep 25.2.5 vs Fuzzy’s 23.2.1

FC Matrix Stability Analysis Goals

  • Analyze FC matrices directly instead of derived graph metrics.
  • Assess global signal regression’s effect on edge-wise NPVR.

Global signal regression has been a contentious point in the field, with no settled agreement on whether it should or shouldn’t be done.

FC Matrix Stability Analysis Results

Figure 9: NPVR heatmap across FC matrix edges with all confounds
Figure 10: NPVR delta heatmap (with vs without global confound)
Figure 11: NPVR histogram across FC matrix edges with all confounds
Figure 12: NPVR delta histogram (with vs without global confound)

Yamashita et al. (2026) PCA Feature Selection Stability Analysis

PCA-based feature selection for robust FC biomarkers (MDD, SCZ, ASD) across datasets and sites.

PCA Feature Selection Pipeline

  1. PCA on all subjects’ FC vectors to \(n\) principal components
  2. Test PCs against target variables:
    • Binary: t-test
    • Multi-level: ANOVA
    • Continuous: Pearson correlation
  3. Select significant PCs (FDR-BH, \(q < 0.05\))
  4. Test FC weights (chi-squared, FDR-BH, \(q < 0.05\))
  5. Retain surviving FCs as biomarkers

PCA Feature Selection Improvements

  • Restructured into Python package (PR #1)
  • Improved PC plots (PR #3)

PCA Feature Selection Stability Analysis Procedure

  • Reproduced on full SRPB (Tanaka et al. (2021)) and BMB datasets
  • Perturbation 1: MCA-instrumented np.corrcoef (100 runs SRPB, 15 BMB)
  • Perturbation 2: PCA inputs 64-bit to 32-bit

PCA Feature Selection Stability Results (SRPB)

Figure 13: SRPB diagnostic FC (original)
Figure 14: SRPB diagnostic FC (100% consensus)
Figure 15: SRPB edges (original)
Figure 16: SRPB edges (100% consensus)
Figure 17: SRPB inter connections (original)
Figure 18: SRPB inter connections (100% consensus)

PCA Feature Selection Stability Results (BMB)

Figure 19: BMB diagnostic FC (original)
Figure 20: BMB diagnostic FC (100% consensus)
Figure 21: BMB edges (original)
Figure 22: BMB edges (100% consensus)
Figure 23: BMB inter connections (original)
Figure 24: BMB inter connections (100% consensus)

Conclusion

  1. Graph metrics NPVR: consistent trends with original study, but higher values using 15 confounds. Confound regression effect more variable across thresholds.

  2. Edge-wise FC matrix NPVR: mean of 11 percent across 4950 edges. Doubles without global signal confound.

  3. PCA feature extraction: fully robust under MCA perturbation and 32-bit precision.

Future work:

  • Assess individual confound contributions to NPVR
  • Perturb the cross-site harmonization step
  • Perturb confound regression process itself

References

Alizadeh, Mina, Yohan Chatelain, Gregory Kiar, and Tristan Glatard. 2026. “Numerical Variability of Functional MRI Graph Measures.” bioRxiv. https://doi.org/10.64898/2025.12.22.695524.
Chatelain, Yohan, Andrzej Soko-lowski, Madeleine Sharp, Jean-Baptiste Poline, and Tristan Glatard. 2026. “The Practical Impact of Numerical Variability on Structural MRI Measures of Parkinson’s Disease.” bioRxiv. https://doi.org/10.64898/2026.01.09.698203.
Denis, Christophe, Pablo De Oliveira Castro, and Eric Petit. 2018. “Verificarlo: Checking Floating Point Accuracy Through Monte Carlo Arithmetic.” https://arxiv.org/abs/1509.01347.
Gonzalez-Pepe, Inés, Hiba Akhaddar, Tristan Glatard, and Yohan Chatelain. 2026. “Fuzzy PyTorch: Rapid Numerical Variability Evaluation for Deep Learning Models.” https://arxiv.org/abs/2605.25991.
Kiar, Greg, Yohan Chatelain, Tristan Glatard, Ali Salari, Pablo de Oliveira Castro, michaelnicht, Antoine Hebert, Eric Petit, and Mayank Vadariya. 2026. “Verificarlo/Fuzzy: Fuzzy V3.5.1.” Zenodo. https://doi.org/10.5281/zenodo.20906259.
Kiar, Gregory, Yohan Chatelain, Pablo Castro, Eric Petit, Ariel Rokem, Gael Varoquaux, Bratislav Misic, Alan Evans, and Tristan Glatard. 2021. “Numerical Uncertainty in Analytical Pipelines Lead to Impactful Variability in Brain Networks.” PLOS ONE 16 (November): e0250755. https://doi.org/10.1371/journal.pone.0250755.
Koike, Shinsuke, Saori C. Tanaka, Tomohisa Okada, Toshihiko Aso, Ayumu Yamashita, Okito Yamashita, Michiko Asano, et al. 2021. “Brain/MINDS Beyond Human Brain MRI Project: A Protocol for Multi-Level Harmonization Across Brain Disorders Throughout the Lifespan.” NeuroImage: Clinical 30: 102600. https://doi.org/https://doi.org/10.1016/j.nicl.2021.102600.
Marek, Kenneth, Sohini Chowdhury, Andrew Siderowf, Shirley Lasch, Christopher S. Coffey, Chelsea Caspell-Garcia, Tanya Simuni, et al. 2018. “The Parkinson’s Progression Markers Initiative (PPMI) – Establishing a PD Biomarker Cohort.” Annals of Clinical and Translational Neurology 5 (12): 1460–77. https://doi.org/https://doi.org/10.1002/acn3.644.
Parker, Douglas Stott, and David Langley. 1997. “Monte Carlo Arithmetic: Exploiting Randomness in Floating-Point Arithmetic.” In. https://api.semanticscholar.org/CorpusID:2321215.
Tanaka, Saori C., Ayumu Yamashita, Noriaki Yahata, Takashi Itahashi, Giuseppe Lisi, Takashi Yamada, Naho Ichikawa, et al. 2021. “A Multi-Site, Multi-Disorder Resting-State Magnetic Resonance Image Database.” Scientific Data 8 (1): 227. https://doi.org/10.1038/s41597-021-01004-8.
Yamashita, Ayumu, Takashi Itahashi, Yuki Sakai, Masahiro Takamura, Hiroki Togo, Yujiro Yoshihara, Tomohisa Okada, et al. 2026. “Extraction of Robust Functional Connectivity Patterns Across Psychiatric Disorders Using Principal Component Analysis-Based Feature Selection.” Imaging Neuroscience 4 (February): IMAG.a.1121. https://doi.org/10.1162/IMAG.a.1121.

Acronyms

Acronym Definition
fMRI functional Magnetic Resonance Imaging
OS Operating System
MCA Monte Carlo Arithmetic
LLVM Low Level Virtual Machine
NPVR Numerical to Population Variability Ratio
FC Functional Connectivity
ATR Advanced Telecommunications Research Institute International
PCA Principal Component Analysis
PR Pull Request
BMB Brain Minds Beyond Dataset

Acronyms

Acronym Definition
CSF Cerebrospinal Fluid
MDD Major Depressive Disorder
SCZ Schizophrenia
ASD Autism Spectrum Disorder
ANOVA Analysis of Variance
FDR-BH False Discovery Rate - Benjamini-Hochberg
SRPB SRPB Dataset

Annex

Here is how the NPVR is calculated in details:

\[ \sigma_{num}^2 = \frac{1}{m} \sum_{j=1}^m \left[ \frac{1}{n-1} \sum_{i=1}^n \left( x_{i,j} - \bar x_{.,j}\right)^2\right] \]

\[ \sigma_{pop}^2 = \frac{1}{n} \sum_{i=1}^n \left[ \frac{1}{m-1} \sum_{j=1}^m \left( x_{i,j} - \bar x_{i,.}\right)^2\right] \]

\[ NPVR = \frac{\sigma_{num}}{\sigma_{pop}} \]

where \(n\) is the number of MCA repetitions, \(m\) is the number of subjects, \(x_{i,j}\) is the FC edge value for subject \(j\) in MCA run \(i\), \(\bar{x}_{.,j}\) is the mean across runs for subject \(j\), and \(\bar{x}_{i,.}\) is the mean across subjects for run \(i\).

Confounds Used for the Fuzzy FC Matrices Analysis

{
    "confound_columns": [
        "trans_x",
        "trans_y",
        "trans_z",
        "rot_x",
        "rot_y",
        "rot_z",
        "global_signal",
        "csf",
        "white_matter",
        "a_comp_cor_00",
        "a_comp_cor_01",
        "a_comp_cor_02",
        "a_comp_cor_03",
        "a_comp_cor_04",
        "a_comp_cor_05"
    ],
    "filtered_out_confound_columns": [
        "global_signal"
    ]
}