When AI Performance Misleads: From Success in Papers to Failure in Practice
| Event Details | |
|---|---|
| Date: | 18.06.2026, 16:00 o'clock - 18:00 o'clock |
| Location: | N 2045, Universitätsstraße 6a, 86159 Augsburg |
| Organizer(s): | Institut für Informatik |
| Topics: | Studium, Wissenschaftliche Weiterbildung, Informatik, Gesundheit und Medizin |
| Series of events: | Medical Information Sciences |
| Event Type: | Vortragsreihe |
| Speaker(s): | Dr. Annika Reinke |
In diesem Semester wird die im WiSe 2022/23 erfolgreich gestartete Vortragsreihe Medical Information Sciences fortgesetzt. Renommierte Wissenschaftlerinnen und Wissenschaftler unterschiedlicher Fachdisziplinen und Forschungsstandorte geben jeden Donnerstag ab 16:00 Uhr Einblicke in aktuelle Fragestellungen und Anwendungsgebiete des breiten Forschungsfeldes Medical Information Sciences.
Despite impressive performance in publications, many AI models fail when deployed in real-world settings. One important reason is poor or misleading validation. This talk explores common pitfalls in validating AI systems, especially the misuse of performance metrics, and shows how they can create false confidence. Practical recommendations will be offered to guide more robust and trustworthy validation practices, aimed at supporting the safe and effective integration of AI into real-world workflows.
More events of this series of events "Medical Information Sciences"
-
„From Modeling Molecular Networks to Predictive Medicine”14. November, 2022
-
„From protein identification to functional interpretation“21. November, 2022
-
Medical Information Sciences28. November, 2022
-
From functional drug profiling to multi-omics biomarker identification in paediatric solid tumors28. November, 2022
-
Tiled Amplicons Panels in NGS-Based Genetic Testing05. December, 2022
-
Retrieval-powered Zero-shot Text Classification16. January, 2023
-
Implementation and Application of Clinical Data Warehousing for Studies in Patients with Heart Failure23. January, 2023
-
Morphological ECG Analysis – Revealing Valuable Features for Medical Diagnosis30. January, 2023
-
Using Anatomical and Physiological Constraints in Machine Learning for Medical Image Processing02. May, 2023
-
Data-driven Quality Improvement in the ICU: How to Improve Quality of Care Using a Quality Registry?09. May, 2023
-
Prognostic and Predictive Models in Multiple Sclerosis - a Task that is Overwhelming the Community16. May, 2023
-
Closing the Loop with Digital Health Systems23. May, 2023
-
Genomic Data Protection and Management [research focus]13. June, 2023
-
Clinical Decision Support in Critical Care20. June, 2023
-
Retinal Vessel Analysis – New Frontiers for Cardiovascular Risk Assessment and Treatment Monitoring27. June, 2023
-
Challenges of Medical Image Analysis in Rare Disorders04. July, 2023
-
From Code to Clinic: LUMC’s Practical Approach to AI Implementation11. July, 2023
-
Multi-Omics Data Analysis [research focus]18. July, 2023
-
Natural Language Processing in the Clinical Narrative24. October, 2023
-
Stress Detection and Management in Daily Life Using Wearable Sensors07. November, 2023
-
Can’t Touch This! - Unobtrusive Sensing Using Artificial Intelligence14. November, 2023
-
Health Data FAIRification: Challenges and Opportunities21. November, 2023
-
GestaltMatcher: an AI that can Delineate Dysfunction from Dysmorphism28. November, 2023
-
Responsible AI for Mental Healthcare: Potential and Pitfalls05. December, 2023
-
Mobile Neurophysiological Monitoring: Novel Sensors and Signal Processing Algorithm12. December, 2023
-
Anaesthesia and Intensive Care Medicine 4.009. January, 2024
-
Live Brain Cancer Diagnostics from Sparse Epignomic Data16. January, 2024
-
Cell Segmentation Free Analysis of Spatial Transcriptomics Data23. January, 2024
-
Magnetoelectric Sensor Systems for (Bio-) Medical Applications30. January, 2024
-
Translational Research Informatics: Projects, Strategies and Structures in Berlin and Germany06. February, 2024
-
Applications of Systems Biology in Drug Research and Development23. April, 2024
-
Modeling & Simulation of Cardiac Function: Combining Mechanistic Models and Artificial Intelligence07. May, 2024
-
3D Computer Vision meets Medical Data and the OR04. June, 2024
-
Quantitative Imaging in Neuroceptor Positron Emission Tomography11. June, 2024
-
From Data to Knowledge: The Impact of Semantic Data Integration18. June, 2024
-
Data-driven Investigation and Characterisation of the Functional Connectome in Spider Phobia25. June, 2024
-
DeepRVAT- Integration of Variant Annotations02. July, 2024
-
Integration and Management of Clinical Routine and Research Data09. July, 2024
-
Opening the Black Box: Interpretable AI Models in Genomics16. July, 2024
-
3D Computer Vision meets Medical Data and the OR24. October, 2024
-
Vortrag "3D Computer Vision meets Medical Data and the OR" von Dr. Benjamin Busam24. October, 2024
-
Vortrag "Advancing and Assessing Computational Methods in Systems Biomedicine"07. November, 2024
-
Advancing and Assessing Computational Methods in Systems Biomedicine07. November, 2024
-
Vortragsreihe Medical Information Sciences „tba“14. November, 2024
-
Development and Application of Biostatistical Methods to Improve Predictive Modeling of Clinical Data for Precision Medicine21. November, 2024
-
Methylation based Classification of Tumours28. November, 2024
-
From Computer Vision to Geometry: Automating Analyses of Movement05. December, 2024
-
Vortrag "Electrical Bioimpedance: Instrumentation and Applications"12. December, 2024
-
AI in Biomedical Research and the Diagnostics of Blood Diseases09. January, 2025
-
Contactless Sleep Monitoring16. January, 2025
-
Novel scATAC-seq Pipeline Reveals Cell Type Specific Chromatin Profiles of Noncoding Elements of the Genome23. January, 2025
-
What Can We “See”? Unobtrusive Psychophysiological Monitoring30. January, 2025
-
Making Sense of Clinical Text: NLP for Risk Detection, Patient Privacy, and Better Communication24. April, 2025
-
Catching the wave: Language models accelerate biomedical evidence management08. May, 2025
-
Towards AI-based Anisotropies of Human Trabecular Bone15. May, 2025
-
Towards Generalist Models in Surgery: Vision and Language Models for Surgical Scene Understanding05. June, 2025
-
Explainable Artificial Intelligence (XAI) in Biomechanics: Insights into Machine Learning Models for Gait Classification26. June, 2025
-
Multimodal AI Analysis of Cardiovascular Health Data03. July, 2025
-
Enhancement of SPECT/CT Imaging in the Context of Radiopharmaceutical Therapy10. July, 2025
-
Computational Modeling of Patient Trajectories17. July, 2025
-
Explaining Graph Convolutional Neural Networks: Patient-Specific Subnetworks and Biomarker Discovery in Cancer24. July, 2025
-
The Precision Oncology Cycle and Technology Updates - Molecular Tumor Board, Decision Support, Secondary Use of Data, Biomarker Development, Da Capo30. October, 2025
-
Image-Based AI in the Regulated Pharmaceutical Production06. November, 2025
-
Bridging Data Science and Clinical Practice: Developing AI Solutions for Neurosurgical Diseases13. November, 2025
-
AI and Digital Medicine Regulatory Science: Not Paperwork, Not Administration, Not a Boring Barrier but the Exiting Route to Meaningful Useful Implementation17. November, 2025
-
Real-Time MRI as a Solution to Motion and Moving Children20. November, 2025
-
Acting, Fast and Slow - Basal Ganglia Control of Decision-Making and Movement in Parkinson's Disease27. November, 2025
-
Linking Genes to Metabolism: Advancing Data Integration for Deeper Insights into Genetic-Metabolic Wiring11. December, 2025
-
Biotechnology, Medicine, Startups and Artificial Intelligence18. December, 2025
-
Enhanced Clinician-AI Computer-Aided Diagnosis System08. January, 2026
-
Towards Small Scale: Additive Manufacturing for the Use of Microfluidics and Biosensors in Biotechnology15. January, 2026
-
Mathematical Modeling in the Context of Cellular Immunotherapy29. January, 2026
-
Smart-GI: AI-Supported Diagnostics and Therapy in the GI Tract05. February, 2026
-
Seeing the Unseen: From Mechanism to Prediction in Ophthalmology16. April, 2026
-
Translational AI-based Movement Analysis in Dystonia: From Animal Models to Humans23. April, 2026
-
Democratizing Ultrasound Imaging with Robotics and Artificial Intelligence07. May, 2026
-
From Data to Discovery: Artificial Intelligence Boosts Biotechnology21. May, 2026
-
Beyond the Scan: An AI-Driven Holistic Approach to Prognostic Profiling in Metastatic Breast Cancer28. May, 2026
-
When AI Performance Misleads: From Success in Papers to Failure in Practice18. June, 2026
-
From Data to Action: Health AI that does more than Raise the Alarm02. July, 2026
More events: Department of Computer Science
-
May 2026
-
04
-
05
-
06
-
07
-
08
-
09
-
10
-
11
-
12
-
13
-
14
-
15
-
16
-
17
-
-
May 2026
-
18
-
19
-
20
-
21
-
22
-
23
-
24
-
25
-
26
-
27
-
28
-
29
-
30
-
31
-
-
June 2026
-
01
-
02
-
03
-
04
-
05
-
06
-
07
-
08
-
09
-
10
-
11
-
12
-
13
-
14
-
-
June 2026
-
15
-
16
-
17
-
18
-
19
-
20
-
21
-
22
-
23
-
24
-
25
-
26
-
27
-
28
-
-
June 2026 / July 2026
-
29
-
30
-
01
-
02
-
03
-
04
-
05
-
06
-
07
-
08
-
09
-
10
-
11
-
12
-
-
July 2026
-
13
-
14
-
15
-
16
-
17
-
18
-
19
-
20
-
21
-
22
-
23
-
24
-
25
-
26
-
-
July 2026 / August 2026
-
27
-
28
-
29
-
30
-
31
-
01
-
02
-
03
-
04
-
05
-
06
-
07
-
08
-
09
-
-
August 2026
-
10
-
11
-
12
-
13
-
14
-
15
-
16
-
17
-
18
-
19
-
20
-
21
-
22
-
23
-
-
August 2026 / September 2026
-
24
-
25
-
26
-
27
-
28
-
29
-
30
-
31
-
01
-
02
-
03
-
04
-
05
-
06
-
-
September 2026
-
07
-
08
-
09
-
10
-
11
-
12
-
13
-
14
-
15
-
16
-
17
-
18
-
19
-
20
-
-
September 2026 / October 2026
-
21
-
22
-
23
-
24
-
25
-
26
-
27
-
28
-
29
-
30
-
01
-
02
-
03
-
04
-
-
October 2026
-
05
-
06
-
07
-
08
-
09
-
10
-
11
-
12
-
13
-
14
-
15
-
16
-
17
-
18
-
-
October 2026 / November 2026
-
19
-
20
-
21
-
22
-
23
-
24
-
25
-
26
-
27
-
28
-
29
-
30
-
31
-
01
-
-
November 2026
-
02
-
03
-
04
-
05
-
06
-
07
-
08
-
09
-
10
-
11
-
12
-
13
-
14
-
15
-
-
November 2026
-
16
-
17
-
18
-
19
-
20
-
21
-
22
-
23
-
24
-
25
-
26
-
27
-
28
-
29
-
-
November 2026 / December 2026
-
30
-
01
-
02
-
03
-
04
-
05
-
06
-
07
-
08
-
09
-
10
-
11
-
12
-
13
-
-
December 2026
-
14
-
15
-
16
-
17
-
18
-
19
-
20
-
21
-
22
-
23
-
24
-
25
-
26
-
27
-
-
December 2026 / January 2027
-
28
-
29
-
30
-
31
-
01
-
02
-
03
-
04
-
05
-
06
-
07
-
08
-
09
-
10
-
-
January 2027
-
11
-
12
-
13
-
14
-
15
-
16
-
17
-
18
-
19
-
20
-
21
-
22
-
23
-
24
-
-
January 2027 / February 2027
-
25
-
26
-
27
-
28
-
29
-
30
-
31
-
01
-
02
-
03
-
04
-
05
-
06
-
07
-
-
February 2027
-
08
-
09
-
10
-
11
-
12
-
13
-
14
-
15
-
16
-
17
-
18
-
19
-
20
-
21
-
-
February 2027 / March 2027
-
22
-
23
-
24
-
25
-
26
-
27
-
28
-
01
-
02
-
03
-
04
-
05
-
06
-
07
-
-
March 2027
-
08
-
09
-
10
-
11
-
12
-
13
-
14
-
15
-
16
-
17
-
18
-
19
-
20
-
21
-
-
March 2027 / April 2027
-
22
-
23
-
24
-
25
-
26
-
27
-
28
-
29
-
30
-
31
-
01
-
02
-
03
-
04
-
-
April 2027
-
05
-
06
-
07
-
08
-
09
-
10
-
11
-
12
-
13
-
14
-
15
-
16
-
17
-
18
-
-
April 2027 / May 2027
-
19
-
20
-
21
-
22
-
23
-
24
-
25
-
26
-
27
-
28
-
29
-
30
-
01
-
02
-