Explainable Artificial Intelligence for neural networks and its evaluation
September 1, 2026
This course will be given in the context of the Enhance Athens week at TU Delft between November 14 and 21, 2026 and is given on-site only.
Lecturers
Dr. Marco Zullich (TU Delft, the Netherlands)
Abstract
Transparency in the context of an AI system is a fundamental property which is remarked by the EU AI Act and its foundational document, the Ethical Guidelines for Trustworthy AI: indeed, modern-day AI models, such as neural networks, are often so complex that their predictive dynamics are unintelligible to humans. One of the ways for enhancing transparency is by providing explanations, human-understandable tokens of information that approximate the functioning of said models. The branch behind the study of techniques for generating explanations is called Explainable AI (XAI). Other regulations, such as the GDPR, introduce a “right for explanation” for users whose data are processed automatically by other entities, further fueling the necessity for reliable XAI tools. However, the reliability of these methods has often been questioned, and the formal evaluation of XAI quality remains an open challenge, hindering the widespread applicability of XAI to real-world applications.
In this course, participants will learn the basics of XAI, including the issues connected with the formal evaluation of these methods and the socio-technical aspect of XAI (i.e., how can XAI be used to satisfy the necessities of the various stakeholders of an AI system).
This course will be held over 4.5 days in presence at TU Delft Campus between 16 and 20 November 2026 (there will be more compulsory activities for ENHANCE students between 14 and 21 November). The first half day will be a presentation to the course and will include some social activities for group building and mutual introductions of participants and organizing team. From the second to the fourth day, the mornings will be dedicated to the theory with frontal, interactive lectures, while the afternoons will be devoted to practical activities. The last day includes a small theoretical lecture, while the rest of the time is dedicated to the closure of the practicals and the final presentations.
Assessment
Students will be assessed on the basis of a group project, which will be carried out during the week. The projects are going to be based on real-life problems, which the students will need to analyze, deciding the optimal solutions for the data and the stakeholders.
The students will prepare a presentation, to be carried out in front of the lecturers and the other students. Finally, the groups will be asked to reflect on the socio-technical impacts of their decisions.
For TU Delft PhD candidates, there will be one extra assessment part to make the course EQF-8 level. This part will consist in a paper read with a presentation/discussion on a date TBD in late November or December 2026.
Learning outcomes
- Describe the main ways in which an Explainable AI tool can be assessed.
- Criticize the various approaches for Explainable AI with regards to their application, strengths, and weaknesses.
- Evaluate which facets of an Explainable AI tool can be important with regards to the various stakeholders of an AI system.
Course program
The course covers the following topics:
- Recap to Neural Networks
- Introduction to XAI
- Model-agnostic XAI
- Neural-network specific XAI
- Evaluation of XAI
- Socio-technical perspective to XAI
The course is worth 2 ECTS. These credits account for both the class activities (16-20/11/2026) and the social activities that are included in the ENHANCE Week (14-16 and 21/11/2026). It is hence important that, in order for the 2 ECTS to be certified, ENHANCE students attend all of the activities, not only the ones indicated in the schedule below.
Schedule
| Date | Topic morning | Room | Topic afternoon | Room |
|---|---|---|---|---|
| Saturday 14/11 | ENHANCE social activities | ENHANCE social activities | ||
| Sunday 15/11 | ENHANCE social activities | ENHANCE social activities | ||
| Monday 16/11 | ENHANCE social activities | Course intro, group building | build. 33 PULSE-4 | |
| Tuesday 17/11 | Deep learning recap & XAI intro | build. 31 TPM Hall C | Practical (PyTorch & groupwork) | build. 31 TPM Hall D2 |
| Wednesday 18/11 | XAI methods | build. 32 IDE Hall R | Practical (PyTorch & groupwork) | build. 32 IDE Hall R |
| Thursday 19/11 | XAI evaluation | build. 33 PULSE-4 | Practical (PyTorch & groupwork) | build. 33 PULSE-3 |
| Friday 20/11 | Sociotechnical perspective | build. 31 TPM Hall H | Practical & final presentations | build. 31 TPM Hall H |
| Saturday 21/11 | ENHANCE social activities | ENHANCE social activities |
Morning activities last from 9:00 to 12:00; afternoon programs last from 13:00 to 16:00 or 17:00.
ENHANCE social activities are available only to ENHANCE-endorsed students
Enrollment
Note: enrollment for EHNAHCE students is closed. You can only attend this course as a self-funded student (see info below).
Course attendance is free for TU Delft students and staff and for members of ENHANCE universities.
Reach out to the course organizer (mzullich at tudelft dot nl) for attending the course on a self-funded basis.
Self-funded attendance requires that the participants covers all of their expenses independently, including lunches (which are offered only to the ENHANCE Week accepted participants). The course will account for 1 ECTS instead of 2.