Computational Intelligence Lab 2022
The goal of the Computational Intelligence Lab is to enable master level students to connect their mathematical background in linear algebra, analysis, probability, and optimization with their basic knowledge in machine learning and their general skill set in Computer Science to gain a deeper understanding of models and tools of great practical impact.
This includes the often underestimated step of conceptualization and critical modeling of the problem at hand, i.e. reflecting on assumptions and simplifications and justifying the appropriateness of the approach taken. It also includes replacing computation by calculation where possible. It is very hard to understand what may happen, when we run code over data so to speak. What biases are introduced? What guarantees can be made? When will the method work, when fail? What would we even look for empirically to measure success? To answer such crucial questions, we need a mathematical model and not just a computational toolbox in which the model remains opaque to our understanding.
CIL is hence a lab in to regards: (1) it teaches “hands-on” use of mathematical methods and (2) it provides “hands-on” training in programming through practical projects. In contrast to other classes in machine learning and data science, the emphasis is not on comprehensive coverage of topics and content. Rather the course works with a compilation of relevant, weakly interconnected topics, which are exemplary in nature. The goal is to enable students to independently apply the learned skills to models and topics not covered.
We will not provide a systematic introduction to the mathematics needed. One can consult excellent undergraduate textbooks or machine-inspired textbooks such as Mathematics for Machine Learning. As far as programming goes, we will make use of Python, its scientific computing library NumPy and some more specialized libraries such as PyTorch when it comes to neural network models.
Useful Links
- Lecture Notes
Zoom Exercise & QA Session Link, Exercise: Friday 16-18, Q&A Session: Thursday 14-15
Schedule and Material
Please find the course schedule and all material on the CIL Moodle page: https://moodle-app2.let.ethz.ch/course/view.php?id=16549.
News
17.12.2021 | The website for CIL 2022 is online! |
25.02.2022 | The material for this course has moved to the CIL Moodle page: https://moodle-app2.let.ethz.ch/course/view.php?id=16549 |
Schedule
Week | Topic | Lecture | Exercises |
---|---|---|---|
8 | Introduction / Dimension Reduction I | ||
9 | Dimension Reduction II | ||
10 | Dimension Reduction III | ||
11 | Matrix Completion I | ||
12 | Matrix Completion II | ||
13 | Matrix Completion III / Compressed Sensing | ||
14 | Latent variable models I | ||
15 | Easter Break (no lecture) | ||
16 | Easter Break (no lecture) | ||
17 | Latent variable models II | ||
18 | Neural Networks I | ||
19 | Neural Networks II | ||
20 | Generative Models I | ||
21 | Generative Models II | ||
22 | Final Remarks |
Old Video Recordings
Written Exam
The written exam takes 120 minutes. The language of examination is English. NO WRITTEN AIDS ALLOWED.
Old Exams
Grade
Your final grade will be determined by the written final exam (70% weight) and the semester project (30% weight). The project must be passed on its own and has a bonus/penalty function. Failing the project results in a failing grade for the overall examination of the CIL course.
Binding performance assessment information can be found in the Course Catalog
Semester Project
To join the Kaggle competition for the projects, please follow the private links here.
The semester project is an integral part of the CIL course. Participation is mandatory. Failing the project results in a failing grade for the overall CIL course.
You work in groups of three to four students (no more, no less) to develop novel solutions to one of three topics.
All further information about the semester project can be found in the document linked above.
You have the option to choose between three different projects this year:
Reading Material
Here is a list of additional material if you want to read up on a specific topic of the course.
Linear Algebra
3Blue1Brown Essence of Linear Algebra (15 short youtube videos)
Introduction to Linear Algebra by Gilbert Strang (2016). See also his 35 Lectures on Linear Algebra (youtube)
Matrices
The Matrix Cookbook by Petersen & Pedersen, (2012). Contains useful formulas for derivatives w.r.t. vectors etc.
Machine Learning
Mathematics for Machine Learning by Deisenroth, A. Aldo Faisal, and Cheng Soon Ong.
Pattern Recognition and Machine Learning Christopher M. Bishop, Springer (2006).
Deep Learning
Deep Learning by Ian Goodfellow, Yoshua Bengio and Aaron Courville (2016)
People
Professor | Gunnar Rätsch |
Head TA | Antonio Orvieto |
Head TA | Leonard Adolphs |
TA | Dario Pavllo |
TA | Gregor Bachmann |
TA | Luca Biggio |
TA | Sotiris Anagnostidis |
TA | Lorenzo Noci |
TA | Rita Kuznetsova |
TA | Xinrui Lyu |
TA | Xiang Li |
TA | Linfei Pan |
TA | Harish Rajagopal |