Computational Intelligence Lab 2022

The goal of the Computational Intelligence Lab is to enable master level students to connect their mathematical background in linear algebra, analysis, probability, and optimization with their basic knowledge in machine learning and their general skill set in Computer Science to gain a deeper understanding of models and tools of great practical impact.

This includes the often underestimated step of conceptualization and critical modeling of the problem at hand, i.e. reflecting on assumptions and simplifications and justifying the appropriateness of the approach taken. It also includes replacing computation by calculation where possible. It is very hard to understand what may happen, when we run code over data so to speak. What biases are introduced? What guarantees can be made? When will the method work, when fail? What would we even look for empirically to measure success? To answer such crucial questions, we need a mathematical model and not just a computational toolbox in which the model remains opaque to our understanding.

CIL is hence a lab in to regards: (1) it teaches “hands-on” use of mathematical methods and (2) it provides “hands-on” training in programming through practical projects. In contrast to other classes in machine learning and data science, the emphasis is not on comprehensive coverage of topics and content. Rather the course works with a compilation of relevant, weakly interconnected topics, which are exemplary in nature. The goal is to enable students to independently apply the learned skills to models and topics not covered.

We will not provide a systematic introduction to the mathematics needed. One can consult excellent undergraduate textbooks or machine-inspired textbooks such as Mathematics for Machine Learning. As far as programming goes, we will make use of Python, its scientific computing library NumPy and some more specialized libraries such as PyTorch when it comes to neural network models.

Schedule and Material

Please find the course schedule and all material on the CIL Moodle page: https://moodle-app2.let.ethz.ch/course/view.php?id=16549.

News
17.12.2021The website for CIL 2022 is online!
25.02.2022The material for this course has moved to the CIL Moodle page: https://moodle-app2.let.ethz.ch/course/view.php?id=16549
Schedule
WeekTopicLectureExercises
8Introduction / Dimension Reduction I
9Dimension Reduction II
10Dimension Reduction III
11Matrix Completion I
12Matrix Completion II
13Matrix Completion III / Compressed Sensing
14Latent variable models I
15Easter Break (no lecture)
16Easter Break (no lecture)
17Latent variable models II
18Neural Networks I
19Neural Networks II
20Generative Models I
21Generative Models II
22Final Remarks
Classes
LecturesFri 10-12ML D28
ExercisesFri 16-18Online
Q&AThu 14-15Online
Written Exam

The written exam takes 120 minutes. The language of examination is English. NO WRITTEN AIDS ALLOWED.


Grade

Your final grade will be determined by the written final exam (70% weight) and the semester project (30% weight). The project must be passed on its own and has a bonus/penalty function. Failing the project results in a failing grade for the overall examination of the CIL course.

Binding performance assessment information can be found in the Course Catalog


Semester Project

Semester Project Information

To join the Kaggle competition for the projects, please follow the private links here.

The semester project is an integral part of the CIL course. Participation is mandatory. Failing the project results in a failing grade for the overall CIL course.

You work in groups of three to four students (no more, no less) to develop novel solutions to one of three topics.

All further information about the semester project can be found in the document linked above.

You have the option to choose between three different projects this year:

  1. Collaborative Filtering

  2. Text Sentiment Classification

  3. Road Segmentation


Reading Material

Here is a list of additional material if you want to read up on a specific topic of the course.

Linear Algebra

3Blue1Brown Essence of Linear Algebra (15 short youtube videos)

Introduction to Linear Algebra by Gilbert Strang (2016). See also his 35 Lectures on Linear Algebra (youtube)

Matrices

The Matrix Cookbook by Petersen & Pedersen, (2012). Contains useful formulas for derivatives w.r.t. vectors etc.

Machine Learning

Mathematics for Machine Learning by Deisenroth, A. Aldo Faisal, and Cheng Soon Ong.

Pattern Recognition and Machine Learning Christopher M. Bishop, Springer (2006).

Deep Learning

Deep Learning by Ian Goodfellow, Yoshua Bengio and Aaron Courville (2016)