Vai al Contenuto Vai alla navigazione del sito

CYBER-PHYSICAL SYSTEMS (575SM)

A.A. 2019 / 2020

Docenti 
Periodo 
Secondo semestre
Crediti 
6
Durata 
48
Tipo attività formativa 
Affine/Integrativa
Percorso 
[PDS0-2018 - Ord. 2018] comune
Mutuazione 
Condiviso: SM35 - 575SM - CYBER-PHYSICAL SYSTEMS
Syllabus 
Lingua insegnamento 

English

Obiettivi formativi 

In this course you will learn fundamental concepts of Cyber-Physical
Systems, with particular attention on modelling, monitoring their
behaviour, and learning.
Knowledge and understanding: basic and some advanced topics in hybrid
modelling, switching systems, real-time verification, Markov Decision
processes, reinforcement learning.
Applying knowledge and understanding: being capable of identifying the
best formalism to describe a CPS, of monitoring CPS behaviour, described in a formal language, and of learning models of the environment and
optimal policies from observations, using reinforcement learning.
Communication skills: being able to explain the basic ideas and
communicate the results to experts and to non-experts.
Learning skills: being capable of exploring literature and find alternative
approaches and combine them to solve complex problems.

Prerequisiti 

Knowledge of Python and scientific Python. Extended knowledge of
statistics and machine learning. Basic knowledge of stochastic modelling.
Knowledge of numerical methods

Contenuti 

1. Introduction to CPS and application domains, with examples (e.g.,
medical CPS, transportation CPS, energy CPS)
2. Modelling formalisms: hybrid and switching systems, stochastic hybrid
and switching systems, Markov Decision Processes.
3. Monitoring CPS: temporal logic, verification, monitoring
4. Dynamic programming for MDP: value and policy iteration
5. Foundations of Reinforcement Learning: TD learning, Q learning, Policy
gradient methods.

Metodi didattici 

Frontal lectures and hands on sessions, both individual and in groups.
Ideally, each lecture will have a part of frontal teaching and a part of
hands-on training. Exercises and group tasks may be left for further
elaboration.

Programma esteso 

1. Introduction to CPS and application domains, with examples (e.g.,
medical CPS, transportation CPS, energy CPS)
2. Modelling formalisms: hybrid and switching systems, stochastic hybrid
and switching systems, Markov Decision Processes.
3. Monitoring CPS: temporal logic, verification, monitoring
4. Dynamic programming for MDP: value and policy iteration
5. Foundations of Reinforcement Learning: TD learning, Q learning, Policy
gradient methods.

Modalità di verifica dell'apprendimento 

The exam will be a project work, either individual or in groups of 2 to 4
students. Each project will consist in one or more tasks, and will be
concluded with a report on achieved results. During the presentation,
extra questions can be asked to asses the individual preparation on the
topics of the course

Altre informazioni 

.

Testi di riferimento 

Reinforcement learning: An introduction, RS Sutton, AG Barto -
Cambridge, 2011.
(http://incompleteideas.net/book/bookdraft2017nov5.pdf)
Principles of Model Checking. Baier, Katoen, MIT press, 2008
Raj Rajkumar , Dionisio De Niz, Mark Klein, Cyber-Physical Systems,
Addison-Wesley Professional 2016 (?)
R. Alur. Principles of Cyber-Physical Systems, MIT press, 2015


Torna all'elenco insegnamenti