Research
I'm interested in multimodal representation learning and reinforcement learning. Here you can find some selective publications. For a complete list check out this link.
|
|
Centralized Training with Hybrid Execution in Multi-Agent Reinforcement Learning
Pedro P. Santos, Diogo S. Carvalho, Miguel Vasco, Alberto Sardinha, Pedro A. Santos, Ana Paiva, Francisco S. Melo
arXiv, 2023
arxiv /
We introduce hybrid-execution, a novel paradigm for cooperative multi-agent systems where agents can passively share their observations at execution time, conditioned on an imperfect communication system.
|
|
Geometric Multimodal Contrastive Representation Learning
Petra Poklukar*, Miguel Vasco*, Hang Yin, Francisco S. Melo, Ana Paiva, Danica Kragic
International Conference on Machine Learning (ICML), 2022
link /
code /
We show how contrastive learning can be used to learn multimodal representations that are robust to missing information at test time. We evaluate our approach in supervised, unsupervised and reinforcement learning tasks.
|
|
Perceive, Represent, Generate: Translating Multimodal Information to Robotic Motion Trajectories
Fábio Vital, Miguel Vasco, Alberto Sardinha, Francisco S. Melo
IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), 2022
link /
We design a robotic system that is able to perceive multimodal instructions from human-users to execute handwriting tasks.
|
|
How to Sense the World: Leveraging Hierarchy in Multimodal Perception for Robust Reinforcement Learning Agents
Miguel Vasco, Hang Yin, Francisco S Melo, Ana Paiva
International Conference on Autonomous Agents and Multiagent Systems (AAMAS), 2022
link /
code /
We introduce MUSE, a multimodal representation learning module for RL agents that allows the robust execution of tasks, regardless of the available modalities at execution.
|
|
Leveraging Hierarchy in Multimodal Generative Models for Effective Cross-modality Inference
Miguel Vasco, Hang Yin, Francisco S. Melo, Ana Paiva
Neural Networks (Special Issue on AI and Brain Science: Brain-inspired AI), 2021
link /
code /
We investigate the problem of cross-modality inference (CMI) in multimodal generative models and show the potential of considering hierarchical representation spaces. We contribute with a novel multimodal MNIST-like dataset.
|
|
Playing Games in the Dark: An Approach for Cross-modality Transfer in Reinforcement Learning
Rui Silva, Miguel Vasco, Francisco S Melo, Ana Paiva, Manuela Veloso
International Conference on Autonomous Agents and Multiagent Systems (AAMAS), 2020
link /
code /
We propose a framework that allows RL agents to transfer policies across different modalities (even to unseen ones during policy training!). We also contribute multimodal Atari environments.
|
|
Learning Multimodal Representations for Sample-efficient Recognition of Human Actions
Miguel Vasco, Francisco S Melo, David Martins de Matos, Ana Paiva, Tetsunari Inamura
International Conference on Intelligent Robots and Systems (IROS), 2019
link /
We propose a representation for sample-efficient motion recognition of human users in a household environment.
|
Teaching
FDD3359 Reinforcement Learning, PhD, KTH Royal Institute of Technology - Spring 2024
DD2430 Project Course in Data Science, MSc, KTH Royal Institute of Technology - Fall 2023
Planning, Learning and Intelligent Decision-Making, MSc, Instituto Superior Técnico, University of Lisbon - Fall 2022, Fall 2021
Computation and Society (AI Ethics), BSc, Instituto Superior Técnico, University of Lisbon - Spring 2019, Spring 2020
|
Students
Current PhD Students
Nona Rajabi, KTH Royal Institute of Technology (co-supervised with Danica Kragic and Mårten Björkman)
Farzaneh Taleb, KTH Royal Institute of Technology (co-supervised with Danica Kragic and Mårten Björkman)
Bernardo Esteves, Instituto Superior Técnico, University of Lisbon (co-supervised with Francisco S. Melo)
Past Master Students
Afonso Fernandes, Instituto Superior Técnico, University of Lisbon (co-supervised with Francisco S. Melo)
Bernardo Esteves, Instituto Superior Técnico, University of Lisbon (co-supervised with Francisco S. Melo)
Fábio Vital, Instituto Superior Técnico, University of Lisbon (co-supervised with Alberto Sardinha and Francisco S. Melo)
|
|