The “Other Me”: Human-Centered AI Assistance In Situ
Project Description
Imagine a situated AI assistant embedded in your smart glasses and collaborating with you to create a novel artifact, e.g., building an Arduino smart lamp. It communicates with you through the natural language to understand your goals and preferences. It works with you to develop and execute the solution. It sees what you see from a first-person view to understand your actions and offer useful tips. It connects with other AI assistants to scale up to complex tasks, e.g., assembling or disassembling an aircraft engine, which require a team solution. AI assistants become the “other me”, an external, but situated extension of our individual minds and senses. The “Other Me” (TOM) is an integrative program of fundamental research to enable wearable, situated AI assistance for everyone in daily life and work.
Research Technical Area
- Decision-making
- Planning
- Man-machine Collaboration
- Machine Learning
- Natural Language Processing
- Computer Vision
- Multi-agent systems
- Reinforcement learning
Benefits to the society
Our research enables a new generation of AI assistants that understand humans’ intent and actions in situ, communicates with humans through natural language, and collaborates with humans to accomplish complex tasks over many steps. By providing just-in-time, situated assistance, they will reinvent the relationship between the human and the device in our daily life and work, from cooking in the kitchen to aircraft engine maintenance in the hangar.
Project’s Publications
–
Team’s Principal Investigator
David HSU is Provost’s Chair Professor in the Department of Computer Science, National University of Singapore, the director of Smart Systems Institute, and also the founding director of NUS Artificial Intelligence Laboratory (NUSAIL). He is an IEEE Fellow. His research interests span robotics, AI, and computational biology. In recent years, he has been working on robot planning and learning under uncertainty and human-robot collaboration.
Recent Notable Awards
- IJCAI-JAIR Best Paper Prize, 2022
- Robotics: Science & Systems Test of Time Award, 2021
- Humanitarian Robotics and Automation Technology Challenge Award, International Conference on Robotics & Automation (ICRA), 2015
The Team
Principal Investigators
David HSU (Lead PI), National University of Singapore
Research Focus: Decision making, Planning, learning, human- robot interaction
Atreyi KANKANHALLI, National University of Singapore
Research Focus: Man-machine collaboration
Wee Sun LEE, National University of Singapore
Research Focus: Machine learning, planning, natural language processing
Pradeep VARAKANTHAM, National University of Singapore
Research Focus: Multi-agent systems, reinforcement learning, planning
Angela YAO, National University of Singapore
Research Focus: Computer vision, machine learning
Co-Principal Investigators
Basura FERNANDO, National University of Singapore
Research Focus: Computer vision, machine learning
Akshat KUMAR, Singapore Management University
Research Focus: Multi-agent systems, reinforcement learning, planning
Gim Hee LEE, National University of Singapore
Research Focus: Computer vision, machine learning
LEONG Tze Yun, National University of Singapore
Research Focus: Responsible AI, decision-theoretic AI, human-AI collaboration
Wei LU, Singapore University of Technology & Design
Research Focus: Natural language processing, machine learning
Malika MEGHJANI, Singapore University of Technology & Design
Research Focus: Planning, human-agent interaction
Desmond ONG, National University of Singapore
Research Focus: Affective Computing
Gergious PILIOURAS, Singapore University of Technology & Design
Research Focus: Multi-agent learning, decision theory, game theory
Harold SOH, National University of Singapore
Research Focus: Human-agent interaction, machine learning
Shengdong ZHAO, National University of Singapore
Research Focus: Interactive design
