Virginia Tech
3 Credit Hours
Description
This course surveys recent literature to formalize human-robot interaction. We explore diverse approaches for algorithmic human-robot interaction, including imitation learning, reinforcement learning, shared autonomy, control, physical interaction, active learning, and communication. Students discuss key papers and conduct a research project.
Prerequisites
Graduate Standing. Background knowledge of probability and multivariable calculus is expected. Assignments will require programming in Python.
Corequisites
None
Learning Objectives
Having successfully completed this course, the student will be able to:- Formulate problems in human-robot interaction
- Describe and implement methods for learning from human data
- Design interactive algorithms that optimize robot behavior
- Develop approaches for assistive robots and physical interaction
- Conduct a research project on human-robot interaction
Instructor
|
Prof. Dylan Losey |
Textbooks
This course focuses on readings from recent journal and conference publications. For background review, see: Artificial Intelligence: A Modern Approach by Russell and Norvig.
Grading
- Homework: 50%
- Midterm: 25%
- Project: 25%
Content
This course is composed of nine sequential modules. Below I list the modules and the topics contained within each module.| Module | Lectures |
|---|---|
| Building Blocks | Probability, Entropy and Information Gain, States, Actions, and Rewards, Multi-Armed Bandits, Markov Decision Processes, Value Iteration |
| Behavior Cloning | Behavior Cloning, Dataset Aggregation, Human- and Robot-Gated DAgger |
| Inverse Reinforcement Learning (IRL) | Intro to IRL, Human Models, Reward Learning, IRL |
| Shared Autonomy | Intro to Shared Autonomy, POMDPs, Autoencoders, Latent Actions |
| Reinforcement Learning (RL) | Intro to Reinforcement Learning, Soft Actor-Critic, Human-in-the-Loop RL |
| Control | Motion Primitives, Avoiding Humans, Risk-Aware Control, Game Theory for HRI, Influence |
| Physical Interaction | Responding to Physical Interaction, Assist-as-Needed Control, Learning from Physical Interaction |
| Active Learning | Intro to Active Learning, Asking Questions, Multimodal Feedback |
| Communication | Intro to Communication, Legible Motion, Communicating Objectives and Policies, Theory of Mind, User Study Design, Statistical Significance |
Office Hours
1 Hour per Week
Office hours are held weekly. All students are invited to attend. I also encourage students to use the Discussion feature on Canvas, which I regularly check to answer questions. I will not respond to emails requesting help on homework.
Assignments
There are 8 total assignments, each of which is worth 6.25% of your final grade. There is also one optional bonus assignment which, if completed, will replace your lowest HW grade. Homework is due on Saturday by midnight. Late assignments are not accepted, except when the student has an illness, emergency, or other pressing issue. If you need to ask for an extension due to one of these reasons do not hesitate to contact me. However, you must make your request before the homework deadline.
Exams
The course has a midterm exam that will be conducted remotely. Students will access the exam on Canvas and have a pre-specified amount of time to submit their solutions. The exam is open book, but students may not work with others when completing the exam.
Project
Students will perform a research project individually or in teams of up to two students. Expectations will be scaled with the size of the team. The objective is of the project is to i) dive into a concept from lecture and ii) apply that concept to a problem you are interested in. The project has three stages:
Project Proposal (not graded): 1 page + references. Your report should clearly explain the problem you are going to solve, how this problem is addressed by related papers, and your proposed solution. Include the concept(s) from lecture that you will be exploring. I will provide feedback and suggestions on these reports to guide your next steps.
Project Presentation (10%): Present your findings in a 10 minute recorded talk. Your talk should summarize your work: explain the problem, related papers, and your method. Include demos of your working solution. All team members are expected to participate in the presentation. The recorded talks will be played during the time of our final exam, and students will answer questions after their video plays.
Final Report (15%): 6 pages + references. Divide your report into the following sections: Problem Statement, Related Work, Methods, Experiments, Conclusion, and References. Include figures (and supplementary videos) to visualize your method and results. You will be graded on both the technical content of your report and the clarity of your writing.
Teamwork
Assignments. Group discussion and collaborative work is encouraged on the homework. However, you must submit your own assignment.
Exam. Students are not allowed to collaborate during the exam.
Project. Students are invited to complete the project in teams. If you are having an issue within your team, you should first meet with your team and attempt to resolve the problem. If the issue persists, please notify me and we can look for an appropriate solution.
All submitted materials are considered graded work and are subject to the Honor Code.
Services for Students with Disabilities
Every student in this course should have an equal opportunity to succeed. If you anticipate or experience academic barriers that may be due to disability, including but not limited to ADHD, chronic or temporary medical conditions, deaf or hard of hearing, learning disability, mental health, or vision impairment, please contact the Services for Students with Disabilities (SSD) office (540-231-3788, ssd@vt.edu, or visit www.ssd.vt.edu). If you have an SSD accommodation letter, please email me as soon as possible so that I can accommodate your needs. I am happy to discuss your accommodations in a private meeting during office hours (or by appointment).
Honor Code
The Honor Code pledge that each member of the university community agrees to abide by states:
“As a Hokie, I will conduct myself with honor and integrity at all times. I will not lie, cheat, or steal, nor will I accept the actions of those who do.”
Students enrolled in this course are responsible for abiding by the Honor Code. A student who has doubts about how the Honor Code applies to any assignment is responsible for obtaining specific guidance from the course instructor before submitting the assignment for evaluation. Ignorance of the rules does not exclude any member of the University community from the requirements and expectations of the Honor Code. Academic integrity expectations are the same for online classes as they are for in person classes. All university policies and procedures apply in any Virginia Tech academic environment. For additional information about the Honor Code, please visit: https://www.honorsystem.vt.edu/
Honor Code Pledge
The Virginia Tech honor code pledge for assignments is as follows:
``I have neither given nor received unauthorized assistance on this assignment.''
The pledge is to be written out on all graded assignments at the university and signed by the student. The honor pledge represents both an expression of the student's support of the honor code and a commitment to uphold the academic standards at Virginia Tech.
Academic Misconduct
If you have questions or are unclear about what constitutes academic misconduct on an assignment or exam, please speak with me. The normal sanction I will recommend for a violation of the Honor Code is an F* sanction as your final course grade. The F represents failure in the course, and * identifies a student who has failed to uphold the values of academic integrity at Virginia Tech.