RoboCup@Home with TechUnited

Since 2011, I've been involved in RoboCup@Home, as a hobby, mostly through the team of TU Eindhoven: TechUnited.

In 2019, TechUnited won the world cup with our robot HERO, a Toyota HSR robot. Before, we've used the custom-built AMIGO robot, with which we've reached several 2nd and 3rd positions over the years.

My focus has always been on the high-level behavior of our robots. Over the years, we've built a large library of hierarchical state machine that can be composed together into robot applications. The abstractions and other considerations allow this code to run on currently 3 different robots, each with different hardware and accompanying software stack. Cool thing about this is that, besides keeping the software general and explicit in it's assumptions if any, that we could swap out robots last minute if eg. their hardware breaks. Most of that code is in TUE RoboCup

RoboCup challenges

Since 2011, there have been a steady flow of new RoboCup challenges replacing older ones. The applications (RoboCup@Home challenges) that I've built include (and I'm forgetting a bunch).

• Cocktail party: taking orders and serving drinks
• Restaurant: taking food orders and serving those, in a new environment
• Following & guiding: Follow a person around and guide another person back, keeping an eye on the follow so they don't get lost
• Storing groceries: pick & place from a tabletop littered with household items. Sort apples with apples etc.
• Receptionist: Guide a person from the front door to the living room
• Set a table: take items from a cupboard (or from a helpful human) and put them in the right place on a table
• RoboNurse: Get granny's pill bottle by a description she gives and keep an eye on here in case she eg. falls.
• ...

'Conversation engine'

One really cool aspect of our robot I worked on the the Telegram bot we have for our robots. This allows to have a 'natural' language conversation with our robots which you can use to command it, eg. "Bring me a coke from the dinner table" or "Take a picture of the kitchen".

This builds on the telegram_ros interface TechUnited made (and which I ported to ROS2: telegram_ros2)

The conversational part, to get all information needed to perform a command from a user, is implemented in https://github.com/tue-robotics/conversation_engine

This uses a Context Free Grammar to parse a thus restricted form of English, to specify robot commands. Yes, that is restrictive and not a cool as much more free-form natural language parsing, but has some benefits:

• Helps the speech recognition, since the language model is much smaller
• After a partial command is known, or nothing at all even, the grammar has a limited number of possible next words. This small set can be presented to users as
  • tab-completions in console applications
  • Custom keyboards in the Telegram bot

Technical committee

The rules of RovoCup@Home are defined by a Technical Committee. I've been elected to be part of that committee for a couple of years. Some rules that I've contributed (besides several challenges and rules for them)

• Average of best 2 out of 3. This scoring rule eliminates both good and bad luck:
  • Each robot takes each challenge 3 times and if something goes wrong, no worries, you have 2 other attempts.
  • Does your robot do well once or consistently? It better do well consistently!
• Allow cloud computing. I kinda regret this by now since it increases complexity and reduces reliability, but in ~2015-2016 it made a lot of sense.
• Alternative input forms: RoboCup@Home robots take their input by voice preferably. Well, not preferred by the teams, since making a robot do stuff is hard enough already. Showing a QR code embedding the command is a workable alternative. Other alternatives can be proposed too, like the TechUnited Telegram interface.