Konstruktion Mobiler Roboter Einführung Software Institut für Softwaretechnologie 1
Kontroll Paradigmen Agenda Robot Operating System (ROS) Selbstlokalisation und Navigation Kinect & Point Cloud Library (PCL) Festival und Sphinx 2
Kontrollparadigmen drei grundsätzliche Paradigmen bestehen aus drei Bausteinen SENSE: Roboter nimmt seine Umgebung durch Sensoren wahr ACT: Roboter beeinflusst die Umgebung mit seinen Aktoren PLAN: Roboter plant auf Grund seiner Wahrnehmung die nächsten Aktionen, die zum Ziel führen 3
Kontrollparadigmen reaktives Paradigma SENSE-ACT deliberatives Paradigma SENSE-PLAN-ACT SENSE ACT hybrides Paradigma Synthese der beiden ersten Paradigmen SENSE PLAN ACT PLAN SENSE ACT 4
Sensoren Weltmodell Wissensbasis Entscheidungen Wissen Auftrag Aktoren Aktionen 5
Robot Operating System (ROS) Framework für die Entwicklung von Roboter Software soll OS-like Funktionalität bieten ursprünglich entwickelt von Stanford AI Lab (SAIL) mittlereile führend weiter entwickelt von Willow Garage Open Source Plafform verwendet von alle führenden Robotik- Labs bietet viel Funktionalität auf allen Level 6
(Einige) Unterstützte Plattformen 7
Peer-To-Peer P Prinzipien von ROS viele eigenständige Prozesse und verschiedene Hosts Multi-Lingual Kommunikation basiert auf XML-RPC unterstützt C++, Phyton, Octave, LISP definierte Datentypen, Interface Definition Language (IDL) Tool-Based Linux Philosophie mit vielen kleinen Bausteinen Thin Funktionalität in viele Stand-Alone Libraries verpackt, auch 3rd Party eigenes Build-System unterstützt Open-Source 8
Simulation Manipulation Task Executive ROS Visualization Message Passing Client Libratries Navigation Word Processing Web Browser Memory Management OS GUI File System Scheduler Drivers 9
ROS - Operation ROS ist definitiv iti auf der operationellen Seite unterstützt Nachrichtentransport publisher/subscriber b ib (nodes, messages, topics) client/server (services, action-server) Runtime Ausführung Threads und verwandte Techniken Konfiguration unterstützt die Entwicklung wiederverwendbare Klassen und Bibliotheken Definition von Interfaces Debugging, Logging und Visualisierungs-Tools Build-System 10
Broadcast 11
Client/Server 12
ROS - Funktion ROS unterstützt tüt t den funktionelle Sicht Behavioral Control local l path planner obstacle avoidance trajectory generation Executive global path planner SMACH state machine for task-level control Planning Cognitive Robot Abstract Machine (CARM) von TUM Knowledge Processing for Autonomous Personal Robots (KnowRob) von TUM 13
Bausteine von ROS a node is an executable that uses ROS to communicate with other nodes and does computation messages are ROS data type used when subscribing or publishing to a topic nodes can publish messages to a topic as well subscribe to a topic to receive messages (many-to-many communication) a service is a node with one defined input and one defined output message type Co oncept: Comp putation Grap ph the master is a node that is the name service for ROS it helps nodes find each other nodes rosout is the decentralized ROS equivalent of stdout/stderr the parameter server provides and manages parameter across the network roscore is Master + rosout + parameter server Basic Infra astructure 14
Computation Graph robot laser imu map localization planner 15
Organisation in ROS a package is an organization unit in ROS the goal is to provide functionality it can contain a node, a library, data, configurations a manifests provide metadata about a package, including its license information and dependencies, as well as languagespecific information such as compiler flags. stacks are collections of packages that provide aggregate functionality, such as a "navigation stack" stack manifests provide data about a stack, including its license information and its dependencies on other stacks a bag is a file format in ROS for storing ROS message data 16
Naming in ROS hierarchische h Namenstruktur verwendet für alle Ressourcen nodes parameter topics services nodes haben einen Namespace /node_name/topic_name /t parameter sind auch hirarchisch strukturiert /p3at/odometry/frequency können wie das Linux Dateisystem verwendet werden (relativ, global) 17
Tools in ROS rxgraph display a visualization of a ROS computation graph client library supports the development of nodes allows easy access to topics, paramter, C++ & Phyton rviz is a 3d visualization tool it is online configurable and is able to display sensor data, point clouds, paths, coordinate systems ROS is able to record and playback all topics in an transparent way and in the proper timing 18
Features in ROS Koordinatentransformation t ti mit tf Verwalten verschiedener Referensysteme automatische Umrechnung vereinheitlichte Datentypen für ähnliche Sensoren oder Aktoren z.b. point clouds für Laser Scanner Daten z.b. velocity_command für Pioneer Remote Access von anderen Rechnern transparentes Netzwerk 19
Driver in ROS ROS verfügt über eine große Menge von Standard- d Nodes für Roboter und Sensoren Pioneer Roboter Familie Forbot Sick Laser Scanner Familie Hokuyo Laser Scanner GPS (NMEA) Xsense IMU Katana Arm Firewire Cameras Kinect 20
Mapping (gmapping) Funktionalität in ROS basiert auf der freien gmapping Implementierung einfaches Erstellen von 2d Karten Navigation kompletter navigation stack ermöglicht Autonomes Fahren Organisation von Verhalten Standard Templates für Verhalten (actionlib) Entscheidungsfindung Finite State Machine (SMACH ) 21
Installation auf Ubuntu ROS unterstützt tüt t voll Ubuntu eigenes Notebook hat Vorteile Ausprobieren abseits von Roboter und Lab Remote Access auf Roboter funktioniert am besten mit Ubuntu 10.0404 und ROS Electric Step-by-Step Anleitung im ROS Wiki http://www.ros.org/ Bootstrap System zusätzliche Packages könne einfach nach installiert (kompiliert) werden 22
Selbstlokalisation fundamentales Problem in der Robotik Bestimmung der Position des Roboters in Relation zu seiner Umgebung Pose = Position + Orientierung Indoor ein 3 dimensionales i Problem (x,y,θ) Outdoor ein 6 dimensionales Problem (x,y,z,φ,θ,ψ) Probleme Sensorungenauigkeiten Mehrdeutigkeiten mehrfache Hypothesen 23
Pose Tracking Lokalisierungsarten Verfolgung einer bekannten Position einfach zu lösen für bestimmte Aufgaben ausreichend Kalman-Filter Globale Lokalisierung löst das kidnapped robot Problem aufwendiger zu lösen generelle Lösung Partikelfilter 24
Geschlossene Darstellung der Unsicherheit 25
Prinzip Partikel Filter (1-dimensional) allgemeines Partikel: <x,y,θ,w> Sebastian Thrun 26
Localization in ROS ROS provides a ready-to use localization stack the Adaptive Monte Carlo Localization (amcl) package laser-based localization particle filter localization (MCL) KLD sampling to control the sample size augmented MCL to recover from localization errors Sensor model beam range finder model likelihood field model Motion model sample-based odometry model (differential drive) sample-based odometry model (omni-directional drive) for usage and parameter look to the tutorialt 27
amcl overview Map Server Laser Node initial i i pose/pose with covariance acml particles/pose array Robot Node odometry/odometry-tf 28
ROS Navigation Stack ROS provides a full navigation stack global planning A* and LPN local planning Dynamic Window Approach or Trajectory Rollout supports differential drive and omni-directional robots workswith and without global localization and map can work with 2d and/or 3d maps and sensors for usage and parameter look to the tutorial 29
ROS Navigation Stack Overview 30
Some Challenges the navigation stack is trimmed towards challenging indoor office-like environments narrow passages or doors difficult 3d structures like tables or wider bases rich variety of objects in the environment unexplored areas like blind corners 31
Cost Maps cost maps are used to generate a path or do achieve obstacle avoidance cost maps are 2d Global Cost Map initialized by the global map updated by sensor information map coordinate frame Local Cost Map rolling window centered around the robot cares about local obstacles updated by sensor information odometry coordinate frame 32
Point Cloud easy representation ti of the world represent the scene as a collection/set (cloud) of nd points usually n = 3 but each point might have additional info, e.g. RGB color, intensity, uncertainty (n>3) point cloud P with m points, P={p 1,pp 2,,pp m-1,pp m }, p i ={x i,y i,z i } large amount of data for detailed representations 33
Point Cloud - Examples [pointcloud.org] org] 34
Point Cloud Library (PCL) an open-source library to deal with point clouds allows to represent and manipulate point clouds easily provides a number of standard building blocks basic algorithms are there based on work of Radu Bogdan Rusu (TUM) stand alone library but developed and used by the ROS community as well a standard representation of 3D data in ROS 35
PCL Features template-based t to support several point types a number of standard types, e.g. {X,Y,Z} allows also more complex types, e.g. including normal vectors, RGB color, highly optimized memory alignment (i.e., simple point types) SSE support (i.e., simple point types) reuses high performance libraries Eigen (linear algebra), FLANN (nearest neighbor) prepared for easy parallelization on multi-cores (Intel TBB) provides a processing pipeline data acquisition computation visualization multi-os support (Windows, Mac, Linux, Android) 36
Useful Operations PCL supports building blocks in several smaller libraries filters standard filter mechanism, different types such as downsampling features I/O calculation of 3D features, surface normals, descriptors reading/writing PCL files segmentation cluster extraction, consensus methods for parametric models (cylinder,..) surfaces meshing, convex hull registration point cloud registration, e.g. ICP, own algorithms 37
OpenCV is a powerful open source Vision i library fully integrated in ROS bridge between ROS and OpenCV image format supports a lot of functionality feature extraction segmentation tracking transformation filtering build-in camera drivers and image capture; USB and IEEE 1394 38
Speech Recognition - Sphinx based on the Pocket Sphinx package by CMU easy to use client publishes a string to a topic if a word is recognized needs a language and dictionary file samples form RoboCup@home available tools to create your own works quite fine with a good microphone and a quite calm background 39
Speech Synthesis - Festival the ROS sound-play package allows to play wav and ogg files standard sounds text to speech text to speech is based on Festival Speech Synthesis System by the University of Edinburgh supports different voices, creation of own voice possible reacts on the reception of a SoundRequest message 40
Dialogue Processing Pipeline Environment Speech Understanding Recognition hypothese interpreted intention Dialogue Manager intended response Production generated utterance Speech Recognition Text-to-Speech t Speech Signal (user utterance) Speech Signal (robot utterance) 41
Avatar Talking Head easier to interact t to a face or head face expressions and moods are possible some hints facegen creation of 3D faces and bodies, face-expression slider makehuman creation of animated bodies xface can import facegen files and replay them h-anim - C++-libary to control animations 42
Empfohlene Literatur Murphy: Introduction ti to AI Robotics Thrun, Fox, Burgard: Probabilistic Robotics ROS Wiki (http://www.ros.org) ROS Tutorials Robotics Wiki am IST (http://www.ist.tugraz.at/robotics) PDFs zum download auf der Praktikums-Webseite 43