The NAO robot

Implementation

This chapter describes all the steps taken to form the prototype system ex-plained in the previous Chapter 5. The prototype system comprises of three major components - the NAO robot, the Integration Layer, and the Hand Signs Recognition Component (HSRC). Since, HSRC is the most critical part of the prototype, initial build/versions of the prototype focused on developing only the HSRC. Other two components were incorporated to the subsequent builds forming the final version. These components are described independently in different sections in this chapter.

6.1 The NAO robot

The NAO robot is the only component that the user interacts with the prototype system. NAO acts as a front end interface of the prototype system for the user.

This research employs a task-based scenario for the user where the NAO robot carries the following activities to simulate the hand signs recognition process:

1. Communicate with the user using its speech recognition modules and begin the hand signs recognition process after detecting the ’Start Sign Recognition’ command.

2. Ask the user to prepare themselves for the hand signs recognition process.

55

56 chapter 6 implementation 3. Capture the image using its camera module, and send it to the Integration

layer for predictions.

4. With the help of Profile Manager, retrieve and perform the actions for the detected hand-sign, as configured by the user.

The NAO robot runs on the NAOqi software framework. NAOqi [29] is a cross-platform programming framework developed in C++ and Python. It allows homogeneous use of different hardware modules (sensors, robot parts) with software processes (information sharing, parallelism, synchronization, event handling). Programming and debugging for the above activities would be a tedious process. A better alternative is to use theChoregrapheprogramming tool, as recommended by the robot manufacturers [27]. Choregraphe is a multi-platform desktop application that allows creating complex scenarios (interact with users, customized response animations, speech recognition, and many more) without writing low-level programming code. The next section provides the implementation details on how NAO is programmed using the Choregraphe for this prototype.

6.1.1 Programming NAO using Choregraphe

Choregraphe is a graphical language for the NAOqi framework that allows using all the NAOqi APIs with additional visual tools that ease the program-ming process of the NAO robot. It employs scenario-based implementation via Projects.Projectsare a collection of files andBehaviorsto simulate a specific scenario, a use-case or an experiment with the NAO robot. Behaviors are a set of instructions sent or installed in the NAO robot to make it interact with its environment, process the input, and generate suitable results. Behaviors consist of variousBoxesthat simulate primary actions (like walking, saying a word) and complex implementations (like exploring a room, recovering from a fall).

The Figure 6.1 illustrates the hand signs recognition scenario implemented in NAO Robot using Choregraphe for this research prototype. Box 1: Start initializes the NAO robot with all the necessary libraries, including a speech-recognition module to understand and recognize predefined words or phrases to begin the hand-signs recognition scenario. The user must initiate this task-based scenario using’Start Sign Recognition’ voice command. NAO uses key-word spotting in its speech recognition module - a technique where it continu-ously searches for specific keywords (’Start Sign Recognition’in this case) in its input data.

On a successful search, NAO begins the hand-signs recognition process

repre-6.1 the nao robot 57 Figure 6.1:Choregraphe Behavior: ’Hand Signs Recognition’

sented byBox 2: Hand-Signs Recognition. The second diagram in Figure 6.1 reveals different modules used inBox 2: Hand-Signs Recognition:

• Box 2.1: Say’Show Sign’- NAO asks the user to get ready by saying’Please show the Sign’ and expects the user to show the hand sign positioned correctly in front of its eyes.

• Box 2.2: Take Picture and Box 2.3: Animate Eyes- NAO captures the image using its top camera and acknowledges the user by saying’Sign Captured Successfully’ and animates its eyes (colour of both the Eye-LEDs are changed green) using the animation handler. NAO saves the captured image having a 1280 x 960 resolution in its internal storage at

"/home/nao/recordings/cameras/"as"image.jpg".

• Box 2.4: Integration Layer- The saved image is passed to the Integration layer using a custom python script module. It uses ’requests’ library to create a POST request containing the "image.jpg". NAO awaits the response from the Integration layer, and the subsequent tasks are carried byBox3: Action Mapping.

On receiving the response from the Integration layer,Box 3: Action Mapping executes different actions/features of the robot to the corresponding hand-sign.

This mapping of a hand-sign to a particular action/feature is predefined in this

58 chapter 6 implementation

scenario as follows:

• Palms-Open: NAO provides some information about the present day (date, time, weather, etc.)

• Thumbs-Up: NAO plays some music to the user using ’Play Music’ module available in Choregraphe.

• OK: NAO recognizes the user’s emotions from facial expressions employ-ing Choregraphe’s Emotion Recognition module.

After executing the mapped action/feature, NAO asks if the user wishes to end the scenario or perform the hand-signs recognition once more. If yes, it repeats the execution from Box 2: Hand-Signs Recognitionelse terminates the hand-signs recognition scenario saying’Good Bye’inBox 4: End.