ST Solutions Australia

©Copyright ST Solutions Australia. All rights reserved.

22 May 2017

Please reload

Latest Articles

Pepper Tutorial <2>: Box manipulation

In Pepper Tutorial <1>, we made Pepper say "Hello" by connecting boxes using Choregraphe.

 

In this tutorial, we will explain about the boxes in more detail through the use of some examples on Choregraphe.

 

 

 

Box Structure

 

 

Boxes are the building blocks of the application behaviour, and they are the parts that represent various functionalities with different levels of complexity.

 

Boxes can be found and used from the Box libraries, and users may also define their own original boxes or import from external sources that someone else has created into the Box libraries.

 

  1. Box name

  2. Box icon

  3. Box description: appears when cursor is on the box

  4. Inputs: connectors for receiving signals to start or stop the box or some other data

  5. Output: connector for sending the data or stop signal

  6. Parameter button: for modifying box settings

 

 

 

Input and Output

 

 

There are different types of inputs and outputs.

 

 

Types defined by icons

 

Each input and output has an associated icon displayed with it.

 

 

Input:

 

[onStart] The box is on start state when the signal is sent to this input. The behaviour of the start state varies depending on the type of box.

 

 

[onStop] The box is on stop state when the signal is sent to this input.

 

 

 

[onEvenet] Input other than Start or Stop. The box state stimulated by the input signal varies depending on the type of box.

 

 

Output:

 

[onStopped] When the signal is sent from this output, it indicates that the box has stopped.

 

 

 

[punctual] The signal from this output indicates different things depending on the type of box.

 

 

 

Types defined by colours

 

Each input and output has an associated colour displayed with it. Colours indicate the kind of data it receives or sends.

 

 [Bang] Simple event. There is no data involvement with these inputs/outputs.

 

 

 [Number] Number data (float or integer) is received/sent with these inputs/outputs.

 

 

 [String] String (word) data is received/sent with these inputs/outputs.

 

 

 [Dynamic] Data with either no value (Bang) or some sort of value. If there is a value, it could be one of Number value, String value, Number array, or String array.

 

 

 

 

Parameter

 

 

Each box's variables can be checked and modified by clicking the parameter button found at the left bottom corner of the box.

 

 

When you click the parameter button, the dialog box that looks something like this comes up:

 

 

The modifiable variables differ by boxes. We will explain about the variables in more detail later in the tutorial.

 

 

 

 

Box contents

 

 

There are four types of boxes which vary by how they are constructed.

 

  1. Python box: boxes that are written with python.

  2. Flow diagram box: when some boxes and their combinations of connections are compiled to make a new box, it is called a flow diagram box.

  3. Dialog box: boxes that are written with QiChat script which is a script language that defines a conversation with Pepper

  4. Timeline box: boxes that are aligned to represent certain poses or flow based on a single timeline

 

In this tutorial, we will explain about Flow diagram boxes and Timeline boxes. Python boxes and Dialog boxes will be explained in a different tutorial.

 

 

 

 

 

Types of Box libraries

 

 

There are several categories in default box libraries.

 

  • standard ... box group with basic functionalities

  • advanced ... box group used for more advanced functionalities

  • tablet ... special box group used for robots with a tablet (like Pepper)

 

In this tutorial, we will only be using standard box group to explain what we can do with boxes.

 

 

 

 

Constructing a flow

 

We will explain about different kinds of boxes, and ways of connecting boxes through constructing a sample flow. We will also be explaining how to simulate a flow using a virtual robot without connecting to an actual robot hardware.

 

 

Execution order

 

By connecting multiple boxes in order, processing operation moves from one box to another.

In this tutorial, we will make Pepper move forward and then say "Hello".

 

 

Connecting boxes

 

1. Prepare the boxes that will be used.

Drag & drop following boxes from the Box libraries onto the flow diagram.

 

  • Movement > Navigation > Move To

  • Speech > Speech Settings > Set Language

  • Speech > Creation > Say

 

Boxes in Box libraries are organised by their types and seperated into different folders.

 

 

2. Connect boxes in order.

 

By connecting from onStart to onStopped in the order that you want the boxes to run, the Execution order is now set to Start of application > Start of Move To box > End of Move To box > Start of Say box

 

 

3. Set Language:

Click on the Parameter button, make sure that the Language is set to English, then click [OK].

 

 

4. Set the movement distance:

Movement distance can be modified in the parameter of Move To box. 

Click on the Parameter button, set the Distance X as 0.2 (m), then click [OK].

 

 

Simulating flow with a virtual robot

 

 

1. Go to the "Connection" menu and click "Connect to virtual robot".

Virtual robot should be displayed on the "Robot view" pane.

The robot view perspective can be modified with dragging a mouse (both right and left clicks)

 

2. Go to the "View" menu and click on "Dialog" to display the Dialog pane.

 

3. Click on the "Upload to the robot and play" button on the tool bar.

 

If Pepper (virtual robot displayed on Robot view pane) moves forward and then the word "Hello" is displayed both on the Robot view pane and Dialog pane then the application is running successfully.

 

You may run the application with Pepper hardware instead of virtual robot by connecting Choregraphe with your Pepper and click "Upload to the robot and play".

To connect Choregraphe with Pepper, view Tutorial 1 for more detail.

 

 

Parallel execution

 

Multiple boxes may be executed at the same time by sending a branching signal into multiple onStart inputs.

In this tutorial, we will be explaining how to make Pepper do shake arms and say "Yay" 3 seconds after the motion starts.

 

1. Prepare the boxes that will be used.

Drag & drop following boxes from the Box libraries onto the flow diagram.

 

  • Animation > Moods > Positive > Pepper > Excited

  • Programming > Time > Wait

  • Speech > Creation > Say

  • Programming > Logic > Wait for Signals

 

Wait for Signals box is only executed when it receives signals from both Excited and Say boxes.

 

 

2. Connect boxes in order.

Connect to the onStart inputs of Excited and Wait boxes from the onStart of the flow diagram. This will start the Excited and Wait boxes at the same time.

Then connect the output of Wait box to the input of Say box.

Finally, connect the outputs of Excited and Say boxes to the inputs of Wait for Signals box, and end the behaviour with onStopped.

 

 

3. Customise the parameters.

Set the Timeout variable of Wait box to 3 (s), and set the Text variable of Say box to "Yay".

 

 

 

 

 

 

 

 

 

 

 

 

 

4. Simulate the behaviour with a virtual robot just like the one before.

 

 

Customise motions

 

Excited box is in a Timeline Box format, in which motions and behaviours are combined based on time frames. Let's customise this timeline to modify the timing of Excited motion.

 

1. Double click on Excited box

 

2. The timeline pane that looks like the picture below will appear.

 

 

  • The timeline represents the Time progress of the change in motions. The grey boxes on the timeline is called the "motion key frame", and each holds a record of Pepper's motion at a particular timing.

  • The unit of a timeline is a "frame", and the number of frames per second to execute can be customised in the "Timeline properties". (25 frames/s by default)

  • "Behavior layers" is used when setting certain behaviours to play along with a timeline.

3. Select a part of the timeline with a left click mouse drag [A], and drag the right corner of the selected area [B] to expand the timeline.

 

4. Select a part of the timeline [A] and drag the left corner of the selected area [B] to shift the timeline.

 

5. The areas where key frames are positioned can also be selected, and be copied and pasted as well.

 

 

Controlling structure

 

With boxes that output set variables, the corresponding flow can be switched depending on the output variable.

In this tutorial, we will make Pepper recognise the speech and say "Red" in response to "Strawberries", and "Yellow" in response to "Bananas". We will also set the application to terminate with Pepper saying "Bye bye" when it hears the word "End".

 

 

1. Prepare the boxes that will be used.

Drag & drop following boxes from the Box libraries onto the flow diagram.

 

  • Speech > Creation > Speech Reco.

  • Programming > Logic > Switch Case

  • Speech > Creation > Say x 3

 

2. Connect boxes in order.

 

 

  • "Speech Reco." box implements a voice recognition process, and it outputs the most relevant words selected out of the choices given.

  • The "Switch Case" box which provides conditional branching functionality branches this output and connect to each "Say" box to reflect the answer to the user. The configuration and connection of these boxes must be done manually.

  • The onStopped output of the third "Say" box that sets Pepper to say "Bye bye" must be connected to the behaviour's onStopped output.

 

 

3. Modify the parameters.

 

 

Click on the parameter button of "Speech Reco." box, and set the Word list variable to Strawberries;Bananas;End (separate each word with a semicolon).

 

 

4. Set the conditions.

 

 

Set the variables of the "Switch Case" box to "Strawberries", "Bananas", and "End" (each word must be surrounded by double quotation marks). Now this box has three dynamic outputs each representing a different word recognised by "Speech Reco." box.

 

 

5. Connect each of the three outputs of "Switch Case" to the corresponding "Say" box.

 

 

 

6. Click on the parameters button and modify the Text variable of each "Say" box.

For the first "Say" box, set it to "Red", "Yellow" for the second one, and finally "Bye bye" for the last one. Each word does not need to be quoted.

 

 

 

Simulating the behaviour with a virtual box

 

The Dialog pane allows you to have a text based conversation with a virtual robot by typing words into the dialog box.

 

 

Connect to the virtual robot and run the application. The appropriate answer comes back from the robot in response to the text input. When the word "End" is sent to the robot, it terminates the behaviour, and the robot will not respond to any input onward.

After checking that the application is running successfully, you may now connect to the hardware robot and have a sound based conversation with Pepper.

 

 

 

Creating an original box

 

 

So far, we have only been using the boxes that are provided by default, but Choregraphe also allows you to create your own original box.

 

 

Creating a box out of a collection of multiple boxes

 

In this tutorial, we will explain how to make a box from a collection of multiple boxes using the voice recognition behaviour created previously.

 

 

1. Select the collection of boxes that are to be converted into a single box with a mouse drag.

 

 

2. Right click and select [Convert to Box].

 

 

 

3. Set the name and description of the new box.

 

 

 

4. The selected boxes are now converted into a single box.

 

 

When you double click on this box, you can see that the group of selected boxes are inside this newly made box, which means that the new flow diagram box has been defined.

 

 

 

 

Converting flow diagram boxes into parameters

 

 

The original parameters can be added to the newly made box. In this tutorial, we will use the Conversation box that was made in the previous step to go through the procedures of converting flow diagram boxes into parameters of the Conversation box.

 

 

Add parameter definition into the new box

 

1. Right click on the Conversation box, and select [Edit box].

 

2. On the Edit box dialog box, click the + button right next to the Parameters.

 

3. On the Add a new parameter dialog box, set the name, description, type and default value of the new parameter, then click OK.

 In this tutorial, we will be creating two string parameters called Strawberry and Banana that set the answers (Red and Yellow) to the corresponding input.

 

 

4. With a same procedure, add a second parameter called "Banana".

 5. Click [OK] on the Edit box dialog box to complete the editing process.

 

Now when you click the parameter button on Conversation box, you can see that new parameters "Strawberry and "Banana" have been added.

 

 

 

Edit the internal boxes to reference the configured parameters

 

1. "Get Parameter" box is used to achieve the parameter data.

 

On the box libraries, go to:

Programming > Data Edition > Get Parameter

 

Replace the "Say" boxes with the "Get Parameter" and "Say Text" boxes just like the picture below:

 

2. Set the parameters of the "Get Parameter" boxes as displayed below:

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

Share on Facebook
Share on Twitter
Please reload

Please reload

Tags

Please reload

Archive