EngageHuman


level_2


Goal - Make Pepper focus on a particular human.

// Get a human.
Human human = ...;

// Build the action.
EngageHuman engageHuman = EngageHumanBuilder.with(qiContext)
                                            .withHuman(human)
                                            .build();

// Run the action asynchronously.
engageHuman.async().run();

Typical usage - EngageHuman is automatically launched and controlled by BasicAwareness. In most of the cases, you should rely on this standard behavior.

However, you should use EngageHuman to:

  • manually control which human must be engaged (based on your own engagement strategy),
  • know when the engaged human is lost by the robot,
  • implement a multi-engagement functionality (switch the engaged human).

How it works

Human-robot interaction can be divided in three main phases:

  • the engaging phase during which Pepper tries to attract humans or waits for those who are keen to interact with him,
  • the interaction phase during which Pepper keeps his focus on the human he is interacting with,
  • the disengaging phase when Pepper should recognize that the human wants to quit the interaction and finish it properly.

The main interactive content of the app, like verbal exchange and/or content display on the screen, should only occur during the interaction phase.

The engaging phase

level_3

To engage a Human, Pepper needs to know which human is the best target to interact with.

HumanAwareness gives a suggestion via getRecommendedHumanToEngage method:

HumanAwareness humanAwareness = qiContext.getHumanAwareness();
Human recommendedHuman = humanAwareness.getRecommendedHumanToEngage();

Warning

getRecommendedHumanToEngage returns null if there is no recommended human to engage.

This engagement strategy is based on engagement intention state of a human, as well as his distance to the optimal interaction position (0.6 meters in front of the current robot position). Moreover, preference is given to humans who did not already interact with the robot.

The interaction phase

After finding a human to engage, the interaction phase starts by building the EngageHuman action:

EngageHuman engageHuman = EngageHumanBuilder.with(qiContext)
                                            .withHuman(recommendedHuman)
                                            .build();

engageHuman.async().run();

The disengaging phase

level_3

Recognizing the disengagement cues via OnHumanIsDisengagingListener listener can be useful to either provide new content in order to re-engage the human or to execute necessary actions (e.g. request an email address) and close the interaction properly by saying goodbye, before stopping the EngageHuman action.

EngageHuman engageHuman = EngageHumanBuilder.with(qiContext)
        .withHuman(recommendedHuman)
        .build();

engageHuman.addOnHumanIsDisengagingListener(() -> {
    Say say = SayBuilder.with(qiContext)
                        .withText("Goodbye!")
                        .build();
    say.run();
    engagement.requestCancellation();
});

engagement = engageHuman.async().run();

Apart from human’s nonverbal behavior (e.g. shift of attention, moving away from the robot), some common goodbye phrases are also considered as disengagement cues. However, this option is only available if the robot is in listening state when using Chat action.

Use cases

Stay focused during an action

You might want Pepper to focus on a specific human when performing an action. For example, this is the case when you want Pepper to talk to a human.

Human human = ...;

EngageHuman engageHuman = EngageHumanBuilder.with(qiContext)
                                            .withHuman(human)
                                            .build();

engageHuman.addOnHumanIsEngagedListener(() -> {
    Say say = SayBuilder.with(qiContext)
                        .withText("Hello!")
                        .build();

    say.run();
});

engageHuman.async().run();

Performance & Limitations

Engaged human exclusivity

Because engagement represents a bond with a specific human, Pepper can only engage one human at a time.

Disengagement cases

Pepper can lose the engagement bond if he does not see the engaged human anymore. In this case, the engagement action finishes. For better performance, multimodal human detection based on cameras and laser/sonar measurements, it is advised to map the environment and localize the robot, as explained in LocalizeAndMap.

See also