Emotions are important when expressing oneself as 60% of communication is expressed through the emotion found in one’s face. This guide will show how to implement emotion display on a screen with ROS.

The first step is to download the module pygame using terminal.

sudo apt-get install python-pygame

The next step is to open up a text editor and begin with importing pygame. Next you would want to create a loop to create a continuous screen.

#! /usr/bin/env python
 import pygame
 
 screen = pygame.display.set_mode((640, 480))
 running = 1
 while running:
     event = pygame.event.poll()
     if event.type == pygame.QUIT:
         running = 0
     screen.fill((0, 0, 0))
     pygame.display.flip()

This allows for a continuous screen to be displayed on the screen.

Although there is currently a screen, there is nothing that is being displayed, so now is the time to add the 7 faces: neutral, happy, surprise, anger, disgust, fear, and sad.

With the 7 faces above, there is a certain gradual factor based on the confidence of that face that allows each face to revert to the neutral face. An example will be provided of the anger face to show how the gradual factor is included. Another important aspect of displaying the face is to utilize an eye coordinate location. Since sympathy also relies on the location of the eyes, it is included into the eyes. When drawing the eyes and mouths, you can utilize this API to check how to draw each face. The program utilizes an array to store values or probabilities of each emotion.

#draws the eyes
pygame.draw.circle(screen, (0, 0, 0), (255 + eyecoordx, 250 + eyecoordy), 15)
pygame.draw.circle(screen, (0, 0, 0), (355 + eyecoordx, 250 + eyecoordy), 15)
pygame.draw.polygon(screen, (255, 255, 255), [(270 + eyecoordx, 235 + (30 * emotionamts[0]) + eyecoordy), (270 + eyecoordx, 235 + eyecoordy), (270 - (30 * emotionamts[0]) + eyecoordx, 235 + eyecoordy)], 0)
pygame.draw.polygon(screen, (255, 255, 255), [(340 + eyecoordx, 235 + eyecoordy), (340 + eyecoordx, 235 + (30 * emotionamts[0]) + eyecoordy), (340 + (30 * emotionamts[0]) + eyecoordx, 235 + eyecoordy)], 0)
#draws the mouth
pygame.draw.lines(screen, (0, 0, 0), False, [(240, 355 + int(20*emotionamts[0])), (305, 355), (370, 355+int(20*emotionamts[0]))], 5)

The next step would be to integrate ROS into the program or Robot Operating System. Since there is not much information to be published. There will be no need to use a publisher. A subscriber will be needed. First, the subscriber would need to collect which emotion is currently expressed and the probability of the emotion.

import rospy
import std_msgs.msg import String

def callback(data):
    text = str(data.data)
    index = text.index(" ",0,len(text))
    emotion = text[:index];
    for x in xrange(0, 7):
        if(emotions[x] == emotion):
             global wanted
             wanted = float(text[index+1:])
             global direcpos
             global pos
             direcpos = x
             if(pos == -1):
                pos = x
                emotionamts[pos] = wanted
def listener():
    rospy.init_node('emotiondisplay', anonymous=True)
    rospy.Subscriber("emotiondisplay2", String, callback)
    rospy.spin()

After integrating ROS into the program, multithreading is needed to collect information as a subscriber as well as continuously printing out to a screen. To multithread, you need to import the threading module and the time module. With these modules, we can sleep a thread to check for possible messages from the publisher. To ensure that threads don’t switch during the process, the use of a lock is necessary.

import thread
import time
import threading

lock = threading.Lock()
def callback(data):
    with lock:
        text = str(data.data)
        index = text.index(" ",0,len(text))
        emotion = text[:index];
        for x in xrange(0, 7):
            if(emotions[x] == emotion):
                global wanted
                wanted = float(text[index+1:])
                global direcpos
                global pos
                direcpos = x
                if(pos == -1):
                    pos = x
                    emotionamts[pos] = wanted
def func(delay):
    done = False
    first = True
    while(not done):
        with lock:
            display()    
        time.sleep(delay)
def main():
    try:
        thread.start_new_thread(func, (.01,)) 
        listener()
    except Exception, e:
        print str(e)

This code  allows for both threads, displaying the screen and collecting data from the publisher, to run simultaneously.  The func method is called by thread.start_new_thread method is given a parameter of .01 seconds in which is the delay time between each print of display unless the lock of callback prevents it.