SB01a-21 Auto-calibration of Humanoid Robot Head Using Eye-gaze Estimator and Active Efficient Coding

Notify of
Inline Feedbacks
View all comments
PUROHIT, Anushka
January 19, 2022 4:24 pm

Wow! So cool that you used this on Sophia the robot

January 19, 2022 4:33 pm

Thank you so much for the appreciation!

LEUNG, Chi Ching Sunny
January 19, 2022 4:14 pm

What hardware you used, I see you tried to run the code in Ubuntu.

LEUNG, Chi Ching Sunny
January 19, 2022 4:14 pm

I use Arch btw.

WU, Chia-wei
January 19, 2022 4:33 pm

Here are some system requirements for Sophia Robot.
OS: Ubuntu 16.04 (amd64) or later LTS
Processor: Quad Core 3.5 GHz or better
Memory: 16 GB RAM
GPU: Nvidia GeForce GTX 1050 or better
Network: Broadband Internet connection
Storage: 35 GB available space
Other Ports available on host:
1024, 1025, 1026, 3306, 5555, 5556, 8000, 8103,
8104, 8105, 8106, 8110, 8111, 9002, 9090, 10001, 11311
Our system setting is listed in below.
OS: Ubuntu 18.04.5 LTS (64-bits)
Processor: Intel Core i5-7400 CPU@ 3.00GHz x 4
GPU: Nvidia GeForce GTX 1060 6GB/PCle/SSE2
Storage: 313.9GB
I hope this answers the question.

LEUNG, Chi Ching Sunny
January 19, 2022 4:50 pm
Reply to  WU, Chia-wei

Do you use GPIO to control the robot, or is it wireless?

AU, Chin Hang
January 19, 2022 5:16 pm


We are using GPIO to control the Robot through USB port. The motor can be controlled by a set of python code provided by the manufacturer. We control the motor by specifying the USB port, motor ID, goal position,and some more other parameters using the python code. Alternately, we can also use the SDK provided by Hanson Robotics to control the robot after proper calibration.

Last edited 2 years ago by AU, Chin Hang
CAO, Xuanyu
January 19, 2022 3:08 pm

What algorithm did you use to extract and utilize the eye-gaze signals?

WU, Chia-wei
January 19, 2022 3:27 pm
Reply to  CAO, Xuanyu

For extracting, we will try on to methods, mpiigaze and GEDDnet, both will return us pitch and yaw value of the person.
Here are some references:
1.    Chen, Z. and Shi, B., 2020. GEDDnet: A Network for Gaze Estimation with Dilation and Decomposition. arXiv preprint
Currently, we are not integrating any existing algorithm to utilize the signal. We will further develop our own algorithm to make use of these values so that the eye motor can saccade to specific person (who makes eye contact with the robot).
Thanks a lot.

SINGH, Dilsher
January 19, 2022 2:28 pm

Hello Param,

I had a doubt about the overall plethora of contents that have created a cornucopia of scientific techniques. Keeping that in mind, what is the status on the update of the position feedback as well as load feedback. Linear or progressive? What is the O(n)?

AU, Chin Hang
January 19, 2022 3:03 pm
Reply to  SINGH, Dilsher

Hi Dilsher,

The calibration process is done by comparing the current load feedback to the load limit predetermined load limit as the motor is moving. The position feedback is used to determine the next goal position in the calibration process as well as locating the position limit. We are both trying for linear and progressive and the complexity should be in order of O(n).

JAIN, Prabhansh
January 19, 2022 2:28 pm

Hi Param,

May I know what all technologies were used in this project?

Thank you

January 19, 2022 2:44 pm

Hi Prabhansh,

Thanks for your question.

An Active Efficeint Coding algorithm is used to complete saccade calibration. Given a reference point in the video stream captured by the eye camera, the robot try to saccade the eye motor to the point.

Q. Zhu, J. Triesch and B. E. Shi, “Integration of Vergence, Cyclovergence, and Saccades through Active Efficient Coding,” 2020 Joint IEEE 10th International Conference on Development and Learning and Epigenetic Robotics (ICDLEpiRob), 2020, pp. 1-6, doi: 10.1109/ICDL-EpiRob48136.2020.9278126.

The GEDDnet is integrated for eye gaze estimation, the GEDDnet will return pitch and yaw value that will be used in deciding whether a person is looking at the robot. The robot will make contact to that person.

Chen, Z. and Shi, B., 2020. GEDDnet: A Network for Gaze Estimation with Dilation and Decomposition. arXiv preprint 

I hope this answers your question.

JAIN, Prabhansh
January 19, 2022 2:49 pm
Reply to  KHURANA, Param

Thanks a lot param, this helped