Realtime Multiple Pair-Eye Tracking on Edge Devices

Abstract

At Vitalify Asia, we are a web & mobile app development company. We provide rapid product development and improvement on the DevOps basis with the team committed to providing value to users.

The application of pair-eye tracking in practice in many different fields such as: observing attention while driving, observing concentration when working or studying, observations about the direction of vision…

To do this, it is necessary to ensure that the conversion of the trained AI model into edge devices works effectively even if the computing power is not as good as those on computers.

How did we do this conversion with the pair-eye tracking model at 60 fps on the Jetson TX2

Introduction

Steps taken as well as evaluate the results

• Model AI is trained on PC
• Convert AI model to Jetson TX2
• Perform post-processing

Regarding the AI model, based on the RetinaFace model, the RetinaFace implementation needs to pay attention to the size of input image (our experiment input size is 640x480) to ensure that the model after converting to the edge device still works effectively.

To RetinaFace model can work on Jetson TX2 need to convert to TensorRT format, TensorRT is the best performance and direct support from Nvidia.

Perform post-processing, this is the last step, but it greatly affects the performance of the entire process. It needs process in the parallel with CUDA functions.

Demonstration with 3D view

Videos

Conclusion

When performing AI model conversion for edge devices (Jetson Nano / Jetson TX / …), the model architecture is simple enough to execute, but also necessary to take care of pre-processing / post-processing because it may affect the execution speed of the device. In addition, should be care the processing in the GPU if possible, this helps a lot in improving the speed.

References

• https://developer.nvidia.com/tensorrt
• https://www.nvidia.com/docs/IO/116711/sc11-cuda-c-basics.pdf
• https://www.nvidia.com/content/nvision2008/tech_presentations/Game_Developer_Track/NVISION08-Image_Processing_and_Video_with_CUDA.pdf

The company

Our company is involved in a wide range of projects, not only AI development, but also mobile/web application development. We would be happy to discuss your needs with you.