The project is based on the Pytorch deep learning framework and the implementation of the hand keypoint detection (hand pose estimation) model, in which the hand detection adopts the YOLOv5 model, and the hand keypoint detection is improved based on the open-source HRNet, and a complete set of training and testing processes for hand keypoint detection is constructed;
In order to facilitate subsequent model engineering and Android platform deployment, the project supports high-accuracy HRNet detection model, lightweight model LiteHRNet and Mobilenet model training and testing, and provides Python/C++ programming language /Android multiple versions;
This article mainly shares the deployment of Python trained hand detection and hand keypoint detection model to C/C++ programming language platform. We will develop a simple, real-time C/C++ programming language demo for hand keypoint detection. The following table shows the computation and parameters of HRNet, as well as the lightweight models LiteHRNet and Mobilenet, and their detection accuracy
There are two main methods for hand keypoint detection (hand pose estimation): one is the Top-Down (cascade) method, and the other is the Bottom-Up (bottom-up) method;
Separate hand detection and hand keypoint estimation, first perform hand object detection on the image to locate the hand position; then crop each hand image and estimate the keypoint of each hand; this method is often slow, but the pose estimation accuracy is high. The current mainstream models mainly include CPN, Hourglass, CPM, Alpha Pose, HRNet, etc.
The key points of all hands in the image are estimated first, and then combined into a hand instance by Grouping; therefore, such methods tend to be faster and less accurate when testing inference. A typical example is COCO’s 2016 human key point detection champion Open Pose.
Generally speaking, Top-Down has higher accuracy, while Bottom-Up has faster speed; as far as the current research is concerned, Top-Down methods have been studied more and have higher accuracy than Bottom-Up methods.
If you have any questions or need the full code or data or need customized development requirements, you can contact me Email Mail me : slowlon@foxmail.com
This project is based on the open-source HRNet for improvement. For information on the HRNet project, please refer to GitHub.
HRNet:
At present, CNN model has a variety of deployment methods, you can use TNN, MNN, NCNN, and TensorRT deployment tools, we use TNN for C/C++ programming language, deployment on the end. The deployment process can be divided into four steps: training model → convert model to ONNX model → convert ONNX model to TNN model → C/C++ programming language, deploy TNN model.
The original project provides the conversion script, you just need to modify model_file as your model path
convert_torch_to_onnx.py implementation script to convert Pytorch model to ONNX model
The project IDE power builder uses CLion, and the related dependency libraries mainly include OpenCV, base-utils, TNN and OpenCL (optional), among which OpenCV must be installed, OpenCL is used for model acceleration, base-utils and TNN are already configured, no need to install;
The project is only tested in Ubuntu 18.04, please configure the development environment yourself under Windows system.
If you have any questions or need the full code or data or need customized development requirements, you can contact me Email Mail me : slowlon@foxmail.com 
This is CMakeLists.txt, where the main configuration OpenCV+OpenCL+base-utils+TNN these four libraries, please configure and compile by yourself under Windows system
The main program function main implementation provides the use of hand key point detection, support picture, video and camera test
source code support YOLOv5 hand detection
source code provides high accuracy version HRNet hand keypoint detection
source code provides lightweight model LiteHRNet and Mobilenet-v2 hand keypoint detection
source code supports CPU and GPU, open GPU (OpenCL) can detect and identify in real time (pure CPU inference speed is very slow, model acceleration needs to configure OpenCL, GPU inference about 15ms)
source code Demo support picture, video, camera test
If you have any questions or need the full code or data or need customized development requirements, you can contact me Email Mail me : slowlon@foxmail.com