arXiv preprint ·
Estimating Accurate Hand Pose in Camera Space with Vision Transformer
- Published
- Source
- arXiv preprint, posted before peer review
- DOI
- 10.1145/3767308.3836423
- Original
- Read it at arXiv
- Topics
Abstract
Monocular RGB-based hand pose estimation has emerged as a critical research frontier in computer vision. The local hand pose estimation methods predict hand poses relative to the wrist, while global hand pose estimation also requires estimating the wrist's position in the camera coordinate system. However, this camera-space estimation confronts two fundamental challenges: (1) depth ambiguity in monocular settings, and (2) the coupling effect of hand local poses and global wrist positions in the perspective projections. In particular, this coupling reflects that the projections are jointly determined by local hand poses, wrist positions, and camera intrinsics. To overcome these challenges, our framework proposes two key innovations: Transformation-Isomorphism Supervision for hand-depth information extraction and Perspective Information Embedding for resolving above coupling effect of local pose and wrist position, both integrated within the mainstream encoder-decoder architecture. Besides, we propose a novel framerate-aware multi-dataset training strategy for sequential pose refinement. Our fully integrated approach achieves at most 37.1\% superiority in CS-MJE over SOTA on HO3D. Project page: https://github.com/Mine268/CS-ViT.
The authors’ abstract, as arXiv lists it. arXiv’s listings, abstracts included, are free for anyone to reuse.
Read it on ReadSearch
Each opens after you create an account, and brings you back to this paper.
More in Computer Vision
Ultra-fast Neural Inference for Stochastic Gaussian Splatting Denoising
When is a closed-form RGB->S/P ratio adequate? A hyperspectral characterization on natural scenes for mesopic display
AgriGen: Large-Scale Scene Generation Framework for Photorealistic Agricultural Robotics Simulation
WorldCrafter: Consistent Video World Model with Implicit 3D-aware Memory
PartLLM: A Unified Multimodal Foundation for 3D Part Segmentation
Get the few papers worth your time in Computer Vision every morning.
ReadSearch reads the new papers in your field each day and brings you the few worth reading, each with its key finding in a sentence.