chen-si-02/AnyHand-Models

AnyHand Models

HaMeR and WiLoR checkpoints co-trained with AnyHand for RGB-based 3D hand pose and mesh reconstruction. The baseline architectures are unchanged; these weights are used with the corresponding model implementations in the official AnyHand repository.

Paper: AnyHand · Project website · Inference code · Dataset · Interactive demo · Colab

The project is accepted to NeurIPS 2026, as announced on the project website.

Released files

File	Purpose
anyhand_wilor.ckpt	WiLoR checkpoint co-trained with AnyHand
model_config_wilor.yaml	Configuration used by the released WiLoR loader
anyhand_hamer.ckpt	HaMeR checkpoint co-trained with AnyHand

These are PyTorch checkpoints loaded by the project code. Use the original architecture-specific loader and preprocessing.

HaMeR configuration requirement: this HF repository currently does not contain model_config_hamer.yaml. The official prepare_hamer.sh tries to download that filename, and the predictor expects a matching model_config.yaml beside anyhand_hamer.ckpt. HaMeR setup therefore needs the verified configuration for the released checkpoint. Do not substitute the WiLoR configuration. The WiLoR checkpoint and its configuration are both available here.

The detector, licensed MANO model, and other runtime dependencies are obtained separately through the official setup instructions. This repository does not include an AnyHandNet-D / RGB-D checkpoint.

Model description

• Input: an RGB image. The unified predictor detects hands, prepares crops, and runs the selected reconstruction backend.
• Outputs: MANO pose and shape parameters, a 778-vertex hand mesh, 21 hand joints, projected 2D keypoints, handedness, and camera estimates through AnyHandPredictor.
• Backends: HaMeR and WiLoR.
• Training: the official project describes these checkpoints as co-trained using AnyHand and real-world hand datasets. The paper studies synthetic/real data mixing while keeping the baseline architectures and training protocols fixed.

The configuration YAML is a loader/training configuration, not a complete record of every dataset, sampling choice, or training run. Refer to the paper for experimental protocols. The published dataset inventory currently contains 6,304,768 samples; its dataset card distinguishes this inventory from the paper's approximately 6.6M-image scale.

Get started with WiLoR

Clone the official code with its submodules:

git clone --recurse-submodules https://github.com/chen-si-cs/AnyHand.git
cd AnyHand

Follow the installation instructions to create the environment and install a compatible PyTorch/CUDA build. From the repository root:

bash scripts/prepare_wilor.sh

The preparation script obtains the WiLoR checkpoint, configuration, and hand detector. Obtain the MANO model from the MANO website under its license and follow the project's MANO setup, including the mean-parameter file.

Then run:

from scripts.rgb_predictor import AnyHandPredictor

predictor = AnyHandPredictor(backend="wilor")
hands = predictor.predict("path/to/image.jpg")
for hand in hands:
    print(hand.is_right, hand.score)
    print(hand.vertices.shape)       # (778, 3)
    print(hand.keypoints_3d.shape)    # (21, 3)
    print(hand.keypoints_2d.shape)    # (21, 2)
    print(hand.mano_pose.shape)      # (48,)
    print(hand.mano_shape.shape)     # (10,)

For projection and rendering, use the helpers supplied by AnyHandPredictor and its documented camera conventions. Its prediction representation and the dataset's stored annotation representation are separate interfaces.

Alternatively, download just the WiLoR files with the HF client, then complete the same detector/MANO/runtime setup:

python -m pip install -U huggingface_hub
hf download chen-si-02/AnyHand-Models anyhand_wilor.ckpt model_config_wilor.yaml --local-dir pretrained_models

For reproducible experiments, specify a commit with --revision COMMIT_SHA, and retain the matching code revision and configuration.

Evaluation

The AnyHand paper reports improved performance from augmenting existing RGB baselines with AnyHand on benchmarks including FreiHAND and HO-3D, together with generalization and data-mixture experiments. Consult the paper for metrics, splits, alignment conventions, and training settings.

This card does not assign paper-table scores to individual checkpoint files without a checkpoint-specific evaluation record. No inference or benchmark evaluation was performed as part of the release-documentation audit.

Intended use and limitations

These checkpoints support research into 3D hand reconstruction, hand-object interactions, and synthetic-to-real generalization. Predictions can be affected by detector failures, strong occlusion, image quality, viewpoints, and appearances outside the training distribution. A reconstruction or confidence score is not a verified measurement of hand geometry. RGB input does not provide measured metric depth.

Training data and synthetic assets may not cover all real-world hands and settings. Applications should evaluate performance on their own data and follow the conditions of the checkpoints, baseline code, detector, and MANO assets.

License

The released checkpoints use CC BY-NC-ND 4.0, consistent with the existing repository metadata and official project license. Baseline implementations, the detector, and MANO assets have their own licenses. See the official repository for component-specific terms.

Citation

If you use these checkpoints, cite AnyHand and the corresponding baseline:

@inproceedings{si2026anyhand,
  title     = {{AnyHand}: A Large-Scale Synthetic Dataset for {RGB(-D)} Hand Pose Estimation},
  author    = {Si, Chen and Liu, Yulin and Ai, Bo and Xie, Jianwen and
               Potamias, Rolandos Alexandros and Zheng, Chuanxia and Su, Hao},
  booktitle = {Advances in Neural Information Processing Systems},
  year      = {2026},
  url       = {https://arxiv.org/abs/2603.25726}
}

@inproceedings{pavlakos2024reconstructing,
  title     = {Reconstructing Hands in 3D with Transformers},
  author    = {Pavlakos, Georgios and Shan, Dandan and Radosavovic, Ilija and
               Kanazawa, Angjoo and Fouhey, David and Malik, Jitendra},
  booktitle = {CVPR},
  year      = {2024}
}

@misc{potamias2024wilor,
  title         = {WiLoR: End-to-end 3D Hand Localization and Reconstruction in-the-wild},
  author        = {Potamias, Rolandos Alexandros and Zhang, Jinglei and
                   Deng, Jiankang and Zafeiriou, Stefanos},
  year          = {2024},
  eprint        = {2409.12259},
  archivePrefix = {arXiv},
  primaryClass  = {cs.CV}
}

For questions about usage or missing release files, use GitHub issues or model discussions.