Multi-view Hand Reconstruction with a Point-Embedded Transformer

Arxiv | Paper | Code

Lixin Yang, Licheng Zhong, Pengxiang Zhu, Xinyu Zhan, Junxiao Kong, Jian Xu, Cewu Lu

POEM is a generalizable multi-view hand mesh reconstruction model which embeds a static basis point within the multi-view stereo space. To infer accurate 3D hand mesh from multi-view images, POEM introduce a point-embedded transformer decoder. By employing a combination of five large-scale multi-view datasets and sufficient data augmentation, POEM demonstrates superior generalization ability in real-world applications.

FAVOR: Full-body AR-driven Virtual Object Rearrangement Guided by Textual Instructions

AAAI 2024 | Paper

Kailin Li*, Lixin Yang*, Zenan Lin, Jian Xu, Xinyu Zhan, Yifei Zhao, Pengxiang Zhu, Wenxiong Kang, Kejian Wu, Cewu Lu

FAVOR is a novel dataset for Full-body AR-driven Virtual Object Rearrangement that uniquely employs motion capture systems and VR. A pipeline for producing digital human rearrangement motion sequences is also presented.

CUT++: Image Style Transfer

GitHub｜ PDF

In this project, we build our CUT++ model for image style transfer on the renowned CUT model. By introducing attention into the GAN-based model and modifying the PatchNCE loss, we achieve decent result on the given dataset.

Chinese Slot Language Understanding

GitHub｜ PDF

In this project, we build a BERT-based pipeline for Chinese slot understanding. We incorporated Lexicon information into the BERT backbone and achieved descent result on the given noisy dataset.