Skip to content

Latest commit

 

History

History
133 lines (92 loc) · 4.7 KB

README.md

File metadata and controls

133 lines (92 loc) · 4.7 KB
Raw

hand-tracking-controls-extras

Disclaimer

When I rushed at a-frames finger tracking I wasn't aware, that I'm kinda reinventing the wheel. A great place with multiple examples is Marlon Lückerts repository.

Another great repo with gesture detection is Ada Rose Cannon handy-work

On the other hand, I've learned a lot doing this, and am fully willing to continue :) To all starring, and forking people - thank You! Any ideas are more then welcome ( i.e. in form of an issue )

Navigation components

Teleport, drag yourself around, or rotate the world like this:

hands.mp4

With a simple setup like this:

<a-entity id="rig">
  <a-camera></a-camera>
  <!-- left hand can teleport, and drag the world position -->
  <a-entity hand-tracking-controls="hand: left" hand-tracking-extras
  hand-teleport="rig: #rig; origin: a-camera" drag-move="rig: #rig; speed: 5">
  </a-entity>
 
  <!-- right hand can rotate the world by dragging -->
  <a-entity hand-tracking-controls="hand: right" hand-tracking-extras drag-rotate="rig: #rig">
  </a-entity>
</a-entity>

More info in the docs

Simplified (?) Joints API

Hands in browser:

Import the script:

<script src="https://gftruj.github.io/hand.tracking.controls.extras/dist/aframe-hand-tracking-controls-extras.js"></script>

Add the hand-tracking-extras to the hand:

<a-entity id="left-hand" hand-tracking-controls="hand: left; " hand-tracking-extras></a-entity>

Wait for the hand-tracking-extras-ready event:

var hand = document.getElementById("left-hand");
hand.addEventListener("hand-tracking-extras-ready", (evt) => {
  var jointsAPI = evt.detail.data.jointsAPI;
});

Or grab the joints manually:

var joints = document.getElementById("left-hand").components["hand-tracking-extras"].jointsAPI
var Wrist = joints.getWrist();
console.log(Wrist.getPosition());

Check out the API and the XRHand docs.

API

Each joint has its "helper" method:

Arguments are optional, but providing them will save memory (as otherwise internal helpers are cloned and returned);

Joint object method Description
getPosition(vector) fills the vector with the joint position
getDirection(vector) fills the vector with the normalized direction
getNormal(vector) fills the vector with the normal vector.
getQuaternion(quaternion) fills the quaternion with the joint orientation.
getRadius() get joint radius
isValid() whether we could read the pose data

Joints

Grab the joints with:

// jointsAPI.get<Finger><Bone>()
// example: const index_tip = jointAPI.getIndexTip();
// 
// Fingers: Wrist(special case, single bone), Index, Middle, Ring, Little
// Bones: Metacarpal, Proximal, Intermediate, Distal, Tip

From the WebXR hand docs:

rawQ vs Q

Similar to the WebXR hand API, the joints are

Joint name XRHand corresponding name (index)
Wrist XRHand.WRIST (0)
T_Metacarpal XRHand.THUMB_METACARPAL (1)
T_Proximal XRHand.THUMB_PHALANX_PROXIMAL (2)
T_Distal XRHand.THUMB_PHALANX_DISTAL (3)
T_Tip XRHand.THUMB_PHALANX_TIP (4)
I_Metacarpal XRHand.INDEX_METACARPAL (5)
I_Proximal XRHand.INDEX_PHALANX_PROXIMAL (6)
I_Intermediate XRHand.INDEX_PHALANX_INTERMEDIATE (7)
I_Distal XRHand.INDEX_PHALANX_DISTAL (8)
I_Tip XRHand.INDEX_PHALANX_TIP (9)
M_Metacarpal XRHand.MIDDLE_METACARPAL (10)
M_Proximal XRHand.MIDDLE_PHALANX_PROXIMAL (11)
M_Intermediate XRHand.MIDDLE_PHALANX_INTERMEDIATE (12)
M_Distal XRHand.MIDDLE_PHALANX_DISTAL (13)
M_Tip XRHand.MIDDLE_PHALANX_TIP (14)
R_Metacarpal XRHand.RING_METACARPAL (15)
R_Proximal XRHand.RING_PHALANX_PROXIMAL (16)
R_Intermediate RHand.RING_PHALANX_INTERMEDIATE (17)
R_Distal XRHand.RING_PHALANX_DISTAL (18)
R_Tip XRHand.RING_PHALANX_TIP (19)
L_Metacarpal XRHand.LITTLE_METACARPAL (20)
L_Proximal XRHand.LITTLE_PHALANX_PROXIMAL (21)
L_Intermediate XRHand.LITTLE_PHALANX_INTERMEDIATE (22)
L_Distal XRHand.LITTLE_PHALANX_DISTAL (23)
L_Tip XRHand.LITTLE_PHALANX_TIP (24)

Roadmap - wishful thinking

Provide:

  1. a simple API for Joints orientation and relations:

    • .islookingUp(), isHorizontallyAligned(), .isCloseTo(other), orientedLike(other)

  2. Integrate navigation components with Adas handy-work