Samples

ConjureKit Ur Module package includes basic samples that demonstrate the basic features of Ur module.

Requirements

• iPhone (Android support is experimental)
• Unity project with Ur package

Basics

The example code in the Demo sample demonstrates how to use Ur to calibrate the Handtracker to specific user's hand size and visualize the 21 3D landmarks that represent the hand pose.

Import

• Import the Demo sample from the package manager Screenshot of the package manager

• Open the Main scene in Assets/Samples/Auki Labs ConjureKit Ur Module/0.5.*/Demo/Scenes

Overview

Build and run the sample on an iPhone.

Landmark detection

Each landmark's relative position to the phone camera is calculated every time the hand tracker updates the hand pose. When running the app, you should see landmark indices displayed on your hand.

_handTracker.OnUpdate += (landmarks, translations, isRightHand, score) =>
{
    for (int h = 0; h < NumberOfTrackedHands; ++h)
    {
        // Toggle the visibility of the hand landmarks based on the confidence score of the hand detection
        ToggleHandLandmarks(score[h] > 0);
        if (score[h] > 0)
        {
            var handPosition = new Vector3(
                translations[h * 3 + 0], 
                translations[h * 3 + 1],
                translations[h * 3 + 2]);

            var handLandmarkIndex = h * HandTracker.LandmarksCount * 3;
            for (int l = 0; l < HandTracker.LandmarksCount; ++l)
            {
                var landMarkPosition = new Vector3(
                    landmarks[handLandmarkIndex + (l * 3) + 0],
                    landmarks[handLandmarkIndex + (l * 3) + 1],
                    landmarks[handLandmarkIndex + (l * 3) + 2]);
        
                // Update the landmarks position 
                _handLandmarks[l].transform.localPosition = handPosition + landMarkPosition;
            }
        }
    }
};

Calibration

The HandCalibration class handles the calibration process. Clicking the Calibrate button will invoke the StartCalibration method on Handtracker to start the calibration process and show progress updates on the screen.

_handTracker.StartCalibration(report =>
{
    calibrationStatusImage.color = _calibrationStatusColors[_handTracker.GetCalibrationStatus()];
    switch (report.StatusReport)
    {
        case HandTracker.CalibrationStatusReport.CALIBRATION_FINISHED:
        {
            calibrationButton.interactable = true;
            calibrationText.text = "Calibrated";
            SaveCalibrationState();
            break;
        }
        case HandTracker.CalibrationStatusReport.CALIBRATION_PROGRESS:
        {
            calibrationText.text = report.Progress.ToString("P");
            break;
        }
        case HandTracker.CalibrationStatusReport.FAILURE_NO_HAND:
        case HandTracker.CalibrationStatusReport.FAILURE_NO_PLANE:
        case HandTracker.CalibrationStatusReport.FAILURE_NO_MEASUREMENTS:
        {
            calibrationButton.interactable = true;
            calibrationText.text = "Try again";
            break;
        }
        case HandTracker.CalibrationStatusReport.CALIBRATION_AR_NOT_READY:
        {
            calibrationButton.interactable = true;
            calibrationText.text = "Scan the room";
            break;
        }
    }
}, false);

If the calibration is successful, the SaveCalibrationState method is called to save the calibration result in the PlayerPrefs. The LoadCalibrationState is called on Start to load previously saved calibration information.

private void SaveCalibrationState()
{
    var state = _handTracker.GetCalibrationState();
    PlayerPrefs.SetFloat(HandSizeKey, state.HandSize);
    PlayerPrefs.SetFloat(ZScaleKey, state.ZScale);
    PlayerPrefs.Save();
}

private void LoadCalibrationState()
{
    if (PlayerPrefs.HasKey(HandSizeKey))
    {
        _handTracker.SetCalibrateState(new HandTracker.CalibrationState()
        {
            HandSize = PlayerPrefs.GetFloat(HandSizeKey),
            ZScale = PlayerPrefs.GetFloat(ZScaleKey)
        });
    }
    
    calibrationStatusImage.color = _calibrationStatusColors[_handTracker.GetCalibrationStatus()];
}