Introduction

What is ConjureKit?

ConjureKit SDK for Unity is a set of packages that allow developers to build applications for the posemesh, with low latency multiplayer and centimeter precision positioning in AR. The posemesh is the universal spatial computing API that makes these features possible and accessible to everyone in a fair and privacy-preserving manner. You can read more about the posemesh at https://www.posemesh.org/.

Additionally ConjureKit includes packages like hand tracker and volumetric features for creating richer applications that understand and interact with the physical world.

The core ConjureKit package

The ConjureKit (com.aukilabs.unity.conjurekit) package is the core package that handles the connection to the posemesh, and acts as an entry hub for most of the other modules. All ConjureKit SDK modules depend on this package. It offers a range of multiplayer features, from basic custom messaging for simple scenarios to a more advanced Entity Component System (ECS). The ECS is specifically designed for complex multiplayer environments, enabling the integration and management of diverse entities, components, and intricate system logic.

Modules

ConjureKit SDK comes with optional packages known as modules. A module always depends on the core package and similarly needs to be instantiated and initialized.

info

While Auki Labs develops and publishes in-house developed modules (listed below), the SDK exposes the interfaces and primitives required for third-party developers to develop their own modules.

Manna

The Manna module (com.aukilabs.unity.manna) package enables instant calibration with centimeter precision positioning into domains or ad hoc shared AR sessions. It lets users scan a QR code of a known size and determine the device's relative position from that QR code at that moment. It then adjusts the Unity coordinate space to match the coordinate space of the QR code so that (x, y, z) coordinates in the Unity coordinate system will represent the same location in AR (e.g., corner of a table) for all participants. Additionally, Manna can connect to the multiplayer session associated with a QR code, so that participants can not only view digital assets together but also interact with them.

Manna uses the Ark module in combination with the device's positioning capabilities to remember and reason about QR code poses. It refines positioning techniques like "gripping" (positioning in relation to landmarks with known poses) and "swimming" (relying on the device's sensors to keep track of its position, e.g. SLAM) through advanced scanning and landmark recognition.

Vikja

The Vikja module (com.aukilabs.unity.vikja) package implements a shared key-value store used for attaching arbitrary properties to Entities. Each change to the store is broadcast to all participants. The current state of the store is sent to joining participants. This can be useful if you want to implement simple multiplayer features, like a chat, and want to avoid the complexity that comes with ECS.

Dagaz

The Dagaz module (com.aukilabs.unity.dagaz) package enables devices to work with Volumetric Features

A Volumetric Feature (VF) represents a volume that is occupied in the real world. Dagaz will detect occupied space during a ConjureKit session and try to piece together large clusters of matter. Dagaz will track the new VF and update its shape and pose as it detects more information from the real world. You can then use this information to make your AR experiences more interactive and aware of their surroundings.

Odal

The Odal module (com.aukilabs.unity.odal) package allows the loading and instantiating of .glb files with animations at runtime. It provides an easy-to-use interface for creating 3D assets in AR that are visible to all participants in the multiplayer session.

Grund

The Grund module (com.aukilabs.unity.grund) package helps with an important aspect of most AR experiences: virtual assets should be properly vertically aligned with surfaces in the physical environment. For example, an AR animal sitting on the floor should neither appear to sink into the floor nor float above the floor. Grund ensures that all the participants of the multiplayer session see the horizontal planes at the same height. So if someone creates an AR pet sitting on the table, it looks correct for all other participants.

Standalone Packages

ConjureKit SDK includes also additional packages like utility libraries or simply additional features that do not require ConjureKit core. These can be used on their own, but are also used by modules to implement some features.

ARFoundation integration

The ARFoundation integration (com.aukilabs.unity.integration.arfoundation) package provides methods for easy integration with Unity's ARFoundation framework.

Ark

The Ark (com.aukilabs.unity.ark) package is a library of computer vision and implementation algorithms for QR code detection and pose estimation. It provides developers with low-level access to QR code data, enabling the creation of more nuanced and customized use cases for the scanner. Ark uses the Ceres module to help Manna achieve sophisticated QR code pose estimation using single or sequential camera images.

Ceres

The Ceres (com.aukilabs.unity.ceres) package is a limited C# wrapper for Ceres Solver, a library natively implemented in C++ for modeling and solving complicated optimization problems. This package includes the full native library compiled for mobile platforms. Ark uses Ceres for pose estimation, but developers are free to use it for optimizing other functions as well.

Ur

The Ur (com.aukilabs.unity.ur) package provides real-time tracking of 3D hand poses from a monocular RGB camera feed (no LiDAR is required), the kind commonly found on mobile phones. Tracking and reconstructing the 3D pose and geometry of multiple hands in interaction is highly relevant for human-computer interaction applications, including AR/VR. Ur doesn't depend on the posemesh and can be used alone if you are only interested in a hand tracker.