Tracking Module SDK
The documentation here references VRCFaceTracking in the v4.0.0
release state. If you want to see the latest documentation that
is reflective of active development, please consider using the
'Current' docs
Summary
VRCFaceTracking provides an external tracking module SDK that allows developers to send face tracking data to any VRCFaceTracking compatible app. Modules will send data to the UnifiedTrackingData interface that VRCFaceTracking will use to send tracking data to applications.
Tracking Module Overview
Project Setup
VRCFaceTracking uses .NET Framework, a relatively
easy and intuitive development platform. VRCFaceTracking has
the ability to load any class library (Managed .dll files) built
on the .NET Framework platform using a built-in module loader.
To get started, create a new Class Library project using the .NET Framework v4.7.2 SDK and framework. Many IDEs provide a template to quickly get started with this environment.
Name the project after the tracking interface being implemented to
VRCFaceTracking; a general naming scheme for tracking modules is along
the lines of ExampleExtTrackingInterface.
Once the project environment is setup, include VRCFaceTracking
as a dependency (this can either be a project reference by using the
VRCFaceTracking project in your own project, or by referencing the
VRCFaceTracking binary), and inherit ExtTrackingModule to your
module's main class. This will be where all the main functionality
of the module will be implemented.
VRCFaceTracking will at this time only load your external module if the base class inherits ExtTrackingModule, otherwise it will be skipped. This is to ensure that developers are implementing a tracking interface extension for devices to use VRCFaceTracking tracking data, or otherwise.
This may be changed in the future to allow different types of modules to be loaded by the module loader (potentially those that can provide an output for VRCFaceTracking tracking data, modify parts of VRCFaceTracking, modify the UI of VRCFaceTracking, and more).
The default architecture platform should be Any CPU or x86 to build
for VRCFaceTracking.
Your project environment should be fully setup and ready to accept your tracking interface!
Tracking Module Architecture
The following API's are referenced when
writing a module. These are included with VRCFaceTracking.
public class ExampleExtTrackingModule : ExtTrackingModule
{
// What your interface is able to send as tracking data.
public override (bool SupportsEye, bool SupportsLip) Supported => (true, true);
// This is the first function ran by VRCFaceTracking. Make sure to completely initialize
// your tracking interface or the data to be accepted by VRCFaceTracking here. This will let
// VRCFaceTracking know what data is available to be sent from your tracking interface at initialization.
public override (bool eyeSuccess, bool lipSuccess) Initialize(bool eyeAvailable, bool lipAvailable)
{
var state = (eyeAvailable, lipAvailable);
//... Initializing module. Modify state tuple as needed (or use bool contexts to determine what should be initialized).
return state;
}
// Polls data from the tracking interface.
// VRCFaceTracking will run this function in a separate thread;
public override Action GetUpdateThreadFunc()
{
return () =>
{
while (true)
{
// Get latest tracking data from your interface and transform to VRCFaceTracking data.
if (Status.EyeState == ModuleState.Active) // Eye Status validation
UnifiedTrackingData.EyeTrackingData.Left.Openness = ExampleTracker.LeftEye.Openness;
if (Status.LipStatus == ModuleState.Active) // Lip Status validation
UnifiedTrackingData.LipTrackingData.LatestShapes[(int)LipShape_v2.JawOpen] = ExampleTracker.Mouth.JawOpen;
// Add a delay or halt for the next update cycle for performance. eg:
Thread.Sleep(10);
}
};
}
// Called when the module is unloaded or VRCFaceTracking itself tears down.
public override void Teardown()
{
//... Deinitialize tracking interface; dispose any data created with the module.
}
}
Building Project
Once the tracking module is fully setup and ready to be deployed, compile your binary. This binary can then be loaded by VRCFaceTracking's module system!
Simply take the compiled binary and place it in %appdata%\VRCFaceTracking\CustomLibs,
VRCFaceTracking will load any external modules in alphabetical order that are included in
this folder.
Setting up build events for your project can make it easy to iterate your project faster. Create a build event to place your compiled binaries into the appropriate destinations!