Skip to main content

Babble Tracker Quickstart

This guide applies to the Official Babble Tracker, and should cover DIY versions that use similar hardware/firmware. It will walk you through the complete setup process, from initial connection to calibration and troubleshooting.

Getting Started​

Initial Setup​

  1. Plug your tracker into your PC
  • Connect using the USB port on your tracker
  • Make sure the switch is in the "ON" position, as seen below
  • Two bright green lights should appear on the back of the tracker
  • You should hear a connection sound when properly connected

Switch Position

  1. Finding your device's name
  • Windows
    • Press the Windows key, type "Device Manager" and press enter
    • Expand "Ports (COM & LPT)"
    • Look for a "USB Serial Device" entry - this is your tracker's COM port

Device Manager

  • macOS/Linux
    • On macOS/Linux, the Babble Tracker with appear as a device with a /dev/ttyACM prefix . You can list what devices are present with ls -ld /dev/ttyACM*.

Download and install the software​

Connect to your tracker​

  • Open the Project Babble software
  • Click on the dropdown next to "Camera Address"
  • Select the namely identified in step 2
  • Click "Save and Restart Tracking" to connect
info

If your camera isn't present in the dropdown, just type it in.

Camera Address Selection

Setting Up The Camera Feed​

Before use, you need to properly set up your camera feed:

  1. Ensure proper positioning
  • Your face should be evenly and well-illuminated
  • The tracker should be able to see your entire lower face and jaw
  • Positioned at a slight angle that shows the tip of your nose/nostrils
  1. Set the crop area
  • Click the "Cropping Mode" button

Cropping Mode Button

  • The feed may appear choppy in this mode - this is normal
  • Click and drag across the image to set the crop area
  • Make sure the cropped area includes:
    • Tip of your nose/nostrils
    • Bottom of your jaw when open
    • Sides of your cheeks when puffed

Proper Crop Example

  • Don't worry if the image is flipped - mirroring options are available at the bottom of the cam tab
  1. Switch to tracking mode
  • Once you've set your crop, click "Tracking Mode"

Tracking Mode Button

  • You should now see a smooth video feed from your camera

Working Camera Feed

Calibration Process​

Proper calibration is essential for accurate facial tracking:

  1. Navigate to the Calibration tab

Calibration Tab

  1. Perform Neutral Calibration
  • Set the calibration mode to "Neutral"

Neutral Mode

  • Return to the Cam tab
  • Check "Enable Calibration"

Enable Calibration

  • Relax your face into its natural resting position
  • Click "Start Calibration"

Start Calibration

  • You'll hear a sound indicating calibration has started
  • After a few seconds, click "Stop Calibration"

Stop Calibration

  1. Perform Full Calibration
  • Return to the Calibration tab
  • Set the calibration mode to "Full"

Full Mode

  • Go back to the Cam tab
  • Click "Start Calibration"
  • After hearing the start sound, make as many facial expressions as possible:
    • Smile, frown, open mouth wide
    • Puff cheeks, stick out tongue
    • Move jaw side to side
  • Tip: Keep movements natural - don't exaggerate expressions
  • Calibration ends when you hear a different sound

Fine-Tuning and Troubleshooting​

If some expressions work better on one side than the other, or you need to adjust tracking sensitivity:

Understanding the Calibration Tab​

Calibration Panel

The calibration panel allows you to manually adjust tracking values:

  • Left/Right: Indicates which side of the face the shape corresponds to

Left Right Options

  • Shapes: The specific facial movement (left and/or right) being tracked (Mouth Open, Cheek Puff, etc.)

Shape Names

  • Min: The threshold at which the model starts detecting a shape

Min Value

  • Max: The maximum value the model detects for the shape

Maximum Value

Adjusting Values​

  • For proper tracking, minimum values should generally start at around 0.1
  • Example with CheekPuff:
    • Setting Minimum to 0.4: You need significant cheek puffing before detection
    • Setting Maximum to 0.1: Slight cheek movement registers as full puffing

Fixing Common Issues​

If your jaw appears to lean in your neutral position:

  • Adjust the minimum value for JawLeft until it's no longer detected when your face is at rest

Pro Tip: For precise adjustments, open your VR application's debug menu for the mouth tracking asset. Watch how values change as you make expressions, then return to the calibration tab to fine-tune.

Maximum Value

Next Steps​

Your Babble Tracker is now set up and ready to use! To get started with specific VR platforms:

WiFi Capability​

Want to use your tracker wirelessly? Check out our Firmware Guide for instructions on enabling and configuring WiFi functionality.