Dynamic Perspective

At Amazon, we built a groundbreaking computer vision-based head-tracking technique called Dynamic Perspective, powering an exciting 3D interface (requiring no special display or glasses). Due to its limited resolution (200x200 pixels) and frame rate (8 fps), the camera module introduced jitter (random motion in 2 dimensions) and lag (delay between motion and the reaction of the interface). Signal processing techniques improved one impairment at the expense of the other (e.g., smoothing reduced jitter at the expense of lag). Trying different settings with users had not resulted in meaningful progress, and beta testers complained about the quality of the experience.

I built a user-test protocol using a tiara-like device with multiple LEDs, which mimicked an ideal sensor by making it possible to track the position of the head with unnoticeable jitter. After introducing the precise amount of lag and jitter (and minimizing the number of combinations by analyzing the users’ ability to distinguish between different amounts of impairments), we created a preference curve indicating the precise tradeoff limits beyond which users could identify a degradation in the experience.

Of all the critiques I read about the device (and there were many before the product was pulled), I did not find one that spoke negatively about Dynamic Perspective’s performance.