Elliptic Labs makes ultrasound-based virtual sensors on consumer electronic devices. Early engagements with smartphone OEMs required that the signal processing experts be onsite to tune parameters. Manual validation necessitated hours of repetitive motions to see if the intended performance improvements materialized.
After I joined in 2014, I hired a team of DevOps and application engineers to build an infrastructure whose primary goal was to remove the guesswork out of parametric tuning. After the completion of this infrastructure, we used an Android app that delivers data collection instructions to a user; the phone sent the recorded data, metadata, and the ground truth, the latter of which was obtained via an external sensor. The acoustic data was fed to the same signal processing libraries and machine learning model that ran on the phone. A comprehensive list of experiential metrics (see case study) was computed and stored in a database. A web app was used to query the metrics database to make decisions about which model to release. Data collection was only needed once per project and was performed by junior personnel at the customer site (requiring no involvement from the engineering team). Heuristics caught most of the data collection errors, and a web app made it easy to perform a quality review before new records are committed.
After its completion, this infrastructure let field engineers autonomously conduct projects end-to-end with no involvement from the experts who designed the algorithms, allowing the latter to focus their efforts on new platforms and new experiences.