I had already established a roadmap for our internal tools to integrate machine learning, but I discovered a need to clearly broadcast the vision for how AI will work in our product. This could be considered a sort of product vision statement – north star for internal stakeholders and an easy to understand conceptual description for others (investors, partners, etc.).
Video
The video portion is an approved clip from a partner. We run our object recognition system on each frame of it, producing bounding boxes and object tags. This is easily painted on each frame, giving a cool effect for each clothing items detected and tracked over time.
Characters
As the video progresses and new characters appear on screen, a new placeholder row appears for the character. It remains a silhouette until enough of the character’s face has been seen to identify who it is.
Clothes and Accessories
Similarly, when an object (clothing) is identified, a placeholder box appear next to that character’s headshot. A progress bar indicates how much of the product has been seen, allowing the system to know better what it is.
It eventually figures out: what kind of clothing it is, what brand, and the retail image (implies we know where you can buy it).
Conclusion
While it took some effort to make, the video was a success with everyone. This idea that the more it watches a video, the more frames it sees, it can progressively make better/more-certain conclusions about what’s appearing on screen – it works (maybe not the same as the actual system).
