Since Snapchat turned down its acquisition attempts, Facebook has transformed Instagram with features from Snapchat, such as stories, chat, and of course, augmented reality.
With regard to augmented reality, Snap is iterating at a pretty fast pace. At the Snap Partner Summit last week, the company released version 4.0 of its Lens Studio AR creation platform and introduced new AR technology such as Connected Lenses.
To keep up with Snap’s AR arsenal, Facebook has added two new AR capabilities to Instagram through version 114 of its own Spark AR development tool: multi-class segmentation and improved target tracking.
Multi-class segmentation (MCS) allows developers to track a person, their hair and their skin and apply virtual textures to them. In example effects from the Spark AR community, MCS can make a subject invisible while keeping their clothing in view or apply pulsating neon lines that track hair and skin separately.
“It may sound simple, as if we’ve simply mashed three models into one, but the reality is much more complex. Reliably identifying a single segment is difficult enough — for example, ensuring that a ponytail draped over one shoulder is seen as ‘her’ while the tank top strap on the other hand is not,” the Spark AR team said in a statement. “MCS does that three times, outlining where hair, skin, and clothing overlap so makers can design effects to make your skin glow like a light bulb or turn green without bleeding on a hat or shirt.”
While Facebook’s Inclusive AI team advised the Spark AR team to track diversity in appearance, there are some gaps. For example, the camera may still confuse beards with skin.
Target tracking improvements give Spark AR creators the ability to track up to five different stationary or moving image targets in a scene. This allows developers to place multiple AR effects in a scene that can even merge with each other. However, the Spark AR team warns developers to consider mobile processing power.
Image tracking has emerged as a popular AR tactic for brands, so improved target tracking offers some interesting marketing opportunities. Imagine that cans of Coca-Cola and Pepsi become virtual warriors fighting it out on your kitchen counter, for example.
The Spark AR update gives its creator community some unique opportunities, but Snap has really raised the bar with its latest batch of updates. With the F8 developer conference just around the corner, Facebook may have more tricks up its sleeve.