Although Snapchat is no stranger to location-based AR searches, the app's new world-centered game adds some environmental insights to the mix.
This week, Snapchat launched a new Snappables game called Scavenger Hunt. Although most Snappables games use the front camera to insert users and their friends into the game, Scavenger Hunt turns the gameplay to the rear camera.
The aim of the game is for users to find eight objects from the real world – a bottle, car, plate, cup, smartphone, TV, watch and chair – within their environment with the camera of their smartphone. As a reward for finding each object, the app personifies each visible object with an animated AR smiley face and arms. Players who find all eight object types unlock a special "Heroic Hunter" selfie lens.
The game uses real-time object recognition functionality that can identify everyday objects such as cups, mobile devices and cars. After the target object is detected in the camera view, the computer vision model estimates the size and position of the object, allowing the app to anchor AR content on the object.
In terms of real-world performance, object recognition can be lightning-fast and very accurate. During our tests, the app in most cases identified the object as soon as the target object was visible. Perhaps the most impressive was car recognition, which first spotted a parked car behind a few bushes and then started following a moving car as it drove past!
and the phone some more work to find the right angle, while a smartwatch with a fairly traditional shape and dial failed the test to be recognized as a watch. Here, however, I discovered that a single image of a watch was sufficient to meet the purpose.
Snap & # 39; s computer vision engineering capabilities are at the basis of its augmented reality technology, which goes all the way back to face recognition in its original lens tool for the front camera.
In recent years, Snap & # 39; s ambitions in the field of computer vision have focused more on understanding the environment, by adding the interactive possibilities to images and objects. in the real world. In 2018, Snap introduced Visual Search, which allowed users to recognize objects and retrieve product information from the Amazon inventory. Last year, the company renamed the "Scan" function as part of the redesigned AR Bar, because the opportunities for image recognition and recognition of landmarks are successful with advertisers such as Nike and HBO.
Although the forward-looking lenses provide fun for the selfie era, the world-focused capabilities are the basis for Snap & # 39; s final trip to smartglasses, which began unofficially with the 3D camera array in Spectacles 3.
With the ability to recognize images, landmarks and object types under the belt, Snap now has a good foundation for software that can provide insight into the environment for smartglasses that, further down the road, smartphones actually replace it for everyday computer use.