This year, as part of Next Reality 30, we are highlighting an emerging innovator in augmented reality in collaboration with Snap. In particular, we were looking for an independent developer or maker who embodies the groundbreaking spirit of AR and the ability to leverage machine learning. This year’s honoree is Aidan Wolf.
• Don’t Miss: The 30 People from Next Reality to Watch in Augmented Reality for 2020
When Magic Leap first came into the hands of developers, an informal group known as the Leap Squad soon formed around the platform. One of the members was Aidan Wolf, an indie developer who has used machine learning to publish some of the more unique mobile augmented reality apps under his name Kevaid.
In those early days, Wolf was working on an application called Project Play, a tool for 3D modeling, animation and game creation. While he didn̵
Ultimately, Project Play didn’t make its way into the Magic Leap World app store, but Wolf would eventually use the work to develop other mobile AR apps.
“It was fun on the rise, but after Magic Leap moved from consumer to enterprise, the community spirit really flared up, so I haven’t really interacted with my fellow Leapers since then,” says Wolf. “I still see the amazing things they’ve done, so I’m happy to see that the spirit is still alive and kicking, but in other places in the XR. industry and beyond. “
Born in rural Connecticut, Wolf relied on gaming to pass the time in his youth. “I come from a small, sleepy farming town in western Connecticut, and the luxury of boredom during my childhood wasted a lot of time making computer games,” says Wolf. His interest in gaming eventually evolved into game development
“I don’t have a formal background in what I do, never graduated or got a cool job, but I’ve been able to make games and apps since I was 16,” says Wolf.
Wolf started in AR with Capsule, a geolocation-based social app he developed through an accelerator program in Kentucky. After it hit beta, Capsule ran out of funding and closed in 2015.
“While that company didn’t work, the experience changed my life course dramatically and brought me to where I am today,” says Wolf. “For the past six years, I’ve been thoroughly obsessed with AR and luckily worked on some cool projects.”
One of those projects is Blue Sky Paint for iOS. The app takes the popular concept of AR doodling, but uses the sky as a canvas. Wolf wrote a machine learning algorithm that identifies and segments the sky for AR projection, with ARKit and sensor APIs to help with that.
“Actually, [the algorithm accounts for] really general things that are usually true in the world: the sky is always up, so if your phone is pointing up, it’s probably looking at the sky. The sky is generally blue (even the clouds), while the ground is usually lit directly by the sun, giving it a yellow tint, “says Wolf.” GPS and compass orientation are used, but we do have a reprojection algorithm. which redraws the drawing to match different perspectives and distances. The nice thing about using the sky is that we can get away with a lot of GPS distortion as the sky is 2.5km above the ground. “
His follow-up, DoodleLens, built on the algorithm for Blue Sky Paint and his work on Project Play, was mixed with some inspiration from another creator.
“It was a Saturday night and instead of working on my side project, I was procrastinating on YouTube to a video from one of my favorite creators, James Rolfe (aka YouTuber Angry Video Game Nerd).
The video included several cameras that Rolfe used, including a 90s camcorder with a feature called “Memory” that could burn monochromatic images into film for creating title cards and other graphics. Rolfe used it to add effects to his one-man films.
“I immediately saw the translation to AR,” said Wolf. “After lifting my jaw off the floor, I immediately broke Unity open, adjusted my Blue Sky Paint celestial segmentation algorithm to detect paper, and finally made the first version of DoodleLens in just a few hours. night before I left. to bed, and the rest is history. “
Like Blue Sky Paint, DoodleLens (formerly Doodle Cam) uses a machine learning algorithm, but instead of identifying the sky, it uses a math tool to segment the camera view based on color contrasts and crop out the areas of the darkest colors . It allows users to cut drawings, logos, text and other images from the real world, modify them in the app, and then paste them elsewhere in the physical world. Wolf has continued to add new features to the app, allowing users to create AR animations, record videos and share them with the entire social media landscape.
Wolf has also delved into the relatively new AR platform known as Snapchat’s Lens Studio. Wolf and his creative partner, Kevin Habich, were admitted to the Snap AR Creator Residency program, which offered the opportunity to learn from Snapchat’s in-house creative teams and receive some funding to develop their project.
“The Snap AR Creator Residency was great, I had never done a residency before, so I would describe the experience as a sort of ‘Lens Accelerator Program’,” said Wolf. “I actually didn’t know about Lens Studio or how Snapchat’s AR platform works, knowing that this is the future of the AR space, and I’ve become addicted to making Lenses ever since.”
The result of the residency was Glam Quest, which is now available on Snapchat. The lens isn’t your typical face or world effect, but rather an AR game with a dozen animated characters, dialogue movies, virtual environments and a hand-tracking combat system, all of which fit within Snapchat’s 4MB limit for Lens.
“With this lens, we became obsessed with erasing people’s preconceptions about what a lens might be,” said Wolf. “Now we rewind a little bit and plan to split Glam Quest into smaller, bite-size gameplay chunks in a new concept we call Adventure Cards.”
His next project, code-named Troubletown, once again attempts to undermine prejudices about AR. It is a cartoon series that unfolds in the real world through AR rather than through the confines of a two-dimensional screen. The pilot series involves Blorp, an alien and member of Beam Bois, an “intergalactic boy band”.
“Ultimately, they need your help to put on the greatest show ever, on a planet that the government is determined to make sure no one knows exist,” said Wolf. “It has a cool physical component that allows players to anchor gameplay and story moments in specific real-world locations, with the ultimate goal of blurring the lines between games and reality and letting outside people play together and experience adventure. “
* Editorial disclaimer: Snap had no role or input in the selection of this year’s honoree, the selection was made solely at the sole discretion of Next Reality’s editors and writers.