Exclusive: Try on your team’s World Cup jersey with Snap’s new AR filter

During the World Cup, it’s not always easy to change allegiances. The quadrennial international soccer tournament, which starts November 20, is watched by billions of spectators around the planet. For those with connections to any of the 32 teams in play, the gradual knockout of national teams leaves most fans reluctantly settling for backup favorites as the field narrows.

Now, for a few select teams, it will be much easier to switch loyalties, at least in the augmented reality app Snapchat. Snap, the company behind the app, has just released a feature called Live Garment Transfer, which allows users to virtually wear the jerseys of six national teams competing in this year’s tournament. In partnership with teams from the U.S., France, Saudi Arabia, Belgium, and Qatar, Snapchat users can overlay a 3D version of those teams’ jerseys on themselves in real-time video within seconds. If and when the heartbreak hits fans of any of those teams, they’ll have at least a digital way to support a backup squad.

Live Garment Transfer is a clever bit of fan engagement during the world’s most watched sporting event, and likely the first of many paid partnerships Snap hopes to translate into the revenue it has been struggling to attract recently. It’s also an indication of where augmented reality itself is heading.

Carolina Arguelles, Snap’s global AR product strategy and product marketing lead, says Augmented Reality Lenses have historically been used as a tool for creativity. Originally launched to allow Snapchat users to overlay AR content into their snaps, Snapchat Lenses were many people’s first taste of augmented reality. In 2017, Snap launched Lens Studio, an application developers can use to create and publish augmented reality experiences for Snapchat users.

The technology to create the Live Garment Transfer Lens has been in the works for years. Since 2017, Snap’s researchers have published more than 20 papers outlining the fundamentals of this type of real-time image-to-video processing and the neural networks behind it. Development of the feature took two years, with hundreds of thousands of hours of image and video training behind the four neural networks that allow the static image of a soccer jersey to drape itself over moving video of the human body in all its varied forms, wrapping to the arms and torso and erasing any traces of the clothing beneath it. By analyzing every pixel of any video using Live Garment Transfer, the technology can segment out each body part, from the shoulders to the belly to the hair, and lay the digital jersey just where we’d expect a shirt to be. For the World Cup teams partnering with Snap, the technology gives fans a deceptively simple novelty.

Most AR experiences up until now have relied on complex digital models that apps like Snapchat layer on top of 3D space. This new feature enables users to upload a simple image and the technology transforms it into a kind of digital garment that can wrap around a body in space, replacing what the user is wearing in the liver camera feed. “That’s really the foundational shift here,” Arguelles says.

Ultimately, it’s a technology that could help Snap and other AR technology companies turn clever digital overlays into long-term revenue streams. Arguelles says there’s obvious application in the world of clothing retail, and Snap has previously created clothing try-on Lenses, but they have previously focused on turning static images of clothes into static images of those clothes on people. Turning the outputs into video is still in its early phases, Arguelles says, and it works best with fitted upper body garments like shirts and hoodies from the front only. But she expects that the technology will soon be able to handle clothes on the entire body, front and back.

Snap’s technology could make it easier for consumers to try on clothing, and potentially lucrative for retailers to sell more of their clothing directly to consumers without the need for physical stores. It could also become a tool for deciding what products these companies bring to market. “Being able to prototype early and get immediate customer feedback to inform things like true manufacturing, I think this is definitely an interesting technology path,” Arguelles says.