New Update from Snap – Lens Studio 3.4 is Here with 3D Body Tracking & More 🤩

On Thursday, Snap launched the new release of Lens Studio. It includes a few big improvements, both when it comes to the new or upgraded functionalities and to the workflow – thanks to the introduced Assets Library.

Learn about what’s new and read what Creators had to say about Lens Studio 3.4 👇

Improved Hand Tracking

Focus your Lenses to specific parts of a hand or let them react to unique hand movements. Improved hand tracking gives you 24 tracking points that allow you to select any joint within the hand for your Lens to react to, rather than just the hand itself.

Learn more about Hand Tracking 👉

Below you’ll see one of the first Lenses with the improved hand tracking by Noland Chaliha aka alwayscodingsomething as well as a Lens from before the update by Jonah Cohn.

Full Body Segmentation

Put your whole body into it! Build experiences that react to full body movements or isolate a person from their background completely. This new upgrade allows you to easily track and segment full bodies in your next Lens.

Learn more about Body Segmentation

Denis Rossiev shows how the new Full Body Segmentation is a step forward:

Before the 3.4 release, Portrait Background segmentation actually worked for the full body, but was glitchy and unstable. But even despite this, using some shader tricks, it was possible to make exciting effects: Neon Line, Fire And Flame, Spectral.

With a brand new Full Body Segmentation we can bring effects like this to the next level.

Just look how stable it is!

And it works not only for the full body: but also for your hand or leg!

P.S. I have to warn the creators: since it’s a new feature, wait a bit before publishing effects! A lot of users still didn’t update the app so new effects may not work for them.

3D Multi-Body Tracking

In September 2020, Snap presented 2D body tracking to the world, and a month later, just before Halloween, they launched 3D full body tracking, though it was not available to everyone. Now you can build 3D full body Lenses for multiple people, allowing more Snapchatters to get in on the fun you create.

Learn more about 3D Body Tracking

Before we go on to the freshly new Lenses, let’s remind ourselves of what Body Tracking Lenses created up until the new release:

Recently, we had an interview with Helen Breznik. Because of the amount and the quality of her Body Tracking Lenses, we named her a Queen of Full Body Tracking 👑💃

Atomic Digital Design shared with us his opinion on what the new update means for digital try-ons:

This new Lens Studio update, with improved hand and body tracking, will take digital try-ons to a whole new level. Luxury, fashion and retail brands will be able to better showcase their products: such as jewels, clothes, shoes and even bags thanks to this advanced technology. We can’t wait to use it!

Atomic Digital Design Team, as well as some other Creators, managed to produce the first 3D Body Tracking Lenses since Thursday release:

Asset Library

Discover new shortcuts while working within your projects. The all-new Asset Library is a collection of 3D models, materials, ML models, scripts, presets (and more!) that are easily accessible to drag and drop into your creations. It features about 30 assets from 10+ Lens Creators, including ML models from Fritz.AI — and involving the community right from the beginning.

You’ll be able to speed up your workflow by saving your assets or even get a jumpstart by using pre-made assets from other creators’ saved in the library.

What you’ll find in the Asset Library:

• 3D: 3D models you can add to your Lenses
• 2D: Textures, hints, masks, and more
• Materials: Custom shaders that changes the look of any mesh in your scene
• Tools: Helper scripts that adds capabilities to to your Lens
• VFX: Custom shaders and effects that change how your Lens looks.
• SnapML: New models like household object detection, face covering classification and more.
• Audio: SFX and background sounds.

Learn more about Asset Library 👉

Here’s what Dan Abdinoor, co-founder and CEO of Boston-based Fritz AI had to say to Snap team:

We are excited for the Asset Library and even more pleased to be a part of the very first assets available at launch. It also makes perfect sense to us that Lens Studio would add the Asset Library, giving Creators a seamless way to use some of its most powerful features right inside the application. This is an important step towards accessibility and ease of use for AR/VR.

As a Machine Learning company, we know how important it is to educate Lens Creators on what tasks ML can perform, and how they can be used creatively in Lenses. So we wanted to provide some illustrative SnapML models that will help Creators learn what’s actually possible. We believe this knowledge will lead to flourishing creativity, and Creators will want to make their own custom ML models with Fritz AI.

We believe that showing Creators what ML models can do in the wild is incredibly important. This will inspire Lens Creators to take it to the next level with custom SnapML model training, and Fritz AI makes that process quick, easy, and no-code, without becoming an ML expert.

Luka Lan Gabriel (check out his Lenses) also send us a few words of his own commentary on Asset Library:

In the latest Lens Studio release (3.4) there is a new feature called Asset Library. It is a repository of assets by both the creator community and Snap’s internal team. The Asset Library is divided into 7 groups of assets, namely: 3D, 2D, Materials, Tools, VFX, SnapML, and Audio. The assets can be easily added to any new or existing projects. This can both spark new ideas and speed up the creative process.

In my opinion, this is the best long term addition to Lens Studio. Specifically, sharing creator-made assets allows beginners to jumpstart their process of creating their first Lens while also allowing established creators to easily spice up their work with high quality assets.

Creating in Lens Studio is essentially like playing at a communal sandbox while adding the Asset Library means suddenly having the option to use everyone’s new and exciting tools, allowing you to produce some amazing creations.

And Lens Creator Max van Leeuwen added:

The most exciting thing to me is seeing what others are creating, especially when it comes to ML. The Asset Library is a big step towards making the Lens Studio software all-encompassing. When it comes to reusable assets like hints, noise textures, modular pieces of code or sometimes even 3d models or audio, it cuts out the middleman of having to open another software or download something online.

I often try to make tools that automate things I have to do regularly by hand. Sometimes I post those tools online, but I never really know where people can actually find them. I think the Asset Library is the most logical place for others and for myself to quickly access them, which hopefully saves them valuable time.

I like how importing from the Asset Library works with just one click of a button, since the assets are all ready-made and clearly structured. This makes it a big time-saver, which I really appreciate.

But besides efficiency, I think what is most exciting is the opportunity to see what others are making, and use that in your own projects. It will be a unique overview of current trends in AR, and it will make learning complex techniques a lot faster since you get to see how others are solving similar, specific problems.

We’re very excited about the next few weeks because of all the new Lenses that will take advantage of this new Lens Studio release. Be sure to visit Lenslist and follow us on Snapchat 👻