What’s New for Instagram AR Effects After 2021 F8 Refresh

New capability #1: Multi-Class Segmentation 👥

Segmentation feature lets you identify and separate parts of the image the camera receives. There are 3 types of segmentation available: person segmentation, hair segmentation and skin segmentation. Now, you can combine them in the same effect to achieve very complex augmentations. The new feature provides Creators with a huge flexibility as it also makes it easier to control multiple texture properties.

The result? Much more natural and real-like full body AR experiences. Remember though that MCS requires good lighting and even better phone to work well 😊

Source: tech.fb.com

It may sound easy, like we simply mashed up three models into one, but the reality is much more complex. Reliably identifying a single segment is hard enough — for instance, making sure that a ponytail draped over a shoulder is seen as “hair” while the tank top strap on the other side is not. MCS is doing that three times, and delineating where hair and skin and clothing overlap so that creators can design effects to make your skin glow like a lightbulb or turn green without having it bleed onto a hat or shirt.

From: tech.fb.com

 
Catch the impressive examples by Piotar Boa and Denis Rossiev:

🔗 Links:

• Tech@ blog post: Three-part harmony: Spark’s new Multi-Class Segmentation tech enables more realistic and complex AR effects
• Spark AR Team blog post
• Updated Spark AR documentation

New capability #2: Updated Target Tracker 💢

Target Tracker triggers an effect when the camera is pointed into a certain 2D object, like a logo or a poster. This feature has been used countless times in very creative ways, eg. by brands to trigger the effect by pointing the camera at their product packaging or by Creators to reinterpret art and bring it to life.

Now, target tracker becomes a much more exciting tool to create memorable, interactive experiences thanks to these changes:

• Moving target – the target doesn’t have to remain in a fixed place anymore: when you move the target, the effect moves with it 👣
• Multiple targets – up until now, an effect could remember only 1 image as a tracker. Now, it can be triggered by up to 5 different images 🖐

💡 Additionally, the Creators are encouraged to update their AR effects that use target tracker for performance improvements.

Check out these 2 effects by Ben Ursu and Denis Rossiev to see the new, multiple-image and moving target tracker in action.

🔗 Links:

• Spark AR Team blog post
• Updated Spark AR documentation

Spark AR Creator Awards 🏅

Just introduced Creator Awards are the automated badges that Spark AR Creators will receive from now on, based on your achievements, eg. the amount of effects published. They can be found in a new section of Spark AR Hub. By default, they are visible only to you, but you can make them public on your Spark AR Portfolio.

Awards available now

🔥 Active Creator – published an effect in the last 28 days? You’re an Active Creator.

🎂 Years with Spark AR – this Award updates every year after the day you started your adventure with AR effects.

💯 Proud Publisher – earn this Award when the number of your effects reach certain milestones.

💗 Trendsetter – earn this Award when your total AR effect impressions reach certain milestones.

💎 SPN Member – earn this Award by becoming a member of the Spark AR Partner Network.

Learn more about the Awards here. And behold – as Spark AR Team promised, more Awards are coming later this year! ✨

Effect collaborations 🤝

Every Spark AR effect has an effect owner. Also, to help Creators and brands cooperate, effect owners can add AR managers who are enabled to manage specific effects.

Now, AR managers can add COLLABOLATORS to give them credit for their artistic work.

Collabolators can showcase the effect they have been assigned to in their Spark AR portfolio. They won’t be able to see the effect’s insights, edit nor delete them, though.

🔗 See more info and the guidelines for effect collaborations

Multipeer API for AR Video Calling (Beta) 🤳

Now, every Spark AR Creator can apply to participate in beta program of the feature available now in Messenger and coming soon to Instagram and Portal.

The concept of Multipeer API is based on an idea of an AR experience shared across multiple devices.

Spark AR presented 3 use cases of Multipeer API as an inspiration for those of you who’d like to add their input into building the future of AR:

Shared Context – give a feeling of shared space, much like when people are physically together.

Shared Moments – use AR to let close ones celebrate together, even when they are far away.

Shared Games – build AR games that can be played together!

Learn more about Multipeer API, watch the videos and apply for beta here ✨

Thank you for being with us! If you’re interested in what we publish, stay in touch via Instagram, Facebook or Twitter.