All about AR on F8 2019 Conference

As all the videos from the conference are available at Facebook Developers Website, the best thing to do with this article was to keep it short and simple – just to point out everything new that had been announced, share some overall thoughts on and from the panels and add some references for what’s worth watching for whom.

I. Coming now & coming soon to Spark AR

The most immediate and noteworthy change – especially for creators – is the brand new Spark AR Studio for both Windows and MacOS which became available during the first day of the conference. It was brought to the Spark AR Community to make AR “more accessible than ever before” (for both creators and users) and as a tool making experimentation easy – for everyone. The new UI, updated Patch Editor and other new functions are meant to “layer complexity”, providing an “interface for everyone” with three levels of the difficulty of creation: (1) scene tree & inspector, (2) patch editor, and (3) scripting.

The new app, rebuild and rewritten from the ground, was based on the experience of the developers and the feedback from the community. Luckily, it answered the main concern expressed on the Spark AR Community Facebook group almost every day: when will it be available for Windows? From now on, Spark AR Studio for Windows is no longer beta but the same app for both platforms.

Probably the second most frequently asked question in Spark AR Community is about getting the access to Spark AR for the Instagram beta program. It has been announced that in the summer Instagram beta and the possibility to publish branded AR effects will be open to all creators.

Another move towards democratizing Spark AR Studio creation experience is an introduction of Spark AR Guidelines which are about to make it even easier to start your adventure as a Spark AR creator and keep AR experiences consistent and thus more intuitive.

Besides a completely new look, probably the most impressive novelty are Blocks – an entirely new concept which will let users organize, re-use, and share parts of their projects – dramatically simplifying what their scene patch will look like. Blocks will be one of the important factors for the announced development of AR shopping, making it a lot easier to create dozens of similar “try-on” effects with different products, and thus a constant layout and better shopping experience.

Other advancements in Spark AR significant for using AR to shop through Instagram and Facebook are new, powerful and easy to use Photorealistic Makeup Shaders and Native Picker API (which will enable to render AR effect’s UI in the client, making the effect lighter and thus more accessible for low and mid-range devices).

Also, AR music is coming to Spark AR soon: new audio controls and functionalities which will let you enhance your camera effect with music and sounds (uploaded or chose from Facebook Sound Collection). What’s more, there will be an option of syncing the beat of the track to the visuals of your effect and trigger sounds by moving your head or hand or using face recognition. For those who want to know more, see AR music.

The other capabilities mentioned to be coming soon are:

• Person and Hair Segmentations
• Blending virtual objects into reality seamlessly, hiding virtual objects behind the real ones
• Updated Spark AR Player with the capability of profiling on devices (testing effects on different phones) and other advance review options like setting FPS, real-time debugging and real-time performance graphs
• Lightning Simulation which will enable to create particular sceneries like daytime or nighttime
• Advancements of Spark AR for Places and Spaces, like vertical plane tracking, semantic scene understanding or tags – deep links into effects from the Facebook, Instagram or even stock camera.

• Multi-Effect launcher
• Command-line interface
• Updated image-based lightning
• Scannable entry points combined with Instagram profiles. They were used for the first time during F8 in scavenger hunt game created by Spark AR Developers
• Updated Hub with changes to metrics over time, creator-level metrics and captures and shares for Instagram.

It is a well-known fact that AR is going to go beyond smartphones, and as we are probably a few years ago from widely used AR wearables, Facebook is trying to connect people in a new ways even now: their last year’s device, Portal, is coming to Canada in June and to Europe in the fall. There are various ways in which Portal is using AR to bring friends and families together, two of which were mentioned at the conference: AR games and Story Time. Also, the main difference between Portal and any tablet is a smart camera technology which keeps track of people present on the screen and follows them around.

II. Wrapping up the rest: some thoughts and references

Above, we’ve pointed out what news relevant to Augmented Reality had been brought to us during F8 2019. But the two-day conference was not just about the announcements: it was also about meeting fellow creators, discussing previous experiences and learning the new ways to develop unforgettable Camera Effects.

The Community

The community was the most often used word during roundtable panels and discussions. It has been presented as the main power behind Spark AR success, which materialized in getting over 1 billion unique camera effect’s users from its launch in 2017. According to the panellists from Spark AR Creators Roundtable, the community helps software to grow by reporting bugs and suggesting changes, gives you the confidence that you’re not alone and you can always ask for help, and finally it is a constant source of inspiration.

Instagram is the future

Instagram was named ‘the place’ to publish AR experiences, which was actually very easy to predict because of its pictorial nature. I’m starting to think that maybe it’s the good thing that, apparently, AR will not come to Messenger after all (but we can’t be sure): it looks like Facebook has a plan concerning the future of its family of apps and a part of that plan is making Messenger light and text-focused again. The only concern is integrating AR with Chatbots which has proven very good for businesses. You can read more about this in our two-part article about using AR together with Messenger Chatbots.

The main Spark AR star at F8 2019 was Johanna Jaskowska (@johwska), who was presented at the keynote and who shared her experiences from working with Spark AR Studio – experiences especially valuable for those who like to experiment and for those without coding knowledge who wanted to create AR content.

Maximizing the Spark AR Engine for Everyone and Optimizing Spark AR Effects for Instagram are must-watch panels for all Spark AR Creators, full of hints concerning effect’s optimization for them to work well on any device and engage as many users as possible.

Pro Tips for Creating in Spark AR Studio panel consists of two interesting examples of creating glasses and an animated poster. Spark AR for Shopping panel brings some really great guidelines on how to create AR try-ons which will be friendly for the user and can be used by creators to show potential clients why using AR in online shopping is the right thing to do. Creating Standout AR Effects for Instagram lecture is full of brilliant remarks on how to make Instagram Effect popular and meaningful. According to Cemre Güngör, the effect to be successful has to:

• serve a real need
• get out of the way – making it possible to use it many times
• not share ads or brands, but affinities
• be easy and fun
• be outstanding, special

Summing up

This year’s F8 2019 was another groundbreaking event for Spark AR. For one, it clearly showed that we’d moved forward on the path to the great social media AR experience. Also, it successfully addressed the most crucial concerns of the community of creators, like Windows app or Instagram beta accessibility, and announced a few exciting new functionalities. F8 2019 was about Spark AR getting better, more polished, reliable and easy to use. Finally, it has been an indicator of how fast the community has been growing, especially from the moment Instagram beta had started, and how much impact it had and will continue to have on the software and on the Instagram audience.

Waiting with the best stuff till the end, we would like to present to you a video commentary to this year’s F8 conference, recorded for our readers by our friend, Piotar Boa:

Below you’ll find links to all of the videos from F8 2019 concerning AR. Enjoy!

Spark AR Outlook: Inspiring Creation and Exploration
Speakers: Michael Slater / Facebook, Matt Hanson / Facebook

Spark AR for Shopping
Speakers: Andrea Alam / Facebook

Pro Tips for Creating in Spark AR Studio
Speakers: Dan Moller / Facebook

Spark AR Creators Roundtable
Speakers: Chris Barbour / Facebook, Johanna Jaskowska / , Tomás Posse / , Marcos Trinca / MoreThanReal, Snehaal Dhruv / Superfan.ai

Creating Standout AR Effects for Instagram
Speakers: Cemre Güngör / Facebook

Maximizing the Spark AR Engine for Everyone
Speakers: Merlyn Deng / Facebook, Nikita Lutsenko / Facebook

AR Music
Speakers: Tom Goldsmith / Facebook, Dren McDonald / Facebook

Optimizing Spark AR Effects for Instagram
Speakers: Houman Meshkin / Facebook, Anaelisa Aburto / Facebook

Storytelling Across Pioneering Platforms
Speakers: Yelena Rachitsky / Facebook, Maureen Fan / Baobab Studios, Mark Walsh / Motional.ai, Greg Reed / Universal Pictures, Jessica Yaffa Shamash / Fable Studio

by Maciej Szulim