Developer Spotlight: Amy Erz on Location-Based Apps for Meta Display Glasses
With the opening of development for Ray-Ban Display, Meta has contributed to a new chapter in wearable technology, where developers play an active role in shaping the platform and defining the use cases that will influence everyday life.
As the ecosystem around Meta AI Glasses continues to grow, developers are now exploring how location, movement, and lightweight interfaces can be used to create new types of experiences that blend digital content with the real world.
Amy Erz is one of the early contributors to exploring this area, building location-based experiences (SceneIn, Apple50, Byte Trail) for Meta Ray-Ban Display.
In this interview, Amy shares her experience working with the platform, her approach to building for a new device category, and how new tools are supporting independent creators in the space of wearable computing.
Can you tell us a bit about yourself, the kinds of apps you create, and what drew you to building for Meta Ray-Ban Display?
I come from cartography, so I’m used to places being points on a map. The glasses flip that. Everything I build is tied to a real location and only works when you’re physically there, and that’s deliberate.
Things like SceneIn, which pins movie scenes to where they were filmed in San Francisco; ByteTrail, a walking tour through the history of the Valley’s biggest tech companies; Apple 50, a WWDC companion built around Apple’s history in Silicon Valley; and a HUD for a BTS concert at Stanford.
What drew me to the glasses was the chance to build them as webapps. After all that time on the web, here was a brand-new device I could make things for with the skills I already had. What excites me most is that the glasses go where you go. I come from cartography, where you spend your time turning real places into dots and lines on a map.
The glasses reverse that: the place stops being a point on a map and becomes something you’re standing inside, with the story layered right onto it. I wanted to dive in and find out how easy it is to build on a new platform, and whether everything I knew from the web could work out in the world, not just on a screen.
Scene in San Francisco maps movie scenes across the city. Can you tell us where the original idea came from, and why you felt it could work well as a smart glasses experience?
SceneIn actually goes back to my very first Ruby on Rails app, about a decade ago. I was looking for public APIs to build with, and I wanted to combine my GIS background with open city data.
This was right when web geocoding was the technology of the moment, so mapping things to real locations became a project I kept coming back to. Movies were a natural subject for me, too: I studied at UCLA, close to Hollywood, and worked in the film industry briefly, so there’s a hidden love baked into this one.
Fast forward to AI and Claude Code. When I first started vibe coding, I wanted to see how far the technology had come, how it compared to building the same kind of app by hand back then. Scenein was the first app I rebuilt that way. So when Meta announced the display glasses ran webapps, there was no question Scenein would be one of my experiments on the platform.
The reason it belongs on glasses is the difference between standing somewhere and pulling it up on a phone or a desktop app. Having the scene right in front of your eyes while you’re standing on the exact spot it was filmed is a completely different feeling: the old clip overlaps the real place as it is today, and you see how time flies and how much the spot has changed.
And experiencing it on a screen indoors is nothing like going out and living it on the spot: the smell and the breeze, the city moving around you. That’s what a screen at home just can’t give you.
You’ve been vibe coding these apps for a completely new device category. What has that process been like, and what are the trickiest parts for you when turning an idea into a working smart glasses app?
The process is fast and a little addictive. I can go from an idea to something running in the glasses in a single sitting, and that quick loop is what keeps me building. The hardest part is being an early adopter. There aren’t many best practices to reference yet, so I lean on the official documentation and Meta’s own code samples on GitHub.
Claude Code let me spin up my first app in about five minutes, which is wild, but the real work is everything after that.
My workflow has actually changed along the way. Before Claude Design, I’d jump straight into code; now I spend most of my time in design first, then pass the handoff file to code for the technical details.
Either way, I wire in Meta’s developer docs as the reference so the build stays grounded in what the device can actually do, and I keep a CLAUDE.md file, basically a running notes file that keeps the AI on track.
Connecting everything to GitHub has been key too: it lets me deploy, pushing updates live, straight from my phone, so I can be out on-site, fixing issues on the spot and checking the result in my glasses in real time.
SceneIn San Francisco can be experienced on phones, computers, and now Meta Ray-Ban Display. After adapting it for glasses, did anything about the experience surprise you or change the way you think about the original idea?
The UX, specifically navigation. Meta Display webapps are driven mostly by the Neural Band, which maps to arrow keys and a select button, so that became the user’s primary way of interacting. That changed how I had to think about Scenein. I pared it back, turning off the YouTube player’s own controls and keeping only the interactions that mattered.
It pulled me straight back to accessibility work on the web: thinking about how someone gets through a page with only a keyboard, on a screen reader, navigating tab by tab. Designing for the glasses meant deliberately eliminating interaction rather than adding it. Less on the page, not more.
It also makes the flow of getting through the app matter more than ever. When you’re navigating with arrow keys, every extra step has a cost. Will someone have to scroll past twenty-plus items just to reach the next button? Keeping that path short is half the design work.
It’s even changed the order I design in. In web development we used to go mobile-first, then expand up to desktop. Now I find myself going glasses-first: I design for the smallest, most constrained surface first, then let that same webapp open up into the fuller mobile and desktop experiences. Starting from the tightest constraint makes everything downstream cleaner.
ByteTrail and Pokémon GO for Apple Fans both turn real-world locations into something people can explore, but in very different ways, from tech history to playful discovery. What do you find most exciting about this kind of experience on smart glasses?
ByteTrail is a walking tour guide for Silicon Valley history. It holds a list of the dominant tech companies in the Valley, geocoded to their locations, sorts them by how close you are, and when you get near a site it surfaces that company’s history.
Apple 50 took that same idea and made it event-focused, built for WWDC week and Apple’s 50th. I gamified Apple’s historic sites, places like Apple Park, Infinite Loop, and Steve Jobs’s garage. As the week went on, it grew: I added a keynote bingo that activates right as the keynote starts so you can play along on-site, and then, inspired by the new macOS Golden Gate, I expanded it into earning patches for visiting the California places macOS is named after, like Yosemite. That idea came together live, right during the keynote, and that’s part of what I love about this work: I never know what will spark the next one.
What excites me most about this on glasses is experiencing the past at a location in the present moment: the nostalgia, and the “I was there” feeling. Our phone photos and videos already carry geotags. I can imagine our grandchildren one day standing where we stood and experiencing our past through smart glasses like these. Grandpa and Grandma were here.
After building several early experiments for Meta Ray-Ban Display, what kinds of apps or experiences are you most excited to explore next, beyond location-based storytelling?
I’m not looking to leave location behind. It’s the heart of everything I make. What I want to change is what I do with it: bringing more play into the storytelling. The one I’m most excited about is a geocaching-style game built for glasses.
I got a taste of it with the patch hunt in Apple 50, and it made me want to go further, turning a whole city into a treasure hunt where the clues and caches surface right in your view as you move, completely hands-free. Geocaching has always been a GPS game you play looking down at a phone. On glasses you finally get to play it looking up, with the real world as the board.
That sits alongside event-driven experiences too: apps that carry you through a conference, a festival, or a concert, the way my HUD did at a BTS concert at Stanford. Apple 50 was really just a starting point for this.
The same format re-skins for any event: for the upcoming Meta Connect, I could pull the Facebook-to-Meta story into the game, with a live keynote-prediction bingo running as the announcements drop. Letting people navigate a live event in real time is the other direction I want to push, and it’s the kind of thing I’d love to build with brands, agencies, events, and cultural institutions that want to be early on wearables.
But whatever the format, none of these apps would work without the location itself.
With projects like Scene in San Francisco and ByteTrail, the city becomes part of the interface. How do you think about designing experiences where the real-world location is not just a backdrop, but the main content?
I come from cartography, so I’m used to places being points on a map. The glasses flip that. Everything I build is tied to a real location and only works when you’re physically there, and that’s deliberate.
The spot becomes the canvas, so instead of reading about a place, you’re standing on it while the experience unfolds around you. The goal is always to send you out exploring the world, not pull you into a screen.
Do you think AI-assisted coding can open up smart glasses development to more independent creators, even if they don’t come from a traditional software development background?
Definitely, and I say that as someone doing it myself. I came in with a decade of web behind me, but the glasses were still new territory, and AI is what let me cross into a platform I had no native experience with. AR and VR were new ground for me, and the community here has been incredibly welcoming, so building on the platform has become my way in.
I’m learning as much from them as from the docs, and there’s still so much to learn.
For people coming in without that background, it opens the door even wider, and that’s genuinely exciting: more apps, more voices. What I’ve learned is that AI takes care of the coding, but building something that lasts is a bigger craft: UX, quality content, and the engineering side, like version control, performance, security, privacy, and maintenance. The tools get you in, and the craft is what you build toward.
Amy’s creative process focuses on clear direction: using location not just as input data, but as the core structure of the experience itself, where content only becomes meaningful when tied to a specific place and moment.
Her work illustrates how the earliest Meta Ray-Ban Display experiences are not only about new formats, but also about rethinking how digital content behaves when it is directly connected to physical space.
This approach points to a direction where wearable technology is not just another screen that separates us from the real world, but a medium that places the real world at the center of the experience, with design starting from physical context and building outward from it.
Want to bring AR into your marketing? 🔍 Use the AR Brainstormer to generate quick ideas, explore the AR Search Engine for inspiration, or book a consultation with us – we’d love to help!
🧑🎨Creators can also showcase their work on XR Bazaar and build a standout portfolio in the XR industry. Get discovered by brands and agencies looking for fresh talent in creative tech 🛠️
