General Questions

Be Positive and Enthusiastic: Dont Oversell yourself
Use Metrics to Describe Impact: Use values like "improved by 30%"
Show Collaboration: Highlight teamwork in the same team and cross-functional interactions.
Tailor Answers to the Role: Emphasize skills most relevant to the position

0. Tell me about yourself.

Keep it under one minute and adjust the details to fit your experience and the specific role.

What I work on:
- Started career over 10 years ago working on pipeline engineering in Anim/VFX/Games Pipeline before transitioning to
  - Tech Art for Synthetic Data Pipelines for AI
  - Product Prototyping for Minimal Viable Products for MR
Current Goal and Connection to Position:
- Working with a team on challenging problems at the intersection of Creativity and Engineering

My name is Victor Leung and I started career over 10 years ago working on pipeline engineering in Anim/VFX/Games Pipeline before transitioning to Tech industry working on

Technical Art for Synthetic Data Pipelines for AI

Product Prototyping for Minimal Viable Products for MR

I work best at the intersection of creativity and engineering and want to be at a place that has challenging problems to solve. I'm interested in this job because....

1. Tell me why you will be a good fit for the position.

Why do you want to work for X?

What are you looking for next role?

Collaborating with a multidisciplinary team on challenging work to launch an impactful product
Increasing Scope beyond individual contributor to work with cross functional partners and customers to aid product roadmap

I’m looking for a role where I can work on challenging pipelines and collaborate with a multidisciplinary team to launch an impactful product.

I’m excited about company’s work in [specific area], and I’m eager to bring my technical expertise and passion to the team. I’m impressed by X company’s innovative work in [specific area]. I admire your commitment to [a value or mission, e.g., open-source contributions or sustainability].

As someone who [your relevant strength,], I’m excited about the opportunity.

2. What happen to your last company?

Acknowledge the situation without overexplaining
- BG Checks from future employers do not show PIPs, only time of employment with previous company. This means you do not need to admit to poor performance.
Stay Positive and Highlight your Value: Shift Focus to achievements and skills you developed, and agedness to contribute to the new role and be confident.

Situation:
I was part of a company-wide re-org that I suspect ended my contract short. The Design Program Management group hired me because of my experience working as a

Engineer in MarCom during launch of PlayStationVR,

Engineer for R&D for Magic Leap and Samsung working on AR/AI

Program management and tech lead for Immersive Pavilion at SIGGRAPH 2019

Tasks:

Present and provide feedback to promising prototypes on the developers behalf to Directors and VPs across departments to aid in unifying multiple AR and AI product roadmaps into a cohesive strategy.

Inherited a deprecated prototyping pipeline for demo submission littered with manual processes from the former DPM. Revamp and automate the workflow and get the buy-in of prototypers across all divisions to use it.

Action:

Learn how the demos are developed from a technical standpoint

Establish a reputation by presenting demos at events on request, as well as leading AR Demo Summit across four sites (Redmond, Menlo Park, London, New York)

Re-architect the demo submission workflow so that it is useful

Result:
In about three months, I lead the AR Demo Summit in June which showcased prototypes across the wearables division.

Over 30 Demos selected from 60ish submissions, 1000 attendees across all sites.

A lot of employees saw Orion, Hypernova, EMG for the first time

Established a reputation amongst prototypers, lead Prototyping Happy Hour group for more frequent smaller scale events.

Understood the end-to-end submissions workflow and improved it

Established Single Source of Truth

Established protocol using generative AI for internal marketing material.

Updated documentation to be generalized instead of a new document every event. Good for Metamates

Plannin with prototypers and started initial work on how to re-architect the submissions pipeline

After the AR Demo Summit, my contract abruptly ended with the re-org announcement,

3. What project are you currently working on?

Augmented Reality: Unity AR Game with Generative AI imageTo3D and Gaussian Splats
Human Centric AI: Realtime digital humans, like open source LLM, shaders, and rigging.

I'm working on upskilling my knowledge in AR and human centric AI

From 2017-2019, I was the lead AR engineer for the official Siggraph conference ScavengeAR App, an app designed for attendees photograph 3D creatures spawned from 2D Artwork hidden throughout the conference. We had over 1000 Daily Active Users throughout the conference. The Pandemic killed the app, but with the return of in person conferences, I've refactored the app to utilize the latest tech stack.

My other specialty is real time digital humans. I have a background in Theater and Filmmaking, so I'm fascinated about solving the uncanny valley. The first part of my career I focused a lot on visuals, but with the emergence of LLM and synthetic voice, I realize that its about solving all the components.

4. How do you stay up to date with latest tech

Converting "doom-scroll" to "micro-learning" by joining tech communities on Linkedin and Reddit
- Read latest papers from such communities
Go to conferences to network and try latest hardware

I stay up to date with latest tech by making information come to me by

Joining communities on Reddit and Linkedin, which customizes my social media feed to content I like.

Those posts lead to links to latest papers, which I can try the latest repos on Hugging face or Github.

I also volunteer for conferences such as GDC, SIGGRAPH, and Visual Effects Society, where I can go and try the latest hardware.

5. Where Do You See Yourself in Five Years?

Launching Product Roadmap [features] to customers on an impactful team
Growing as an expert Individual Contributor, and making important decisions that impact the product

In five years, I see myself growing both technically, creatively, and professionally in a role that challenges me and allows me to make a meaningful impact on a successful product.

I aim to deepen my expertise in [specific area]

Continuing to work cross functionally to better aid the product roadmap at a larger scope, whether that is working more closely to customers and stakeholders, or leading a team to create the next feature for the product

STAR

Communication Questions

6. What frustrates you?

Poor communications among teams leading to longer delivery time and a poorer product.
Talk about requirements in machine learning between art and engineering dept.

Situation:
At Samsung Research, I was the first tech artist to bridge the gap between the design team and engineering team for for a video centric real-time digital human concierge product.

Task:
It took two weeks for artists to do production and post production before they can deliver to the machine learning team. Not scaleable in the long run. They want to speed up our delivery.

Action:
I setup individual meetings with artists and engineers to understand their workflow in order to narrow down our requirements. I ran the current process from end-to-end myself, and identified inefficiencies.

Artists are use to delivering data for creative studios, which include video at the highest quality raw, 12k resolution, and multiple layers. This takes a lot of time to shoot and process. I identified in code that machine learning team ultimately down rez, changed the codec, and flatten the data in code.

Due to the black box nature of a machine learning model, Artists and Engineering are never sure if its the input data's fault or the machine learning model. I built their preprocessing pipeline which generate video diagnostics and graphs which ruled out bad data on artist's end, and gave confidence to the pipeline downstream.

After improving the post production pipeline, I converted our production pipeline to better suit R&D. The bigger picture is to speed up delivery, so I had to teach myself production pipeline. I added multicam, mocap, increased network capacity.

Result:
Sped up data delivery from 2 weeks to 1 day.

7. How do you handle multiple stakeholders in crossfunctional team?

Understand the bigger agenda.
- Planning a Priority Matrix and Kanbam with my manager since they have increased scope within the company
- Make decisions based on being an expert in the industry.
- Identify the stakeholder's expertise and speak their language. Levy relationships to increase my own scope

Situation:

As a prototyper, I create presentations and Minimal Viable Products to present to decision makers.

At Meta I was a Product Design Prototyper on the Design Program Management Team in at AI/AR Wearables Division, where I had to improve and present other peoples prototypes to VPs and External Partners to aid product roadmap

At Samsung, i reported directly to the Senior VP of R&D and division CEO, and managed a design group where we worked on MVPs to present every two weeks to leadership

Task:

Work with manager, who has a higher scope, on my priorty matrix from P0 to P4 on what demos should present to what audience.

Instead I created a kanbam board that keeps track of Requested, Doing and Done Tasks.

Make good documentation for when we do this again, or when AI crawls for their LLM.

Action:

When talking to people

Be positive and acknowledge responses

Do research on the background of the stakeholder, and use analogies to explain complex ideas.

Build genuine connections through small talk but know when to be short and succinct when needed.

Worked at alot of companies, and connected with excoworkers on their needs

Marketing didn't know what they were marketing, so invited them to AR Demo Summit and make sure they try the demos and understood the product

Categorize specific communication thread as Single Source of Truth.

Result :
AR Demo Summit was success.

8. Describe a situation where you had to explain a complex idea to a non-technical person.

Know Your Audience: Understand their level of familiarity based on their role
Start with the Big Picture: Begin with the "why" and explain the purpose of the topic before diving into details
Use analogies: Relate the concept to something they already understand
Break it down to chunks:
- When explaining how an API works:
  1. "Imagine you're at a restaurant.
  2. The menu is like the API—it lists what you can request.
  3. You place an order with the server (the API), and it brings back your food (the response)."
Use Visual Aids
Avoid overly technical jargon: Instead of "distributed systems," say, "a setup where multiple computers work together to handle large tasks."
Check for Understanding and Be Patient: Don't rush and ask "Would you like me to explain another way?"

Situation:

Tasks:

Cloud computing is like renting storage and tools in a warehouse instead of owning them. Instead of buying expensive hardware, you can use someone else’s equipment and only pay for what you need, like storing photos or running applications. It’s convenient because you can access it from anywhere with the internet.

9. Tell me about a time you had a disagreement with your manager.

Tell me about a time when you had a conflict with a co-worker.

Have you ever had to advocate for using a framework?

Choose an example where advocating for something resulted in positive change.
Show persistence and the ability to influence others.

Situation:
My manager and I had differing views on what needs to be prioritized for a major project video centric real-time digital human concierge was going from R&D to Product. Like most research groups, we captured ourselves and had the data stored on the filesystem. This was fine when we needed to maintain flexibility during research, but won't work in production.

Task:

Action:

I requested a one-on-one meeting to discuss my concerns privately.

I shared past experience of implementing a Digital Asset Management system at Sony PlayStation and Magic Leap, and knew we needed on for our product.

Provided Presentation and live demo

Organized: Proper tagging and metadata allow for quick filtering by attributes (e.g., image resolution, format, or labels). Solved using one of our deprecated tagging tools

Easily Accessible: Centralized data prevents duplication and streamlines data access.

Version Control and Asset History Tracks versions of assets, ensuring changes are logged and reversible. Allows you to compare different versions of assets.

Programmatic access via APIs automate data extraction and preprocessing for your pipeline.

Scalability: Easily handle large datasets and integrate with cloud-based pipelines

Security and Permissions Encryption and secure file transfers

Role-based permissions access and modify datasets, reducing errors and ensuring auditability.

Manager wanted to keep things on budget and hitting a timeline, so we brainstormed a compromise

Used budget to hire our first Digital Asset Manager, to remove QC from engineers and artists

No engineering resources to code custom solution, no budget to pay for third party. Went for Opensource option

Result:
Became a Hiring Manager and hired a DAM.

Was the lead of all things data.

Launched our DAM solution for our Product pipeline.

Samsung was actually hacked but our data was safe

10. Give an example of a time you received critical feedback and how you respond?

Early in my career, my manager pointed out that I have trouble retaining information in the mornings, but not the afternoons, which lead to me asking redundant questions in the morning at times. They encouraged me to take the same focus i exhibit in the afternoon to mornings. On a more personal note, i didnt know at the time i was suffering from Sleep Apnea, so every morning i woke up with headaches. I solved this buy carrying a notebook around to jot everything down. A physical notebook, as intitally i used my cellphone to take notes but people felt like i wasnt listening when taking notes on cellphone. I then put it in the notes in the secure internal wiki. I also took my health more seriously and went to a few doctors appt which lead to my diagnosis. Now, not only do i take notes at work, it just became a habit to take notes in life. I maintain my own personal wiki that i maintain for skil building like cooking and working out, and i also use Flashcards at night to help me maintain my knowledge via a question answer format.

12. How Do You Use AI to Increase Productivity in Your Work?

I use GitHub Copilot and ChatGPT to learn new topics and write boilerplate code, while keeping in mind data security
Use role prompting for AI to give design feedback, like show it an animated GIF or using image2image genai for UI.
Recognize halluncinations and that AI can take you down the wrong path if not familiar with architectural thinking and good practices. AI does not know the latest APIs depending on when data is scraped, so need to refactor accordingly.

Situation:

Task

Action:

Result

I use ChatGPT to streamline coding by suggesting boilerplate code or offering solutions for repetitive tasks. This allows me to focus more on solving complex problems and refining the architecture of my applications. I also leverage input like animated GIFs and images for feedback and generating new UI based on design terminology.

Software Engineering Questions

14. Tell me about a time you solved a difficult technical problem

Focus on a technical or team-related challenge you’re tackling.
Explain how you're addressing it and what you’re learning in the process.

Technical Artist

The most challenging aspect of my current project is ensuring high availability while transitioning to a new cloud provider. We need to maintain uptime during the migration, which requires careful planning and thorough testing of failover strategies. I've been collaborating closely with the team to simulate different failure scenarios and refine our approach.

Software Engineer

The most challenging aspect of my current project is ensuring high availability while transitioning to a new cloud provider. We need to maintain uptime during the migration, which requires careful planning and thorough testing of failover strategies. I've been collaborating closely with the team to simulate different failure scenarios and refine our approach.

15. What was the most difficult bug that you fix?

Choose a bug that highlights your technical and debugging skills.
Focus on the process and tools you used to solve it.

Technical Artist

I recently fixed a memory leak in a microservice that caused intermittent crashes during peak traffic. Identifying the leak was challenging because it only occurred under specific load conditions. Using tools like Valgrind and custom logging, I traced the issue to a third-party library that wasn’t releasing resources properly. I updated the library and wrote additional tests to ensure it didn’t recur. It was a great reminder of the importance of monitoring and profiling in production systems.

Software Engineer

I recently fixed a memory leak in a microservice that caused intermittent crashes during peak traffic. Identifying the leak was challenging because it only occurred under specific load conditions. Using tools like Valgrind and custom logging, I traced the issue to a third-party library that wasn’t releasing resources properly. I updated the library and wrote additional tests to ensure it didn’t recur. It was a great reminder of the importance of monitoring and profiling in production systems.

17. Tell me about a project where you faced unexpected challenges. How did you handle them?

Focus: Adaptability, resilience, and creativity.

Building Motion capture lab asap

Problem, we thought 2d data was enough but realized we need 3d data. 3d data from mediapipe is poor
Start with rented equipment. I used my connections at Magic Leap to find the best price for data. Solves short term problem
Getting Vendor option from Real Mocap
Narrow down machine learning requirements
1. Art team didnt ask question about delivery other than deliver best quality data
  1. includes so many render passes we dont need
2. Research group take data and transcode them to smaller data for ML
Experiment with AI, off the shelf and repos
Build System and make sure the limits of the

Situation:
Task:
Action:
Result:

18. Tell me about a time you met a tight deadline.

How do you prioritize your tasks?

Focus: Time management and decision-making.
Emphasize planning, teamwork, and focus under pressure. Trust

Deadline for Leapcon and Royalshakeaspeare.

Sitiuation:
Our team was tasked with delivering a critical feature for a client demo in just two weeks.

Task:
I needed to ensure the feature was fully functional and aligned with the client’s requirements within the deadline.

Action:
I worked with the team to define the MVP, prioritized key tasks, and streamlined communication to avoid delays. We worked extra hours when necessary and conducted daily stand-ups to track progress.

Result:
We delivered the feature on time, and the demo was a success. It reinforced the importance of prioritization and maintaining focus under pressure.

19. Describe a time when you had to refactor legacy code. How did you approach it?

Situation:
In 2024, I refactored a 5-year-old AR project originally built with Vuforia and Unity for augmented reality experiences. The project was outdated and relied on legacy libraries, which no longer aligned with modern AR frameworks like AR Foundation. Additionally, the codebase lacked modularity, and maintaining or expanding features had become cumbersome.

Task:
The legacy code used Vuforia 9, which had limitations in compatibility with newer Unity versions and modern AR SDKs. Furthermore, features like image tracking and ground planes were tightly coupled, making it difficult to switch to AR Foundation. Performance was also a concern due to inefficiencies in the original code, such as redundant object hierarchies and overuse of runtime-generated assets.

Action:
I began by analyzing the legacy project to identify reusable components, such as 3D models and animations, and separated them from code that required updating. Next, I mapped out the feature set provided by Vuforia and determined equivalents in AR Foundation. I set up a new Unity project with AR Foundation 5.1, progressively integrating updated features like tracked image management and ground plane detection. To ensure scalability and maintainability, I restructured the codebase to use modular design patterns, such as decoupling AR tracking logic from scene-specific behaviors. This also allowed me to implement sprite animations and improve performance with optimized lighting settings for AR environments.

Result:
The refactored project became significantly more maintainable and scalable. By transitioning to AR Foundation, I ensured compatibility with both iOS and Android devices using a single framework. The modular design allowed for easier integration of new features, such as XR simulation, and reduced build times by optimizing texture handling. The updated app achieved better performance and provided a smoother user experience, while also aligning with current AR standards.

6000 were full conference attendees

had 50 percent download rate which is considered between mid and high success

3000 downloads, and 2000 Daily active users

20. Describe a project where you improved the performance of a system.

- Focus: Optimization, technical skills, and impact.

scavengeAR?

21. Describe a project where you improved the scalability of a system.

Renderfarm

22. Can you give an example of a time you made a mistake in your code? How did you fix it?

ScavengeAR. made everythign in HLSL.

Creating an entire Unity UI in HLSL (High-Level Shader Language) instead of using Unity's Canvas system can be problematic due to several technical and practical reasons. While HLSL is powerful for creating custom visual effects, using it exclusively for a UI introduces significant challenges that make it less suitable compared to Unity's Canvas-based system. Here's why:

1. Complexity of UI Layout and Interaction
Canvas:

Unity's Canvas system provides built-in tools for layout management, such as anchors, pivots, and RectTransforms.
Easily handles dynamic resizing, positioning, and responsiveness across various screen sizes and resolutions.
Includes event systems for detecting clicks, drags, and other user interactions (e.g., buttons, sliders).
HLSL:

HLSL is primarily designed for rendering and lacks the concept of layout or user interaction.
To recreate layout management in HLSL, you would need to manually calculate positions, handle transformations, and account for screen resolution changes, which is extremely time-consuming.
Implementing interactive elements like buttons or sliders would require additional logic in scripts, effectively recreating Unity’s existing UI framework from scratch.
2. Lack of Accessibility Features
Canvas:

Unity's UI system supports accessibility features such as screen readers and keyboard navigation.
You can easily add animations, transitions, and tooltips to UI elements.
HLSL:

You would need to manually program accessibility features, which is not only challenging but also prone to errors.
Building animations and transitions would require custom shader logic, making maintenance and iteration harder.
3. Performance Considerations
Canvas:

Unity's Canvas system is optimized for UI rendering. The engine batches and manages draw calls efficiently for most common UI use cases.
Unity provides tools like Canvas Scalers to adjust the UI for different screen sizes without extra performance overhead.
HLSL:

Writing the entire UI in HLSL would require a full-screen quad (or multiple quads) to render elements, which means every pixel might be processed unnecessarily.
Without careful optimization, this approach can result in excessive GPU usage, especially if shaders include complex calculations for every frame.
4. Lack of Unity Editor Integration
Canvas:

The Canvas system integrates seamlessly with the Unity Editor, allowing you to design UI visually with tools like the RectTransform Editor and Prefabs.
Designers and artists can contribute without needing to write code or shaders.
HLSL:

Designing a UI in HLSL would require writing code for every single visual element and interaction.
This lack of a visual editor makes the workflow slower and limits collaboration with non-programmers.
5. Debugging and Maintenance
Canvas:

The Canvas-based UI leverages Unity's debugging tools, including the Scene view and UI event system.
Issues like misaligned elements or broken interactions are easy to identify and fix.
HLSL:

Debugging shader-based UI involves interpreting pixel-level behavior, which is far less intuitive.
Small changes to the design could require significant rework of shader code.
6. Scalability
Canvas:

Unity's UI system scales well for typical 2D and 3D applications, supporting features like nested canvases, localization, and animations.
It’s easy to add or remove UI elements without disrupting the entire layout.
HLSL:

Adding new UI elements in HLSL requires modifying shader code, which can make the system fragile and error-prone.
Scaling the UI to different screen sizes or adding responsive layouts becomes a major challenge.
When to Use HLSL for UI
HLSL can still be a good choice for specific visual effects in the UI, such as:

Creating custom shaders for buttons, text, or backgrounds (e.g., animated gradients, outlines, or glows).
Implementing unique effects like holographic or glitch effects for menus.
Enhancing Canvas-based UI with shaders rather than replacing it entirely.
In these cases, HLSL complements the Unity Canvas rather than replacing it, allowing you to benefit from the strengths of both.

Conclusion
Using HLSL to create the entire Unity UI is not recommended because:

It lacks the layout, interaction, and accessibility features of Unity's Canvas system.
It introduces unnecessary complexity and performance overhead.
Maintenance and iteration become significantly harder.
Instead, leverage Unity's Canvas system for the core UI structure and use HLSL sparingly to add custom visual effects. This approach balances usability, performance, and flexibility, ensuring a more robust and maintainable solution.

- Focus: Accountability, troubleshooting, and learning from mistakes.

Technical Art Questions

23. Tell me about a time you optimized a 3D asset pipeline.

- Focus: Problem-solving, efficiency improvements, and technical skills.

24. Have you ever worked on a project where the artistic vision conflicted with technical constraints?

Performance is critical. Art is about hitting the essence, not hitting exact the concept art.

Focus: Negotiation, technical expertise, and artistic understanding.

25. Tell me about a time you implemented a tool or workflow that improved efficiency for your team.

Start with experience building multiple tools (like pyqt tools, photogrammetry, etc), but talk avbout the QC preprocess pipeline

Focus: Tool development and process improvements.

build Preprocessing for samsung

26. Give an example of a time you had to troubleshoot a rendering or asset issue in production.

Focus: Debugging and technical understanding. Learning from multiple occurances.

3D? Deadline logs, understand trends in graphs

2D? QCtools and preview diagnostics

Program Management

What’s your experience with planning and executing technology-driven experiences for live events?

Situation:
How to plan a conference from the ground up?

Task:

Action:

Before the Conference

Work with visual designer on a theme and a strong and consistent visual design to create promotional material for event marketing. Use GenAI to save time and market event using socials and onsite.

Gather data on past events, work on getting attendee/submission numbers, past submitters and feedback surveys, to anticipate turnout and work on improving experience

Work with Producers on booking event space, overflow space, and avoid conflicting events on the same day

Gathering submissions to select from

Once Selected, work on submitters requirements

Lighting, dark or well-lit

Sound requirements, quiet

Tracking requirements for MR, textured walls

Booth size, walking around and line

Proprietary Equipment

IT, Computer equipment and Network (Wifi or Local Network)

Safety and Hygiene, More walking space and masks

How many submitters at the booth

Work with Facilities on Table Arrangement, secure storage, and exit protocols

Work with IT and Network Engineers on Infrastructure, such as WiFi and Internal network

Work with media team on event capture and streaming

Work with Admin Execs on VIP arrangements, swag, and decoration delivery, Happy Hour

Work with Security on white list and VIP list

Conference Setup

Have contact list ready in case of emergency in all fronts (Security, Medical, IT)

Make sure secure storage is accessible

Have each exhibitor sign up for setup time slot on day prior or morning of event.

Trying out demos to ensure they are presenting what they say are presenting

During the Conference

Enforce the Timeline

Be main POC

Event photography and videography of important moments

Setup, Keynote, Peak Attendance, Attendee interaction with demos, VIPS, Closing, and Happy Hour

Line management and overflow preperation

After Conference

Breakdown

Follow up with surveys to submitters and attendees

Post Mortum with partners involved on how to improve workflow

Result

100s of submissions, 30ish selected exhibits, 18,700 total Attendees, Amongst highest in 10 years before Pandemic
100s of submissions, 30ish selected exhibit, and 1000 attendees AR Demo Summit data, largest at the time across 4 sites

What constitutes as a successful event?

Attendee satisfaction on trying demos and learning new things

Technical execution and Line management

Submitters getting feedback from attendee testing to improve their prototype

Cross functional networking post event

How would you design an interactive exhibit that demonstrates the power of Meta’s AR tools to a business audience?

Situation:

If single player,

User should feel like they have super powers, solving real world problems

If multiplayer, At Magic Leap, we followed the C3.

Collaboration: Multiple users to work together in a shared virtual or augmented environment.

Copresence: The sense that users share the same physical or virtual space, even if they are remote.

Communication: The ability for users to interact verbally and non-verbally within the experience

Task:

Build a AR mobile app that the attendees can use to lead them througout the conference

Lead them to art installations where they can try AR

Action

Build Scavenge AR. Destination spots are Experiences

Result.

Taught users about AR t,hat wouldnt normally put on headset through a device everyone has. The Phone

How would you improve audience engagement in a mixed-reality event experience?

Situation:
MR is a great experience to the user, but boring to those seeing from the outside.

Tasks:

Well designed booth in real environment, if budget allows. No fiduciary markers

Stream to a monitor

Instead of First Person View which spoils the experience, a Spectator View creates a similar experience and allows their friends to record their first time experience. Free Marketing

Have prepared video to switch to during technical difficulties

Have a handler engage with the attendees with an elevator pitch about problem trying

Have a tutorial an interactive tutorial in the experience, but allow users to view slide show version of it.

Gather immediate feedback after the demo to improve for next time

Action:

Unreal's Composure Plugin or Live Compositing Tools helped mix real and digital elements in real-time for static spectator camera

Was able to increase Level of Detail on 3D model quality on spectator cam, offload processing

Skip calibration and environment tracking by specifyin the environment beforehand, for faster experience

Result

People were posting videos online about them experiencing the experience via spectator view.

Deterred bad actors of people doing things since they are on camera.

Provided a lot of data for our User Researchers to go over to improve the experience.

Imagine core feature is unstable. How would you handle this?

Can you describe a time when you had to quickly prototype a technical solution for an event?

Situation:
Originally our Magic Leap experience was going to be utilizing two devices, one that goes through the experience with our digital human, and the handler watching the experience streamed to a monitor for attendees. We had a hard deadline for LeapCon, but the hardware features were not stable, leading to low performance, resulting in high latency and frequent crashes.

Tasks:

Redesign experience to run on one Magic Leap, but keep the spirit of the experience

Fix performance issues regarding to slow network

Have backup plan

Action:

Utilizing our exeprience in Visual Effects Virtual Production, we Use Unreal's Composure Plugin or Live Compositing Tools to mix real and digital elements in real-time.

Changed dynamic Handler headset to static higher quality witness camera connected to computer.

Lock the natural environment in place, not relying on environment tracking, spatial anchors or fiducial markers

Higher Image Quality on witness camera compared to headset

Don't rely on WIFI or even private wifi for pixel streaming, use direct connection via NDI for higher level LOD on the witness camera

Backup plan

Have a deployment system ready on site via ADB if need adjustments to static environment

Have offline rendered videos prepared on monitors and technical posters in case demo goes down

Result

Not having to calibrate headset per user made the experience quicker to demo

A lot of people posted their spectator camera experience with Mica, free marketing

Was going to utilize the pipeline we developed to future projects.

What do you do when things fail?

Describe a time when you had to troubleshoot a technical issue at an event.

What challenges have you faced when working with AR/VR/MR in a live event setting?

Situation

The demo is not working, what do you do?

Tasks

Prepare backup plans

Stay Calm and Positive and ensure audience still has a good experience

Actions

If performance issues

Sofware: Have fresh build to headset via ADB, or reboot the experience

If have access to the gamecode, be familiar with it for quick fixes. Code should be modular

Ex. Exploratorium, had to remove our character rig mapped to user because we were refused a physical chair at the last minute.

Ex. Had to rearrange and modify furniture in Unreal engine to match physical space and furniture

Hardware: Is device overheating? Have backup device charged with existing build

If network issues:

Get off Wifi and use Private Network.

Can the experience run locally with wires?

If all else fails

Have offline rendered videos ready

Have access to demo engineers ready to troubleshoot remotely

Be a good handler and a elevator pitch

Result

good things

What makes a good elevator pitch?

Use improv experience to stay positive and always say yes and

Situation

Come up with a elevator pitch that makes the demo look good.

Tasks

Talk to developers what is their mission objective and roadmap

Have a story ready on what problem this could sove

Have metrics to show why the solution is viable

Action

Have you ever setup your MR device and you are asked to type your wifi password? The options are to find a keyboard which we not have lying around, or go through the frustrating experience where we have to do air typing with a virtual keyboard. I’m here to present surface typing, a better way to type in MR. Surface Typing utilizes handtracking and a virtual keyboard projected on a flat surface, such as a table, so you can type on top of it.

In this demo, we have a typing game where you type the paragraph using surface typing and it will determine your words per minute. The average typer is around 40 words per minute on a keyboard, but air typing is at around 15 words per minute. Advanced typists are 100 words per minute.

Lets see how fast you can type!

Result

Majority of users were able to reach the 40 wpm, much faster than airtyping

Individuals that were 6 foot or taller struggled with accuracy.

It could be longer fingers occluding the front camera and letters

Lower resolution of the front camera to virtual keyboard, making tracking harder

People had more fun when you provide metrics to beat. There was no highscrore system but we kept track for best of the day to further add competition. We had users coming back to try to beat their coworkers times!