top of page
SkillSphere MR.png

Innovative mixed reality training to educate field technicians for efficient onboarding


Capstone Project

Task given 
Develop an advanced professional training application that harnesses the potential of immersive technologies and 5G to deliver a more cost effective, engaging, and richer learning experience in scenarios where access to the real physical environment or scenarios are difficult, expensive or dangerous.


Professional Training

My Team

Lauren Liao (Full Stack Developer) 

Yilia Liu (Videographer)

Zhengyang Yang (UX Designer & XR Developer)

My Role

UX Researcher

UIUX Designer

Product Manager



Gradient Background
Onboarding 3.png
Immersive Baseline Training Modules
Immersive MR training modules that replicate real-world scenarios, providing junior technicians with hands-on experience without the need for physical equipment.
Interactive Learning Components
Interactive MR learning components with which new hires can actively engage in practical tasks and simulations.
Home 6.png
Transceiver Module Installation 3.png
Virtual Mentorship in Mixed Reality
Componenets with WiKi guide allowing junior technicians to gain necessary knowledge 
Point -Based Reward System
Motivating users to engage in learning, progress tracking and allowing technicians to earn points upon completing a learning module and redeem for T-Mobile souvenirs.
Desktop - 7.png
What is the problem

We are responsible for break-fix, installation, mainly maintenance work, inspection and repairment for nokia airscale everyday

Telecommunication Field Technicians 

We train junior field technicians for them to work independtly,  however current training is extremely time consuming and expensive... How can we improve it with XR technology? 

Managers/ Trainers 
Mask group (4).png
Nokia AirScale for
network tower
4 - Ordinary Male - Neutral - White.png
3 - Ordinary Female - Neutral - Black.png
In the 6-month product development cycle, I led the team in research, ideation, verification, and development. Over the course of the 6 months dedicated to chasing the project goal, we iterated between stages based on feedback and new insights to ensure an effective and visually appealing solution
Define Goal & Problem

Business Goal

Secondary Research

Primary Research


Competitive Analysis

Scope Stakeholder Need

Experts Interview

User Interview


Affinity Map

Design Solution

Design Solutions, Testings and Iterations

Design System

Usability Testing


Design Handoff


Stakeholder Analysis

Product value proposition
The product can be used by T Mobile Verizon, AT & T, XFinity, Nokia and other telecommunication companies customizable to their training needs. 
Secondary Research

After we conducted secondary research from design, technology, and business perspectives to inform our approach, here are some takeaways: 

Black Metal
Black Metal
Black Metal

Decrease in senior technicians in field service industry 

Employees have limitations learning by tutorial video 


Employers spend 413billion yearly on informal on- the- job training 

skill 2.png
How can we leverage mixed reality to transform the expertise of field technicians, their domain knowledge and workflows into a high-fidelity, scalable training platform? 
skill 2.png
skill 2.png
Primary Research & Findings

User Interviews

I led the team in  a total of 4 user interviews (Operations Manager X1 + Senior Field Technician X1 + Senior RF Engineers  X3) to understand current training process and discuss its strength and weaknesses to identify any pain points. 2 expert interviews (XR Designer X1 + AR/VR Technical Artist X1) and cell site tour for technical possibilities and considerations.


Interview Analysis

We then created an affinity diagram using the quotes from each person. We organized them into categories: Field Technician Responsibilities, Training, AR/VR, Cell Site, RF Engineering, Cost Considerations, and Safety to visually identify patterns and insights across the different user interviews.


4 - Ordinary Male - Neutral - White.png


RF Engineer

Eric is a seasoned RF engineers with over 20 years of experience, dedicated to optimizing T-Mobile's network performance.


  • Broaden understanding of equipment maintenance

  • Enhance collaboration between field operations and engineering teams

Pain Points

  • Equipment training

  • Interdepartmental coordination

3 - Ordinary Female - Neutral - Black.png


Field Technician 

As a new hired field technician at T-Mobile, Olivia aims to ensure seamless network operations through proactive equipment maintenance and rapid response to issues.


  • Be able to work on cell sites independently

  • Improve preventive maintenance strategies

Pain Points

  • Lacks formal baseline training to address skill gaps

  • Training takes too much time and cost

Identified Problems

Junior field technicians still invest hours handling real equipment—rather than following a well-organized vocational training program—new hires struggle to gain hands-on practice. This frustrating slog is caused by 3 common yet critical pain points:



Disengagement in remote video learning is often observed, primarily stemming from a preference for hands-on learning experiences.


Heavy Reliance

The heavy reliance on shadowing senior technicians presents a notable challenge as it consumes a considerable portion of seniors' time.


High Expense

The provision of real equipment incurs substantial expenses, especially when factoring in the need for regular updates, which are often required annually.

Design Requirements

SkillSphere MR addresses the identified problems with specific design requirements tailored to the needs of junior field technicians. With SkillSphere MR, they'll receive a formal vocational training that:


Learn By Doing

Immerse junior field technicians in a variety of realistic simulations designed to replicate the challenges and scenarios they may encounter in their daily work.


Virtual Mentorship

Enable junior technicians to virtually shadow senior technicians in their day-to-day activities, fostering knowledge transfer through real-time guidance.


Low- Cost Expandability

Emphasize cost-effective strategies to ensure that technicians receive up-to-date instruction and the frequent updates to training content and materials.

Enabling Hands-On Practice Anytime, Anywhere

Sketches & Storyboarding

Olivia is a newly hired field technician. She just graduated from college with an EE degree and is passionate about RF engineering. However, when she came onboard at T-Mobile, she had just missed the annual baseline training workshop. Although she wanted to learn on real equipment, she had limited opportunities to be exposed to real scenarios. She could only spend time watching online video tutorials instead of practicing in person.


User Flow & Wireframes

To provide an authentic and effective MR training program for Olivia, I began by selecting a module from their current training tutorial to test the new concept. Previously, Olivia had learned to install components of a 5G baseband through manuals or videos; now, I have transformed such tutorials into MR space, utilizing virtual models and step-by-step instructions.


User Flow- Ground Wire Installation Tutorial 


1. User Instruction

2. Select a tutorial

3. Learn " Ground wire installation" 

4. Complete learning

Wireframes - Ground Wire Installation Tutorial

Early User Testing

Rapid Iterative Testing & Evaluation

With the first MR tutorial prototype, after stakeholder reviews and a pilot user testing with an engineer at T-Mobile, a vital issue arose: the ground wire installation tutorial might be too simple for individuals with an engineering background. Meanwhile, a new direction was suggested: technicians often struggle with installing optical modules if they lack proper training, a tutorial of which is potentially more valuable.


Added User Flow - Optical Module Installation Tutorial


1. User Instruction

2. Select a tutorial

3. Learn " Optical module installation"

4. Complete learning

Added Wireframes - Optical Module Installation Tutorial

More Testings: ThinK Aloud


The goal now is to evaluate which of these two tutorial prototypes is more valuable and to assess the usability of each prototype.

Since these tutorials are intended for individuals without prior hands-on experience, we then recruited 4 engineers from T-Mobile who possessed basic knowledge of base stations but had limited experience performing tasks manually. We set up 4 scenarios and 8 tasks using a Figma prototype, and participants were tasked with completing them either in-person or via Zoom. They were encouraged to think aloud to facilitate our recording of their reactions to the design.



Usability Testing with 5 Engineers at T-Mobile

Prioritizing Issues

To determine the most prioritized issues for improvement, we assigned each issue a priority level from P0 to P3 for severity and scale, with P0 being the highest and P3 the lowest. Based on the average severity and scope priorities, four issues were identified as the most urgent to be solved for an enhanced user experience, as they currently impede user flow and affect the greatest number of people.


Prioritizing Issues: Assessing Severity and Scope

How to Improve Them 

With these two tutorial prototypes, the goal now is to evaluate which one is more valuable and the usability of each prototype.

Method: Since it's a tutorial designed for those who previously don't have hands-on experience, we recruited 5 engineers from T-Mobile who had basic knowledge of base stations but seldom performed tasks on hand. We set up 4 scenarios and 8 tasks using a Figma prototype and asked participants to complete them either in-person or through Zoom. They were encouraged to think aloud so we could better record their reactions to the design.

P0- More Detailed Onboarding Instructions



  • Users don't know what this app is used for.

  • Users don't have an opportunity to practice these gestures.


  • Walk users through all app modules using a step-by-step introduction.

  • Incorporate gestures into real tasks to make the learning more intuitive.

P0 - Enhance Real-World Equipment Matching



  • Users struggle to differentiate between terminals.

  • The absence of color-coding and clear labeling akin to real-world equipment poses challenges.


  • Implement color-coding to align virtual terminals with their real-world counterparts.

  • Label the terminals consistently with the labeling on real equipment.

P1 - Progress Tracking



  • Users don't know what components they have learned and what they haven't.


  • Distinguish the learning progress of each component: Not Started, In Progress, and Completed.

  • Provide an overview of all components users need to learn so that they can recognize rather than recall.

P1 - Interactive Components Checklist



  • Users require a checklist detailing all components comprising a unit.


  • Introduce an interactive components checklist, enhancing users' comprehension of the unit's inner workings.

  • Gestures allow users to dissemble and assemble all components intuitively and safely.

Final Design

New User Onboarding

As most users have limited experience with MR apps, providing clear instructions on navigating within an MR space is crucial. To accomplish this, I have designed detailed onboarding instructions that inform users about the components included in this MR app and the gestures they can use to interact with both 2D and 3D components.


(1/8) Welcome


(3/8) Select the Equipment


(5/8) Palm Movement Interaction


(7/8) View Component Details

661892f07b8435e7503a1bc8_tmo-onboarding-2 (1).jpg

(2/8) Dashboard Introduction


(4/8) Pinch to Pick


(6/8) Palm Movement Interaction


(8/8) Start Learning

Comprehensive Understanding of Base Stations

Unlike the current training methods, which involve viewing a base station through images or videos, this design allows users to explore a 3D base station from any perspective they desire. Interactive labeling of components enables users to reinforce memory retention, and they can even manipulate parts, removing and reinserting them to observe the internal structure of a base station. This level of interaction is impossible in the real world due to safety concerns and the risk of economic losses.


Dashboard Overview


View Baseband Components


Assemble Components Virtually


Place Baseband in MR Space


Dissemble Components Virtually


Getting Started with Component Learning

Step-By-Step Installation Instructions

Muscle Memory through Learning by Doing

Taking optical module installation as the sample tutorial, the interactive training motivates users to learn by doing, enhancing muscle memory through realistic task simulations. The instant feedback and tailored instructions serve as virtual mentorship, saving time compared to shadowing senior technicians in the real world.


Wear Virtual Gloves


Manipulate the Virtual Model


Wear Virtual Gloves


Manipulate the Virtual Model

Muscle Memory through Learning by Doing


Tooltip for Avoiding Errors


Real-Time Error Feedback


Tooltip for Reinforcing Memory


Tooltip for Avoiding Errors


Real-Time Error Feedback


Congratulations upon Completion

Points-Based Reward System

With the points earned from completing learning modules, I designed a points-based reward system to monetize these points. Employees can redeem them for rewards and T-Mobile retail store merchandise, leading to increased employee engagement and stronger team connections.


Browse All T-Mobile Rewards


Preview Physical Rewards in MR


Browse All T-Mobile Rewards


Preview Physical Rewards in MR

Final Product

A new way to train

Quest 3 Headset Deployment

Following the completion of our UX design, we've seamlessly integrated key features onto the Quest 3 headset, where users can visit the learning dashboard, play the base station model, and follow an optical module installation tutorial.

Navigating through Dashboard

Pop-up window explanation

Through the learning dashboard users can browse all equipment that the program supports and select one to learn.

Solution 1(AR).gif

Performing Realistic Tasks

Increase muscle memory on given tasks

Error reduction with cable & port installation practice

Hand Gesture 2.gif


Interactive Learning Components

Familiarize different component, color coding

Zooming in & out & all angles for informational learning

Save user learning process

Hand Gesture 3.gif

Push in

Hand Gesture 4.gif

Push away

Solution 3.gif

Learn Components from All Perspectives

Allow users to view every part of the equipment in detail, which was impossible at real cell sites due to safety and economic concerns.

MR Usability Testing

Now it's time for another round of usability testing! For the developed final product, we recruited 4 field technicians and engineers at T-Mobile, providing them with a Quest3 equipped with our deployed program. The final results strongly demonstrate its value as an effective, cost-saving, and safe training solution.


Usability Testing in MR


Usability Testing in MR

Significantly Reduced Task Completion Time


Completion Time Reduction


Revolutionized Vocational Training

The impact of this project extends beyond just efficiency gains; it has the potential to positively influence employee performance, satisfaction, and ultimately, the company's success in providing quality service to its customers.

Screenshot 2024-03-27 at 3.39.54 PM.png


Decreased Training Cost

Each Nokia AirScale baseband unit is worth $1,499, and typically, a field technician needs to learn 5 or more different versions. However, with our MR training program, a newly hired technician only needs to spend $499 for a Quest 3 to unlock unlimited immersive training tutorials.

Saved Training Time

Screenshot 2024-03-27 at 3.38.36 PM.png


Screenshot 2024-03-27 at 3.36.03 PM.png


Traditional training methods typically require newly hired technicians to shadow senior technicians for 3-6 months, which includes learning and transit time. With SkillSphere MR, however, junior technicians can complete the entire training in approximately 80 hours and receive remote assistance from anywhere, even from their bedroom.

Promoted Long-term Knowledge Retention

Each Nokia AirScale baseband unit is worth $1,499, and typically, a field technician needs to learn 5 or more different versions. However, with our MR training program, a newly hired technician only needs to spend $499 for a Quest 3 to unlock unlimited immersive training tutorials.

How Do T-Mobile Stakeholders Like It?

"It's super cool! Field technicians can't pull out the wire on real sites because things will break, and our service will be down. But with this tutorial, they can learn about these parts in whatever way they like."

"This is exactly what the equipment looks like in the real world! It's amazing that you've created such 3D models that people can interact with. This tutorial is exactly what new engineers need!"

Candice Boyd
Senior Manager, Tech Experience

Vadim Savin
Sr. Engineer, System Design

Collaboration with T-mobile Tech Experience

As a key part of this project, we collaborated closely with the production team at T-Mobile Tech Experience. By improving vocational training through the leverage of 5G technologies, we have demonstrated how this project could potentially contribute to 5G technologies and the prospects of T-Mobile's business.


Ask for usability testing participants as early as possible 

Although verifying the product's usability is vital, the most difficult and time-consuming aspect was recruiting participants. After attempting cold emails, LinkedIn messages, and stakeholder recommendations, the most effective approach appeared to be having stakeholders recommend participants through their networks and encouraging participants to refer more people, making the project more convincing for those potential participants.

Designer's Journey into Technical Implementation

In this project, apart from leading the product design of the team, I also managed to design and technically implement a feature all by myself. Together, based on the deep understanding of a problem, the way of building and shipping the feature became more autonomous. Constantly experimenting with new tools and mediums, it not only widened my array of skills but also allowed me to push what's possible for a designer with curiosity in engineering.

bottom of page