Innovative mixed reality training to educate field technicians for efficient onboarding
T- MOBILE X UW
Capstone Project
Task given
Develop an advanced professional training application that harnesses the potential of immersive technologies and 5G to deliver a more cost effective, engaging, and richer learning experience in scenarios where access to the real physical environment or scenarios are difficult, expensive or dangerous.
AR/VR/XR
Professional Training
My Team
Lauren Liao (Full Stack Developer)
Yilia Liu (Videographer)
Zhengyang Yang (UX Designer & XR Developer)
My Role
UX Researcher
UIUX Designer
Product Manager
Timeline
2023.9-2024.3
Outcome
Immersive Baseline Training Modules
Immersive MR training modules that replicate real-world scenarios, providing junior technicians with hands-on experience without the need for physical equipment.
Interactive Learning Components
Interactive MR learning components with which new hires can actively engage in practical tasks and simulations.
Virtual Mentorship in Mixed Reality
Componenets with WiKi guide allowing junior technicians to gain necessary knowledge
Point -Based Reward System
Motivating users to engage in learning, progress tracking and allowing technicians to earn points upon completing a learning module and redeem for T-Mobile souvenirs.
What is the problem
We are responsible for break-fix, installation, mainly maintenance work, inspection and repairment for nokia airscale everyday
Telecommunication Field Technicians
We train junior field technicians for them to work independtly, however current training is extremely time consuming and expensive... How can we improve it with XR technology?
Managers/ Trainers
.png)
Nokia AirScale for
network tower
Process
In the 6-month product development cycle, I led the team in research, ideation, verification, and development. Over the course of the 6 months dedicated to chasing the project goal, we iterated between stages based on feedback and new insights to ensure an effective and visually appealing solution
Define Goal & Problem
Business Goal
Secondary Research
Primary Research
HMW
Question
Competitive Analysis
Scope Stakeholder Need
Experts Interview
User Interview
Persona
Affinity Map
Design Solution
Design Solutions, Testings and Iterations
Design System
Usability Testing
Prototype
Design Handoff
Iterations
Stakeholder Analysis
Product value proposition
The product can be used by T Mobile Verizon, AT & T, XFinity, Nokia and other telecommunication companies customizable to their training needs.
Secondary Research
After we conducted secondary research from design, technology, and business perspectives to inform our approach, here are some takeaways:
Decrease in senior technicians in field service industry
Employees have limitations learning by tutorial video
Employers spend 413billion yearly on informal on- the- job training
How can we leverage mixed reality to transform the expertise of field technicians, their domain knowledge and workflows into a high-fidelity, scalable training platform?
Primary Research & Findings
User Interviews
I led the team in a total of 4 user interviews (Operations Manager X1 + Senior Field Technician X1 + Senior RF Engineers X3) to understand current training process and discuss its strength and weaknesses to identify any pain points. 2 expert interviews (XR Designer X1 + AR/VR Technical Artist X1) and cell site tour for technical possibilities and considerations.
Interview Analysis
We then created an affinity diagram using the quotes from each person. We organized them into categories: Field Technician Responsibilities, Training, AR/VR, Cell Site, RF Engineering, Cost Considerations, and Safety to visually identify patterns and insights across the different user interviews.
Persona
Eric
RF Engineer
Eric is a seasoned RF engineers with over 20 years of experience, dedicated to optimizing T-Mobile's network performance.
Goals
-
Broaden understanding of equipment maintenance
-
Enhance collaboration between field operations and engineering teams
Pain Points
-
Equipment training
-
Interdepartmental coordination
Olivia
Field Technician
As a new hired field technician at T-Mobile, Olivia aims to ensure seamless network operations through proactive equipment maintenance and rapid response to issues.
Goals
-
Be able to work on cell sites independently
-
Improve preventive maintenance strategies
Pain Points
-
Lacks formal baseline training to address skill gaps
-
Training takes too much time and cost
Identified Problems
Junior field technicians still invest hours handling real equipment—rather than following a well-organized vocational training program—new hires struggle to gain hands-on practice. This frustrating slog is caused by 3 common yet critical pain points:
😞
Dis-engagement
Disengagement in remote video learning is often observed, primarily stemming from a preference for hands-on learning experiences.
👬
Heavy Reliance
The heavy reliance on shadowing senior technicians presents a notable challenge as it consumes a considerable portion of seniors' time.
💰
High Expense
The provision of real equipment incurs substantial expenses, especially when factoring in the need for regular updates, which are often required annually.
Design Requirements
SkillSphere MR addresses the identified problems with specific design requirements tailored to the needs of junior field technicians. With SkillSphere MR, they'll receive a formal vocational training that:
🖐️
Learn By Doing
Immerse junior field technicians in a variety of realistic simulations designed to replicate the challenges and scenarios they may encounter in their daily work.
🔗
Virtual Mentorship
Enable junior technicians to virtually shadow senior technicians in their day-to-day activities, fostering knowledge transfer through real-time guidance.
⌛️
Low- Cost Expandability
Emphasize cost-effective strategies to ensure that technicians receive up-to-date instruction and the frequent updates to training content and materials.
Ideation
Enabling Hands-On Practice Anytime, Anywhere
Sketches & Storyboarding
Olivia is a newly hired field technician. She just graduated from college with an EE degree and is passionate about RF engineering. However, when she came onboard at T-Mobile, she had just missed the annual baseline training workshop. Although she wanted to learn on real equipment, she had limited opportunities to be exposed to real scenarios. She could only spend time watching online video tutorials instead of practicing in person.
User Flow & Wireframes
To provide an authentic and effective MR training program for Olivia, I began by selecting a module from their current training tutorial to test the new concept. Previously, Olivia had learned to install components of a 5G baseband through manuals or videos; now, I have transformed such tutorials into MR space, utilizing virtual models and step-by-step instructions.
User Flow- Ground Wire Installation Tutorial
1. User Instruction
2. Select a tutorial
3. Learn " Ground wire installation"
4. Complete learning
Wireframes - Ground Wire Installation Tutorial
Early User Testing
Rapid Iterative Testing & Evaluation
With the first MR tutorial prototype, after stakeholder reviews and a pilot user testing with an engineer at T-Mobile, a vital issue arose: the ground wire installation tutorial might be too simple for individuals with an engineering background. Meanwhile, a new direction was suggested: technicians often struggle with installing optical modules if they lack proper training, a tutorial of which is potentially more valuable.
Added User Flow - Optical Module Installation Tutorial
1. User Instruction
2. Select a tutorial
3. Learn " Optical module installation"
4. Complete learning
Added Wireframes - Optical Module Installation Tutorial
More Testings: ThinK Aloud
Goal
The goal now is to evaluate which of these two tutorial prototypes is more valuable and to assess the usability of each prototype.
Since these tutorials are intended for individuals without prior hands-on experience, we then recruited 4 engineers from T-Mobile who possessed basic knowledge of base stations but had limited experience performing tasks manually. We set up 4 scenarios and 8 tasks using a Figma prototype, and participants were tasked with completing them either in-person or via Zoom. They were encouraged to think aloud to facilitate our recording of their reactions to the design.
Method
Usability Testing with 5 Engineers at T-Mobile
Prioritizing Issues
To determine the most prioritized issues for improvement, we assigned each issue a priority level from P0 to P3 for severity and scale, with P0 being the highest and P3 the lowest. Based on the average severity and scope priorities, four issues were identified as the most urgent to be solved for an enhanced user experience, as they currently impede user flow and affect the greatest number of people.
Prioritizing Issues: Assessing Severity and Scope
How to Improve Them
With these two tutorial prototypes, the goal now is to evaluate which one is more valuable and the usability of each prototype.
Method: Since it's a tutorial designed for those who previously don't have hands-on experience, we recruited 5 engineers from T-Mobile who had basic knowledge of base stations but seldom performed tasks on hand. We set up 4 scenarios and 8 tasks using a Figma prototype and asked participants to complete them either in-person or through Zoom. They were encouraged to think aloud so we could better record their reactions to the design.
P0- More Detailed Onboarding Instructions
Before
-
Users don't know what this app is used for.
-
Users don't have an opportunity to practice these gestures.
After
-
Walk users through all app modules using a step-by-step introduction.
-
Incorporate gestures into real tasks to make the learning more intuitive.
P0 - Enhance Real-World Equipment Matching
Before
-
Users struggle to differentiate between terminals.
-
The absence of color-coding and clear labeling akin to real-world equipment poses challenges.
After
-
Implement color-coding to align virtual terminals with their real-world counterparts.
-
Label the terminals consistently with the labeling on real equipment.
P1 - Progress Tracking
Before
-
Users don't know what components they have learned and what they haven't.
After
-
Distinguish the learning progress of each component: Not Started, In Progress, and Completed.
-
Provide an overview of all components users need to learn so that they can recognize rather than recall.
P1 - Interactive Components Checklist
Before
-
Users require a checklist detailing all components comprising a unit.
After
-
Introduce an interactive components checklist, enhancing users' comprehension of the unit's inner workings.
-
Gestures allow users to dissemble and assemble all components intuitively and safely.
Final Design
New User Onboarding
As most users have limited experience with MR apps, providing clear instructions on navigating within an MR space is crucial. To accomplish this, I have designed detailed onboarding instructions that inform users about the components included in this MR app and the gestures they can use to interact with both 2D and 3D components.
(1/8) Welcome
(3/8) Select the Equipment
(5/8) Palm Movement Interaction
(7/8) View Component Details
.jpg)
(2/8) Dashboard Introduction
(4/8) Pinch to Pick
(6/8) Palm Movement Interaction
(8/8) Start Learning
Comprehensive Understanding of Base Stations
Unlike the current training methods, which involve viewing a base station through images or videos, this design allows users to explore a 3D base station from any perspective they desire. Interactive labeling of components enables users to reinforce memory retention, and they can even manipulate parts, removing and reinserting them to observe the internal structure of a base station. This level of interaction is impossible in the real world due to safety concerns and the risk of economic losses.
Dashboard Overview
View Baseband Components
Assemble Components Virtually
Place Baseband in MR Space
Dissemble Components Virtually
Getting Started with Component Learning
Step-By-Step Installation Instructions
Muscle Memory through Learning by Doing
Taking optical module installation as the sample tutorial, the interactive training motivates users to learn by doing, enhancing muscle memory through realistic task simulations. The instant feedback and tailored instructions serve as virtual mentorship, saving time compared to shadowing senior technicians in the real world.
Wear Virtual Gloves
Manipulate the Virtual Model
Wear Virtual Gloves
Manipulate the Virtual Model
Muscle Memory through Learning by Doing
Tooltip for Avoiding Errors
Real-Time Error Feedback
Tooltip for Reinforcing Memory
Tooltip for Avoiding Errors
Real-Time Error Feedback
Congratulations upon Completion
Points-Based Reward System
With the points earned from completing learning modules, I designed a points-based reward system to monetize these points. Employees can redeem them for rewards and T-Mobile retail store merchandise, leading to increased employee engagement and stronger team connections.
Browse All T-Mobile Rewards
Preview Physical Rewards in MR
Browse All T-Mobile Rewards
Preview Physical Rewards in MR
Final Product
A new way to train
Quest 3 Headset Deployment
Following the completion of our UX design, we've seamlessly integrated key features onto the Quest 3 headset, where users can visit the learning dashboard, play the base station model, and follow an optical module installation tutorial.
Navigating through Dashboard
Pop-up window explanation
Through the learning dashboard users can browse all equipment that the program supports and select one to learn.
Click
Performing Realistic Tasks
Increase muscle memory on given tasks
Error reduction with cable & port installation practice
Pull
Interactive Learning Components
Familiarize different component, color coding
Zooming in & out & all angles for informational learning
Save user learning process
Push in
Push away
Learn Components from All Perspectives
Allow users to view every part of the equipment in detail, which was impossible at real cell sites due to safety and economic concerns.
MR Usability Testing
Now it's time for another round of usability testing! For the developed final product, we recruited 4 field technicians and engineers at T-Mobile, providing them with a Quest3 equipped with our deployed program. The final results strongly demonstrate its value as an effective, cost-saving, and safe training solution.
Usability Testing in MR
Usability Testing in MR
Significantly Reduced Task Completion Time
Completion Time Reduction
Impact
Revolutionized Vocational Training
The impact of this project extends beyond just efficiency gains; it has the potential to positively influence employee performance, satisfaction, and ultimately, the company's success in providing quality service to its customers.
>93%
Decreased Training Cost
Each Nokia AirScale baseband unit is worth $1,499, and typically, a field technician needs to learn 5 or more different versions. However, with our MR training program, a newly hired technician only needs to spend $499 for a Quest 3 to unlock unlimited immersive training tutorials.
Saved Training Time
>77%
75-90%
Traditional training methods typically require newly hired technicians to shadow senior technicians for 3-6 months, which includes learning and transit time. With SkillSphere MR, however, junior technicians can complete the entire training in approximately 80 hours and receive remote assistance from anywhere, even from their bedroom.
Promoted Long-term Knowledge Retention
Each Nokia AirScale baseband unit is worth $1,499, and typically, a field technician needs to learn 5 or more different versions. However, with our MR training program, a newly hired technician only needs to spend $499 for a Quest 3 to unlock unlimited immersive training tutorials.
How Do T-Mobile Stakeholders Like It?
"It's super cool! Field technicians can't pull out the wire on real sites because things will break, and our service will be down. But with this tutorial, they can learn about these parts in whatever way they like."
"This is exactly what the equipment looks like in the real world! It's amazing that you've created such 3D models that people can interact with. This tutorial is exactly what new engineers need!"
Candice Boyd
Senior Manager, Tech Experience
Vadim Savin
Sr. Engineer, System Design
