workkate
ARena is a multimodal system using augmented reality to improve athlete coaching and training.
Product Design
Motion Design
3D Modeling
Design Research
ARena improves communication of football gameplay strategies using Microsoft HoloLens. Players and coaches experience game reels and simulations through their own point of view, view stats on demand, and drop plays into a physical space.
SCOPE
Augmented reality has been a game-changer for interaction design due to its 3-dimensional, immersive, yet realistic nature.

The scope was to enhance and explore any topic of choice as a holographic interaction experience for Microsoft HoloLens, with a focus on producing a high-fidelity AR prototype.
DESIGN CONCEPT
We recognized athletic training as a potential use case for AR because it is crucial for these interactions to be immersive without disrupting or disconnecting from the real world. From our research analysis, the majority of AR experiences in sports emphasized the viewing experience. We thought, why not consider the athlete's point of view?

HoloLens’ capabilities to track movement and recognize gestures are suitable for enhancing the physical activity experience. Additionally, multi-angle and speed control capabilities have been essential to the viewing experience and would translate well to athletic training.

The result was our guiding question: how might we leverage augmented reality to help athletes practice gameplays more effectively?


COACH
Introducing ARena: a multimodal application suite for sports strategy communication and simulation using Augmented Reality and Over-the-Air deployment. The coach will be using a tablet.
The coach can add, edit, and ultimately choose which strategy (play) to be deployed to the athlete's HoloLens for practice using a flick up motion.
ARena also collects individual athlete's practice statistics (completion time, accuracy, drills completed) for easy viewing, managing and comparison.
ATHLETE
The athlete will be using a HoloLens. Play visualization is procedurally generated and anchored in space by the athlete's headset. The athlete can clearly visualize the intended plays, follow the highlighted trail to practice movement path, and observe virtual teammates' positions.
Pause
Resume
Rewind/fast forward
Using HoloLens' gesture recognition, the athlete can fully control the experience to better comprehend the strategy.
MULTIMODALITY
From our secondary research, we decided that a multimodal experience would be a good fit for ARena because of its distinct groups of stakeholders (coach and athlete) with varied level of exposure to technology and involvement in the physical activities. Inspired by Apple’s AirDrop, we designed the over-the-air deployment concept for seamless interactions between the coach and athlete.
INTERACTION FLOW
We moved forward with fleshing out an interaction flow to identify necessary screen or scene designs.
Interaction flow by Hy Nguyen
PROTOTYPING IN UNITY
Our initial introduction to AR prototyping started with building the scene in Unity. Using simple 3D geometries and standard assets pack, we built out the initial setup for our experience.

To calibrate object and UI positioning in 3D space, a first-person camera was implemented for rapid prototyping before porting to HoloLens. Our initial athlete's UI implementation featured a static fixed-on-screen display for play information.
SHIPPING TO HOLOLENS
Shipping to the HoloLens enabled us to fully experience the initial build in augmented reality. But more importantly, the HoloLens build also uncovered various usability issues and technical constraints that needed addressing:

Constraint 1: Eye-level and fixed-on-glass interfaces were obstructive to training activities.

Constraint 2: Due to studio policy, we could not take the HoloLens outdoor for further testing.

Constraint 3: Developing mainly on MacOS, our workflow was substantially affected by the HoloLens’ Windows environment requirement.
PORTING TO IPHONE
As a workaround to technical and policy constraints, we ported the prototype to the iPhone using ARKit and XCode. This final prototype enabled us to experience true augmented reality on the field, but without gesture recognition capabilities. However, making this trade-off allowed us to move quickly and focus on the main experience.
FUTURE STEPS
+ TAKEAWAYS
The same model of strategy communication and visualization in AR has potentials to be implemented in other team and individual sports. Additionally, an essential part of team practice is, well... Teamwork. In the future development of ARena, we must consider the possible athlete-to-athlete and coach-to-coach interactions.

The tradeoff of designing for the forefront of developing technology is working through many technical difficulties. ARena was designed for a near future when AR technology is more refined and portable. We had to intentionally make assumptions about the technology to complete the project in time. In retrospect, we would have dedicated more time to fully understanding the technical limitations before starting.

Prototyping for a 3D space uncovered many factors that I hadn't considered while designing 2D scenes, and made me appreciate the difficulties that it presents.