Flow — Christine Jung

About

A controller-free journey through a folklore-inspired world

Flow is an immersive interactive experience that takes guests on a spiritual journey through a folklore-inspired world. Utilizing a 270-degree curved projection screen and multiple body-tracking devices (Azure Kinect), players use full-body gestures to interact with the narrative — no controllers required.

Over 5 months, we conducted 70+ playtest sessions with 100+ guests, continuously iterating on the physical-to-virtual interactions based on user data. My primary contributions spanned technical environment art, custom shader development, and spatial UX design.

Unreal Engine 5 Houdini Custom Shaders Azure Kinect Body Tracking 270° Projection

Demo

Experience Demo

Flow — Full Demo Video

Narrative & Experience Design

The goal was to create an emotionally resonant interactive story where guests intuitively use body gestures to help the main characters — without standard controllers.

Initial rough sketches for narrative pacing — Initial rough sketches to brainstorm narrative pacing and 270° spatial interaction before moving into the engine

Final narrative structure with interest curve

Design Approach

Spatial Partitioning: By dividing the massive 270-degree screen into three distinct interactive zones, we optimized the spatial layout to seamlessly match the pacing of the story and user inputs.

Concept art by Ashling Tu — scenes 1 & 2 — Concept art — Ashling Tu

Concept art by Ashling Tu — scenes 3 & 4 — Concept art — Ashling Tu

Challenge 1 — Lack of Intuitive Affordances

The Problem

Early playtests revealed that guests felt stuck. The system lacked sufficient feedback, leaving players unsure of when and how to interact using their bodies.

Solution 1: Onboarding (Pre-show)

Designed a live tutorial phase where a host guided guests into the environment, teaching them core gestures associated with elemental powers.

Host guiding guests in the pre-show tutorial — Pre-show: Host teaching guests core gestures with the lantern prop

Solution 2: Blinking Outline Shader

To provide non-intrusive UI cues, I developed and implemented a blinking outline shader on interactable objects. Combined with the virtual character's movement hints, this clearly indicated interaction opportunities and completely resolved user confusion.

Blinking outline shader on tree — Blinking outline shader — tree interaction

Blinking outline shader on firepit — Blinking outline shader when interaction is needed

Challenge 2 — Scattered Focus in a 270° Environment

The Problem

The massive scale of the curved screen distracted guests. Players often looked at the wrong sections, missing key narrative moments and failing to perform gestures in time.

Solution: Dynamic Fog as a Spatial UX Tool

I conceptualized and created a Dynamic Fog shader that dynamically controlled the player's Field of View (FOV) — subtly obscuring non-active areas and naturally guiding their attention to the active interaction zone.

Result: Combined with visual affordances, 100% of playtesters successfully navigated the experience without confusion.

Fog blocking areas 2 and 3 so guests focus on area 1 — Fog blocking area 2 and 3 so the guest can focus on area 1

Fog clearing when characters enter area 2 — Fog clearing when the characters enter area 2

Playtesting & Cross-Functional Iteration

Data-Driven Iteration: Co-facilitated 70+ playtest sessions with 100+ guests. Documented qualitative feedback and collaborated closely with engineers to refine the interaction mechanics and narrative pacing.
Hardware & UX Integration: Brainstormed and tested paradigm-shifting inputs with the engineering team — such as attaching a Vive Tracker to a physical lantern, successfully bridging the gap between tactile props and virtual interactions.

Host using lantern and Vive tracker in the pre-show scene — Host using lantern + Vive tracker to light up the Pre-show scene

Physical lantern prop with Vive tracker mounted — Physical lantern with Vive tracker attached

Hand indicator particle for intuitive interaction — Another solution for more intuitive interaction: Hand indicator particle

Programmer testing hand particle — Programmer testing the hand particle system

Motion capture session with CS department — Mocap sessions with help from CS department

Motion capture data applied in Unreal Engine scene — Mocap data applied to character animations in-engine

Immersive World-Building & Tech Art

Environment & VFX: Built highly immersive scenes in UE5 and developed various visual effects (water shaders, weather systems, character VFX) to enhance the emotional storytelling.
Procedural Workflows: Built procedural natural asset generators (mountains, rocks, ivy) using Houdini to rapidly populate the virtual world.
Motion Capture Pipeline: Configured the initial pipeline to import and apply motion capture data into Unreal Engine for character animations.

Virtual Environment Setup

Virtual environments in different seasons — Spring, Summer, Fall, Winter — Environments in different seasons

Final Interaction — Scene 3

Final Interaction Demo — Scene 3

VFX & Shader

Procedural Natural Assets

Procedural mountain generated in Houdini — Mountain

Procedural rocks generated in Houdini — Rocks

Procedural ivy — Houdini node graph — Procedural Ivy — Houdini