Project Overview

Sphere Drop: Gesture-Driven Shared Memory
Sphere Drop: Gesture-Driven Shared Memory
Sphere Drop: Gesture-Driven Shared Memory
Engineering a gesture-driven spatial interface that transforms digital photography into a shared ritual.
Engineering a gesture-driven spatial interface that transforms digital photography into a shared ritual.

[Role]

Interaction Designer (Solo)

[Duration]

4 Weeks, 2025

[Tech Stack]

React Three Fiber, WebGL, Ultraleap, WebSockets, Firebase, Claude, Google AI Studio

Playful Computing

Generative UI

AI-Assisted Dev

The Challenge & Logic

Execution & Craft

Impact & Takeaway

The Challenge & Logic

Engineering a Zero-UI Memory Ritual

Photography is typically a solitary, screen-bound action. My primary challenge was transforming this flat, transactional click into a highly engaging, shared physical ritual. I wanted users to capture moments not through traditional UI buttons, but by instinctively reaching into the air.

To achieve this Zero-UI approach, I designed a gesture-driven logic system. By eliminating explicit text instructions and relying entirely on natural body language, the interface significantly lowers the barrier to entry, successfully transforming isolated interactions into moments of collective play.

Translating human gestures into realistic spatial physics required complex vector mathematics. To bridge this technical gap within a strict timeframe, I leveraged Google AI Studio and Claude as technical partners to rapidly prototype and iterate the custom WebGL physics engine.

The system logic operates on a precise interactive loop: open palms are detected, closing the hand crystallizes the live feed, and subsequent gestures trigger immediate collisions. This AI-assisted architecture ensures the digital memory spheres react to human input with perfectly grounded weight.

Execution & Craft

A Real-Time Spatial Physics Pipeline

The visual core is a React Three Fiber application utilizing custom WebGL shaders. I engineered a real-time data pipeline where Ultraleap hand-tracking data streams via local WebSockets, ensuring ultra-low latency. Captured memories instantly upload to Firebase for seamless mobile sharing.

I designed the memory spheres using a 'Glassmorphism' visual language. Sitting conceptually between transparent glass and fragile soap bubbles, these custom shaders refract faces and physical spaces, granting a beautifully analog, tactile quality to what is a purely digital asset.

As hundreds of spheres accumulate, they form an organic, unpredictable cluster of generative art. This overlapping arrangement serves as a visual metaphor for a collective archive, built entirely by shared participation and reflecting the joyful, chaotic energy of the space.

To balance this poetic interaction with practical usability, I implemented a secondary 'Grid Gallery' view. This structured retrieval mode completely eliminates the friction of searching through dynamic physics, allowing users to effortlessly locate, select, and download their specific captured memories.

Execution & Craft

A Real-Time Spatial Physics Pipeline

The visual core is a React Three Fiber application utilizing custom WebGL shaders. I engineered a real-time data pipeline where Ultraleap hand-tracking data streams via local WebSockets, ensuring ultra-low latency. Captured memories instantly upload to Firebase for seamless mobile sharing.

I designed the memory spheres using a 'Glassmorphism' visual language. Sitting conceptually between transparent glass and fragile soap bubbles, these custom shaders refract faces and physical spaces, granting a beautifully analog, tactile quality to what is a purely digital asset.

As hundreds of spheres accumulate, they form an organic, unpredictable cluster of generative art. This overlapping arrangement serves as a visual metaphor for a collective archive, built entirely by shared participation and reflecting the joyful, chaotic energy of the space.

To balance this poetic interaction with practical usability, I implemented a secondary 'Grid Gallery' view. This structured retrieval mode completely eliminates the friction of searching through dynamic physics, allowing users to effortlessly locate, select, and download their specific captured memories.

Impact & Takeaway

Validating Playful Computing

During the exhibition, the installation captured over 300 photos, with visitors instinctively collaborating to bat and bounce each other's memories. It proved that by stripping away explicit UIs, technology stops isolating users and becomes a powerful catalyst for social connection.

Sphere Drop validates "Playful Computing" by proving joy is a measurable metric. Furthermore, it demonstrated how utilizing LLM-assisted coding empowers solo designers to build, iterate, and deploy highly complex spatial computing systems within an extremely aggressive four-week timeframe.

Validating Playful Computing

Impact & Takeaway

During the exhibition, the installation captured over 300 photos, with visitors instinctively collaborating to bat and bounce each other's memories. It proved that by stripping away explicit UIs, technology stops isolating users and becomes a powerful catalyst for social connection.

Sphere Drop validates "Playful Computing" by proving joy is a measurable metric. Furthermore, it demonstrated how utilizing LLM-assisted coding empowers solo designers to build, iterate, and deploy highly complex spatial computing systems within an extremely aggressive four-week timeframe.

All Works