Transparent AR Markers

Tags: Augmented Reality Tangible Interaction Compositional Input

Date: May 2023 - Oct. 2023

Collaborator: Max Chen

A novel marker-based AR system that uses stacked transparent fiducial markers as a tangible, compositional input method, demonstrated through Stackable Music, an AR music synthesis game.

Website Article Presentation

PROJECT INTRODUCTION

We explored the interaction affordances of transparent PVC film as a material for tangible augmented reality. Stack AR introduces an alternative input method for marker-based AR: stacking transparent fiducial markers so that the system can recognize not only individual markers, but also the combinations produced by layered markers. Unlike conventional AR interactions that rely on single opaque markers or touchscreen input, Stack AR treats physical stacking as a compositional interaction. We developed Stackable Music, an AR music synthesis game, to demonstrate how this input method can be used in a playful interactive system.

This work was published in the Companion Proceedings of the Annual Symposium on Computer-Human Interaction in Play (CHI PLAY Companion 2023). It was later selected for presentation at MIT Reality Hack 2025 (Jan. 23–27) as part of an art grant. The system can be set up with a desktop computer, a camera, and printed transparent markers.

Presenting Stack AR at MIT (Massachusetts Institute of Technology) Reality Hack 2025

Technical Highlights

Custom ArUco Marker Dictionary: We created a custom ArUco dictionary to support marker stacking on transparent film. The marker design minimizes visual interference between layers and helps ensure that stacked combinations produce unique, detectable codes. Marker regions are arranged in concentric rings, allowing the system to distinguish layers more clearly when multiple transparent markers are placed on top of one another.

Computer Vision Integration in Unity: The Unity application uses the OpenCV for Unity plugin to support real-time marker recognition and pose estimation from a top-down camera. Custom logic parses the detected markers, identifies their layered combinations, and maps those combinations to distinct musical and visual elements in the AR environment.

Detection and Interactive Synthesis: By precomputing viable marker combinations, Stack AR can identify both individual and stacked markers within a single frame. Detected combinations trigger corresponding audio, 3D models, and particle effects in Stackable Music. The system is designed to remain usable under partial occlusion, variable lighting, and the visual ambiguity introduced by transparent materials.

Stack AR Demo Video

Stack AR Presentation Video