How to build immersive augmented reality experiences using Unity and the Microsoft Hololens

Abhishek Singh, a resident researcher at NYU, explores the technical and design hurdles of creating augmented reality projects. This talk examines his process for developing interactive experiences, including his viral project that brought the first-person perspective of Super Mario Bros into a real-world, life-sized environment using the Hololens.

Abhishek Singh leverages a diverse background in fabrication, programming, and 3D modeling to bridge the gap between physical and digital worlds. His work spans from creating robotic desktop assistants that communicate via GIFs to developing interactive installations for Times Square. In this session, he details the specific challenges of using Unity to build for the Hololens, emphasizing the design considerations necessary to make virtual elements feel grounded in a physical space.

The talk highlights his experience recreating classic gaming environments as life-sized, first-person AR experiences. By combining his skills in circuit design and software development, Singh demonstrates how creators can push the boundaries of current AR hardware. His approach underscores the importance of integrating varied technical disciplines—from backend programming to 3D printing—to craft compelling, immersive projects that move beyond simple screen-based interactions.

Source: Building Augmented Reality Experiences with Unity3D with Abhishek Singh, Resident Researcher at NYU