Perspective Tracking in Triple Screens CAVE

Created between Mar.2015 and Apr.2015

As continuation of a series of one-screen perspective tracking experiments (Portal 2.0Contra Base 1Doge Chorus, etc.) I did in late 2014, Shawn Lawson and I changed the lab layout into a three-screens box CAVE environment late last month. This video documentation covers multiple new demos I’ve developed for the new environment in the past couple of weeks.

针对中国用户,若Vimeo 视频无法显示,可以观看下方MANA版本

Each screen is attached perpendicularly to its neighbor(s), aligned with its corresponding rear projection. Four Vicon Bonita cameras on top corners cover the central area surrounded by the screens. The origin point locates at center of the central area.

Data collected by Vicon is sent through VRPN (Virtual-Reality Peripheral Network), then OSC (Open Sound Control) protocol to Processing, and is mapped as frustum parameters to compute three projection matrices, which coincide with the screens’ position in reality.

Demo 2: Pikachu Quadtree is a rehash of my quick sketch Pikaworm in 2013.

The real-time sound effect in Demo 4: Lightsaber is created through the overlap of a sine wave and a sawtooth wave generated using Beads.

The floating face in Demo 5: Zordon is a 3D reconstruction based on the oni file data recorded using Kinect and SimpleOpenNI.


  1. Wonderful!!!!

  2. Truly amazing.

    What did you used for the virtual ambiences? Unity? or straight OpenGL?

    I’d love to hear more on the technicals !

    PD: I see you got SimpleOpenNI

    • Thanks Oscar 🙂

      The demos were built using Processing (OpenGL es2). SimpleOpenNI was merely used for recording the depth and image data into an oni file for playback.

      Regarding the matrices computing, maybe this paper could be of help.

  3. Fantastic demos!

    What did you use to go from VRPN to OSC?

    • Thx. I believe it was Panda3D (built-in VRPN Client) with an OSC python module.

  4. Pingback: MEETING3 – Environments in (e)motion

  5. John

    Great demos! Are any of your demos available for use? Or how would you create a similar demo using Unity 3D to float an image and show depth?

  6. Charlie Baby

    Oh raven… you are so wonderful and kind. The link on the matrices info….I could hug you!!. thank you so much. beautiful work!

Leave a Reply

This site uses Akismet to reduce spam. Learn how your comment data is processed.