Perspective Tracking in Triple Screens CAVE

By Raven Kwok / March 25, 2015 / Application, Dynamic Perspective, Installation, Interactive, Works / 8 Comments

Created between Mar.2015 and Apr.2015

As continuation of a series of one-screen perspective tracking experiments (Portal 2.0, Contra Base 1, Doge Chorus, etc.) I did in late 2014, Shawn Lawson and I changed the lab layout into a three-screens box CAVE environment late last month. This video documentation covers multiple new demos I’ve developed for the new environment in the past couple of weeks.

针对中国用户，若Vimeo 视频无法显示，可以观看下方MANA版本。

Each screen is attached perpendicularly to its neighbor(s), aligned with its corresponding rear projection. Four Vicon Bonita cameras on top corners cover the central area surrounded by the screens. The origin point locates at center of the central area.

Data collected by Vicon is sent through VRPN (Virtual-Reality Peripheral Network), then OSC (Open Sound Control) protocol to Processing, and is mapped as frustum parameters to compute three projection matrices, which coincide with the screens’ position in reality.

Demo 2: Pikachu Quadtree is a rehash of my quick sketch Pikaworm in 2013.

The real-time sound effect in Demo 4: Lightsaber is created through the overlap of a sine wave and a sawtooth wave generated using Beads.

The floating face in Demo 5: Zordon is a 3D reconstruction based on the oni file data recorded using Kinect and SimpleOpenNI.

8 Comments

  1. Tung
    May 13, 2015 at 4:30 pm
    Reply

    Wonderful!!!!

  2. Oscar
    August 25, 2015 at 10:44 pm
    Reply

    Truly amazing.

    What did you used for the virtual ambiences? Unity? or straight OpenGL?

    I’d love to hear more on the technicals !

    PD: I see you got SimpleOpenNI

    • Raven Kwok
      September 30, 2015 at 4:29 am
      Reply

      Thanks Oscar 🙂

      The demos were built using Processing (OpenGL es2). SimpleOpenNI was merely used for recording the depth and image data into an oni file for playback.

      Regarding the matrices computing, maybe this paper could be of help. http://csc.lsu.edu/~kooima/pdfs/gen-perspective.pdf

  3. Madeline
    October 13, 2015 at 2:33 am
    Reply

    Fantastic demos!

    What did you use to go from VRPN to OSC?

    • Raven Kwok
      December 2, 2015 at 10:44 am
      Reply

      Thx. I believe it was Panda3D (built-in VRPN Client) with an OSC python module.

  4. Pingback: MEETING3 – Environments in (e)motion

  5. John
    February 24, 2016 at 8:59 pm
    Reply

    Great demos! Are any of your demos available for use? Or how would you create a similar demo using Unity 3D to float an image and show depth?

  6. Charlie Baby
    January 30, 2019 at 3:35 pm
    Reply

    Oh raven… you are so wonderful and kind. The link on the matrices info….I could hug you!!. thank you so much. beautiful work!

Leave a Reply

This site uses Akismet to reduce spam. Learn how your comment data is processed.