麻豆社madou
麻豆社madou Centre for Interactive Cinema Research

Conversations @the Studio
Edols & Elliot Glass Studio Brookvale. Spherical view

Conversations@the Studio聽presents an innovative mixed-reality narrative model of the Powerhouse Museum (Sydney)鈥 Decorative Arts collection within a uniquely designed Intelligent Interactive Visualisation Environment. It takes the natural navigation of a real-world situation 鈥 a contemporary Glass Studio 鈥 as its point of departure, using this as a framework for organising a set of narrative formations that further elaborate the thematics of the collection. By visualising a 360 degree global video recording, made on location at the Glass Studio, it provides the tele-present experience of an actual visit to the Studio, giving full interactive freedom to the viewer鈥檚 gaze.
Installation at the Powerhouse Museum

This project aims to dramatically enhance the presentational flexibility of museum information delivery through the application of an immersive system of cinematic visualisation, modelling the integration of three kinds of interactive narrative. By allocating selective agency to both objects displayed and to viewers within a virtual environment, the system allows museum visitors to invest even static artefacts with a range of vivid narrative purpose. This project has been adjusted to accommodate the fact that only 50% of the requested聽ARC聽budget was awarded. This has necessitated a concentration on the experimental demonstrator for the Powerhouse Museum and web site evaluating and documenting the study. It has also necessitated a simplification of the original model whereby the presence of narrative agents within the聽IIIE聽has been replaced by an interaction strategy where the multilayered narrative extensions are embedded within the global video recording. This is articulated by multiple audio streams, and by conjunctions between the global scene and close-up sequences that detail the glass handling process.
Space-Time relation of close ups in the 360潞 global recording

Conversations@the Studio聽sets a new museological multi-media benchmark. Its interactive and immersive frameworks allow the viewer to experience themselves present in a distant space where artists are engaged in their daily creative craft making activities. They can move their gaze freely anywhere in the live performance space whose unedited real time narration converges with the real time engagement of the viewer. This space-time continuum between the local (Powerhouse Museum) and remote (glass studio) location enables the viewer to achieve an immediacy of experience that has hitherto not been achievable in multi-media exhibits. This is reinforced by the dynamic surround sound reproduction of the glass studio environment that is sensitive to the virtual position of the viewer鈥檚 gaze. A key feature of聽Conversations@the studio聽is the dynamic tracking of the viewer鈥檚 gaze in both space and time that visually magnifies specific events of interest. This polychronic cinematic operability gives the viewer a multi-visual faculty that moves seamlessly between an apprehension of the total active space and the amplification of its significant details. The movement is further emphasised by the viewer鈥檚 freedom to move between the live ambient surround sound recording made in that space, and the focussed commentaries of the glass studio artists which refer to specific processes and events that are unfolding.

Project Development

The first stage of research in 2004 involved: The analysis of the core components to be integrated from the experimental models; the planning and design of the聽IIIE; the 360 degree global recording; testing with sample audio and video; design of the interface system; and post-production of the 360 degree video and audio recording. The second stage of research involved the completion of all the implementation tasks and the installation of the demonstrator, in 2005, at the Powerhouse Museum as part of the museum鈥檚 new permanent decorative arts wing for an exhibition entitled Inspired! design across time. The final stage of the research ensued from the evaluation of public user interaction with the installation at the Powerhouse Museum, and adjustments of the interaction design to facilitate improved functionality.
Edols & Elliot Glass Studio Brookvale. Equirectangular view

The highly successful installation at the Powerhouse takes the form of a custom designed three-meter diameter hemispherical fibreglass projection screen that is mounted vertically so that viewers look straight ahead into its immersive concave projection space. A high-resolution projector fitted with a unique 360 degree fisheye lens is mounted in front of the screen, and the projection is augmented by a 5 channel surround sound system. The specifically developed user interface is an over-sized illuminated track ball that allows navigation of the global video recording, and which changes colour in dynamic relation to the content displayed on the screen. This latter functionality is adapted to signalling the availability of close-up sequences within the global scene. Additional controls activate the voice-over commentary and offer a fast forward function. The聽IIIE聽forms the central digital focus of the Powerhouse Museum鈥檚 new permanent gallery showcasing artefacts from its Decorative Arts and Design collection. The聽IIIE聽utilizes an integrated design strategy that is driven by the specific nature of the Museum鈥檚 decorative arts content, enriching the content by offering multiple layers of multimedia production that augment methods of documentation using 360 degree global recording. Innovative visualisation and interface strategies incorporate these recordings in an environment where the viewer can encounter and explore the content in a dynamic manner.
Conversations@the Studio installation model

Technology

  • Conversations@the Studio聽utilises a 3m dome as the surface for 180 degree projection made possible by a projector equipped with a fish eye lens. The lens is close to the centre of the sphere to ensure total coverage of the screen with minimal distortion. The size and shape of the projection set-up covers the peripheral vision of the user standing directly in front of it and thus ensures a truly immersive experience.
    Installation 3D model

  • The video for聽Conversations@the Studio聽was shot with a Ladybug camera system from Point Grey Research. It allows digital spherical recordings at 360 degree horizontal and 240 degree vertical field of view. The camera has a tightly packed cluster of 6聽CCD聽sensors with wide angle lenses and a slight overlap between the images. In post production the individual frames were colour and geometrically corrected and stitched to a high resolution equirectangular image (3600脳1800 pixel). Custom algorithms where used to double the frame rate from 15 (the camera鈥檚 maximal frame rate) to 30 frames per second. The final delivery format of this 30 min movie is聽MPEG-2 at a very high bit-rate to allow for high quality playback.

  • The sound was recorded using a 4 channel Sound-Field microphone accompanied by a 5.1 Holophone. The individual directional sound channels are mapped to a point of view within the video and synchronised for playback in post production.

  • A custom built 3D software engine takes the high resolution video stream from a disk array and uses it as a texture map projected on the inside of a sphere. The geometric correction for the fisheye lens is undertaken through a high resolution distortion mesh in the render pipeline.

    The video texture is translated inside the sphere according to the point of view of the user, this simulates 鈥渓ooking around鈥 with in the scene.

    A look up table supplies the playback engine with location and time/duration of a video close-ups or hot-spots. If the point of view is within a hot-spot area in time, a close up is superimposed as a second layer on top of the main movie.
    Distrorted video texture
  • Integrated projector stand

    The spherical movie is projected inside the dome with a coverage of 180 degree. A track ball integrated in the projector stand allows the user to rotate the projection freely while the movie is still running. Together with the image the multi-channel sound field is rotated accordingly. A simple vector based panning algorithm is used to distribute the multi-channel sound direction dependent on the user鈥檚 principle point of view.
    A control button activates a voice over commentary channel, whose content is related to the point of view and the time within the main spherical movie. This voice channel is mixed into the centre channel of the directional ambient audio. A second control button allows fast forward play back of the movie.

础搁颁听滨苍惫别蝉迟颈驳补迟辞谤蝉:听Dennis Del Favero, Jeffrey Shaw,听,听, Nikos Papastergiadis, Scott McQuire, Andy Arthurs, Sarah Kenderdine, Kevin Sumption, Grace Cochrane
Programmer: Joachim Tesch
Project Funding:听ARC聽LP0453638, Australia Council for the Arts, Powerhouse Museum
2005-2012
Industry Partner: Museum of Applied Arts & Sciences (MAAS), Sydney
  • Australasian Planetarium Society Conference, Horizon Planetarium, Perth, 2008
  • Darling River Journey, Back of Bourke Exhibition Centre, Bourke, 2005-2007
  • Inspired! Design across time, Powerhouse Museum, Sydney, 2005-2012

Directors:聽Jeffrey Shaw, Andy Arthurs, Neil Brown, Grace Cochrane, Dennis Del Favero, Volker Kuchelmeister, Scott McQuire, Nikos Papastergiadis, Sarah Kenderdine, Kevin Sumption

Glass Artists:聽Alexandra Chambers, Benjamin Edols, Kathy Elliott, Tom Rowney

Production and Technical Co-Ordinator:聽Volker Kuchelmeister

Lead Software Engineer:聽Joachim Tesch

Production Manager:聽Damian Leonard

Project Manager:聽Kate Dennis

Human Interface Designers:聽Volker Kuchelmeister, Jeffrey Shaw

Camera Operator:聽Greg Ferris

Lighting Designer:聽Max Harrison

Production Assistant:聽Sue Midgley

Spherical Video, Close-ups and Voice Commentaries:聽Volker Kuchelmeister

Sound Field Design:聽Andy Arthurs

This project has been funded by the Commonwealth Government through the Australian Research Council鈥檚 research funding body and the Australia Council鈥檚 arts funding and advisory body.

The project has been co-produced by the聽鲍狈厂奥听iCinema Centre for Interactive Cinema Research, The University of Melbourne, Queensland University of Technology and the Powerhouse Museum, Sydney.

The producers wish to thank the following for their generous support in the realization of the project:

Paul Bourke

Edols & Elliott Glass Studio

NoseCone Australia P.L

Videocraft Equipment Pty Ltd

Xenon Systems Pty Ltd

Sound on Stage Pty Ltd

Brendan Lloyd

Powerhouse Museum Staff

Integral Event Management

Funded by:
Australian Research Council Linkage Project: Reformulating museological narrative using three models of cinematic interactivity.